2024-03-23T14:43:01.959 INFO:root:teuthology version: 0.0.1.dev262+ge691533 2024-03-23T14:43:01.960 DEBUG:teuthology.run:Teuthology command: teuthology --name pdonnell-2024-03-23_09:56:53-fs-wip-batrick-testing-20240323.003144-squid-distro-default-smithi --archive /home/teuthworker/archive/pdonnell-2024-03-23_09:56:53-fs-wip-batrick-testing-20240323.003144-squid-distro-default-smithi/7618407 --owner scheduled_pdonnell@teuthology --description fs/shell/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1-mds-1-client-coloc conf/{client mds mgr mon osd} distro/ubuntu_latest mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/cephfs-shell} --verbose -- /home/teuthworker/archive/pdonnell-2024-03-23_09:56:53-fs-wip-batrick-testing-20240323.003144-squid-distro-default-smithi/7618407/orig.config.yaml 2024-03-23T14:43:02.004 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-23T14:43:02.091 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2024-03-23_09:56:53-fs-wip-batrick-testing-20240323.003144-squid-distro-default-smithi/7618407 branch: wip-batrick-testing-20240323.003144-squid description: fs/shell/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1-mds-1-client-coloc conf/{client mds mgr mon osd} distro/ubuntu_latest mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/cephfs-shell} email: pdonnell@redhat.com first_in_suite: false job_id: '7618407' kernel: kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: pdonnell-2024-03-23_09:56:53-fs-wip-batrick-testing-20240323.003144-squid-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: wip-batrick-testing-20240323.003144-squid ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability sha1: 21b3bd672163092b4e542f772797fcba02d3d4de ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 21b3bd672163092b4e542f772797fcba02d3d4de thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-batrick-testing-20240323.003144-squid sha1: d39ab7bb1f1162ec7b83b4f13a6bb876d4baf21b owner: scheduled_pdonnell@teuthology priority: 85 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - - mon.b - mon.c - mgr.x - mds.b - osd.4 - osd.5 - osd.6 - osd.7 seed: 4284 sha1: 21b3bd672163092b4e542f772797fcba02d3d4de sleep_before_teardown: 0 subset: 405/512 suite: fs suite_branch: wip-batrick-testing-20240323.003144-squid suite_path: /home/teuthworker/src/github.com_batrick_ceph_d39ab7bb1f1162ec7b83b4f13a6bb876d4baf21b/qa suite_relpath: qa suite_repo: https://github.com/batrick/ceph.git suite_sha1: d39ab7bb1f1162ec7b83b4f13a6bb876d4baf21b syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi164.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBG8pFKkZbSowF13bo8/baC7ODVq9vuxuXd51QOBTdiI3/0A2eM6xZaLeP6WOiDQ2yG3mqamUVAmozkvqaQGo+hw= smithi173.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDYERFStNwl9aEPr/ypxY64Yl00iN9HbehUIiBGulOVP8YqeAQM6Ms8/NjAJgIZLbg0ZS+tGZV+5TDftGxwMUgA= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - sequential: - mgrmodules - ceph-fuse: null - cephfs_test_runner: modules: - tasks.cephfs.test_cephfs_shell teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: e691533f9cbb33d85b2187bba20d7102f098636d timestamp: 2024-03-23_09:56:53 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-03-23T14:43:02.091 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_batrick_ceph_d39ab7bb1f1162ec7b83b4f13a6bb876d4baf21b/qa; will attempt to use it 2024-03-23T14:43:02.091 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_batrick_ceph_d39ab7bb1f1162ec7b83b4f13a6bb876d4baf21b/qa/tasks 2024-03-23T14:43:02.092 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-03-23T14:43:02.093 INFO:teuthology.task.internal:Checking packages... 2024-03-23T14:43:02.117 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash '21b3bd672163092b4e542f772797fcba02d3d4de' 2024-03-23T14:43:02.117 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-23T14:43:02.117 INFO:teuthology.packaging:ref: None 2024-03-23T14:43:02.117 INFO:teuthology.packaging:tag: None 2024-03-23T14:43:02.117 INFO:teuthology.packaging:branch: wip-batrick-testing-20240323.003144-squid 2024-03-23T14:43:02.117 INFO:teuthology.packaging:sha1: 21b3bd672163092b4e542f772797fcba02d3d4de 2024-03-23T14:43:02.118 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=wip-batrick-testing-20240323.003144-squid 2024-03-23T14:43:02.339 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-1855-g21b3bd67-1jammy 2024-03-23T14:43:02.340 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-03-23T14:43:02.346 INFO:teuthology.task.internal:no buildpackages task found 2024-03-23T14:43:02.346 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-03-23T14:43:02.384 INFO:teuthology.task.internal:Saving configuration 2024-03-23T14:43:02.401 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-03-23T14:43:02.407 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-23T14:43:02.431 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi164.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-03-23_09:56:53-fs-wip-batrick-testing-20240323.003144-squid-distro-default-smithi/7618407', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-23 14:37:20.370993', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAINw7uIJbRvw65iIO2X13VusjUBH0HZLGARZrA1LW07Wo'} 2024-03-23T14:43:02.460 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi173.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-03-23_09:56:53-fs-wip-batrick-testing-20240323.003144-squid-distro-default-smithi/7618407', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-23 14:37:20.369037', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIODRY0BGKzrJQaXuoW/V+v2n1v83nWvNqlGI1hZgeM0g'} 2024-03-23T14:43:02.460 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-03-23T14:43:02.467 INFO:teuthology.task.internal:roles: ubuntu@smithi164.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'] 2024-03-23T14:43:02.467 INFO:teuthology.task.internal:roles: ubuntu@smithi173.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'osd.4', 'osd.5', 'osd.6', 'osd.7'] 2024-03-23T14:43:02.467 INFO:teuthology.run_tasks:Running task console_log... 2024-03-23T14:43:02.538 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f6582c3b3a0>, signals=[15]) 2024-03-23T14:43:02.538 INFO:teuthology.run_tasks:Running task internal.connect... 2024-03-23T14:43:02.544 INFO:teuthology.task.internal:Opening connections... 2024-03-23T14:43:02.544 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi164.front.sepia.ceph.com 2024-03-23T14:43:02.546 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-23T14:43:02.630 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi173.front.sepia.ceph.com 2024-03-23T14:43:02.631 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi173.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-23T14:43:02.712 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-03-23T14:43:02.720 DEBUG:teuthology.orchestra.run.smithi164:> uname -m 2024-03-23T14:43:02.727 INFO:teuthology.orchestra.run.smithi164.stdout:x86_64 2024-03-23T14:43:02.727 DEBUG:teuthology.orchestra.run.smithi164:> cat /etc/os-release 2024-03-23T14:43:02.773 INFO:teuthology.orchestra.run.smithi164.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-03-23T14:43:02.773 INFO:teuthology.orchestra.run.smithi164.stdout:NAME="Ubuntu" 2024-03-23T14:43:02.773 INFO:teuthology.orchestra.run.smithi164.stdout:VERSION_ID="22.04" 2024-03-23T14:43:02.774 INFO:teuthology.orchestra.run.smithi164.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-03-23T14:43:02.774 INFO:teuthology.orchestra.run.smithi164.stdout:VERSION_CODENAME=jammy 2024-03-23T14:43:02.774 INFO:teuthology.orchestra.run.smithi164.stdout:ID=ubuntu 2024-03-23T14:43:02.774 INFO:teuthology.orchestra.run.smithi164.stdout:ID_LIKE=debian 2024-03-23T14:43:02.774 INFO:teuthology.orchestra.run.smithi164.stdout:HOME_URL="https://www.ubuntu.com/" 2024-03-23T14:43:02.774 INFO:teuthology.orchestra.run.smithi164.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-03-23T14:43:02.775 INFO:teuthology.orchestra.run.smithi164.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-03-23T14:43:02.775 INFO:teuthology.orchestra.run.smithi164.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-03-23T14:43:02.775 INFO:teuthology.orchestra.run.smithi164.stdout:UBUNTU_CODENAME=jammy 2024-03-23T14:43:02.775 INFO:teuthology.lock.ops:Updating smithi164.front.sepia.ceph.com on lock server 2024-03-23T14:43:02.797 DEBUG:teuthology.orchestra.run.smithi173:> uname -m 2024-03-23T14:43:02.803 INFO:teuthology.orchestra.run.smithi173.stdout:x86_64 2024-03-23T14:43:02.803 DEBUG:teuthology.orchestra.run.smithi173:> cat /etc/os-release 2024-03-23T14:43:02.848 INFO:teuthology.orchestra.run.smithi173.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-03-23T14:43:02.848 INFO:teuthology.orchestra.run.smithi173.stdout:NAME="Ubuntu" 2024-03-23T14:43:02.848 INFO:teuthology.orchestra.run.smithi173.stdout:VERSION_ID="22.04" 2024-03-23T14:43:02.848 INFO:teuthology.orchestra.run.smithi173.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-03-23T14:43:02.849 INFO:teuthology.orchestra.run.smithi173.stdout:VERSION_CODENAME=jammy 2024-03-23T14:43:02.849 INFO:teuthology.orchestra.run.smithi173.stdout:ID=ubuntu 2024-03-23T14:43:02.849 INFO:teuthology.orchestra.run.smithi173.stdout:ID_LIKE=debian 2024-03-23T14:43:02.849 INFO:teuthology.orchestra.run.smithi173.stdout:HOME_URL="https://www.ubuntu.com/" 2024-03-23T14:43:02.849 INFO:teuthology.orchestra.run.smithi173.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-03-23T14:43:02.849 INFO:teuthology.orchestra.run.smithi173.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-03-23T14:43:02.850 INFO:teuthology.orchestra.run.smithi173.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-03-23T14:43:02.850 INFO:teuthology.orchestra.run.smithi173.stdout:UBUNTU_CODENAME=jammy 2024-03-23T14:43:02.850 INFO:teuthology.lock.ops:Updating smithi173.front.sepia.ceph.com on lock server 2024-03-23T14:43:02.879 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-03-23T14:43:02.892 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-03-23T14:43:02.898 INFO:teuthology.task.internal:Checking for old test directory... 2024-03-23T14:43:02.899 DEBUG:teuthology.orchestra.run.smithi164:> test '!' -e /home/ubuntu/cephtest 2024-03-23T14:43:02.901 DEBUG:teuthology.orchestra.run.smithi173:> test '!' -e /home/ubuntu/cephtest 2024-03-23T14:43:02.906 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-03-23T14:43:02.911 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-03-23T14:43:02.912 DEBUG:teuthology.orchestra.run.smithi164:> test -z $(ls -A /var/lib/ceph) 2024-03-23T14:43:02.946 DEBUG:teuthology.orchestra.run.smithi173:> test -z $(ls -A /var/lib/ceph) 2024-03-23T14:43:02.964 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-03-23T14:43:03.018 INFO:teuthology.run_tasks:Running task kernel... 2024-03-23T14:43:03.037 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2024-03-23T14:43:03.038 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2024-03-23T14:43:03.038 DEBUG:teuthology.orchestra.run.smithi164:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-23T14:43:03.039 DEBUG:teuthology.orchestra.run.smithi173:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-23T14:43:03.043 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-23T14:43:03.043 DEBUG:teuthology.orchestra.run.smithi164:> uname -r 2024-03-23T14:43:03.044 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-23T14:43:03.044 DEBUG:teuthology.orchestra.run.smithi173:> uname -r 2024-03-23T14:43:03.090 INFO:teuthology.orchestra.run.smithi164.stdout:5.15.0-56-generic 2024-03-23T14:43:03.090 INFO:teuthology.task.kernel:Running kernel on smithi164: 5.15.0-56-generic 2024-03-23T14:43:03.090 DEBUG:teuthology.orchestra.run.smithi164:> sudo apt-get clean 2024-03-23T14:43:03.092 INFO:teuthology.orchestra.run.smithi173.stdout:5.15.0-56-generic 2024-03-23T14:43:03.092 INFO:teuthology.task.kernel:Running kernel on smithi173: 5.15.0-56-generic 2024-03-23T14:43:03.092 DEBUG:teuthology.orchestra.run.smithi173:> sudo apt-get clean 2024-03-23T14:43:03.211 DEBUG:teuthology.orchestra.run.smithi173:> sudo apt-get update 2024-03-23T14:43:03.217 DEBUG:teuthology.orchestra.run.smithi164:> sudo apt-get update 2024-03-23T14:43:03.356 INFO:teuthology.orchestra.run.smithi164.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-03-23T14:43:03.375 INFO:teuthology.orchestra.run.smithi173.stdout:Get:1 http://security.ubuntu.com/ubuntu jammy-security InRelease [110 kB] 2024-03-23T14:43:03.388 INFO:teuthology.orchestra.run.smithi164.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [119 kB] 2024-03-23T14:43:03.456 INFO:teuthology.orchestra.run.smithi173.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-03-23T14:43:03.461 INFO:teuthology.orchestra.run.smithi164.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [109 kB] 2024-03-23T14:43:03.479 INFO:teuthology.orchestra.run.smithi164.stdout:Get:4 http://security.ubuntu.com/ubuntu jammy-security InRelease [110 kB] 2024-03-23T14:43:03.538 INFO:teuthology.orchestra.run.smithi173.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [119 kB] 2024-03-23T14:43:03.592 INFO:teuthology.orchestra.run.smithi173.stdout:Get:4 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [428 kB] 2024-03-23T14:43:03.597 INFO:teuthology.orchestra.run.smithi164.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [594 kB] 2024-03-23T14:43:03.670 INFO:teuthology.orchestra.run.smithi173.stdout:Get:5 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,286 kB] 2024-03-23T14:43:03.686 INFO:teuthology.orchestra.run.smithi164.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [1,502 kB] 2024-03-23T14:43:03.719 INFO:teuthology.orchestra.run.smithi173.stdout:Get:6 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [229 kB] 2024-03-23T14:43:03.727 INFO:teuthology.orchestra.run.smithi173.stdout:Get:7 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [11.4 kB] 2024-03-23T14:43:03.728 INFO:teuthology.orchestra.run.smithi173.stdout:Get:8 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [1,591 kB] 2024-03-23T14:43:03.745 INFO:teuthology.orchestra.run.smithi164.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [289 kB] 2024-03-23T14:43:03.753 INFO:teuthology.orchestra.run.smithi164.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [16.1 kB] 2024-03-23T14:43:03.754 INFO:teuthology.orchestra.run.smithi164.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [35.3 kB] 2024-03-23T14:43:03.755 INFO:teuthology.orchestra.run.smithi164.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [1,619 kB] 2024-03-23T14:43:03.770 INFO:teuthology.orchestra.run.smithi173.stdout:Get:9 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [33.7 kB] 2024-03-23T14:43:03.772 INFO:teuthology.orchestra.run.smithi173.stdout:Get:10 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [267 kB] 2024-03-23T14:43:03.778 INFO:teuthology.orchestra.run.smithi173.stdout:Get:11 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [520 B] 2024-03-23T14:43:03.778 INFO:teuthology.orchestra.run.smithi173.stdout:Get:12 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [851 kB] 2024-03-23T14:43:03.798 INFO:teuthology.orchestra.run.smithi173.stdout:Get:13 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [599 kB] 2024-03-23T14:43:03.798 INFO:teuthology.orchestra.run.smithi164.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [271 kB] 2024-03-23T14:43:03.808 INFO:teuthology.orchestra.run.smithi164.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [520 B] 2024-03-23T14:43:03.808 INFO:teuthology.orchestra.run.smithi164.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,059 kB] 2024-03-23T14:43:03.814 INFO:teuthology.orchestra.run.smithi173.stdout:Get:14 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [162 kB] 2024-03-23T14:43:03.818 INFO:teuthology.orchestra.run.smithi173.stdout:Get:15 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [16.8 kB] 2024-03-23T14:43:03.819 INFO:teuthology.orchestra.run.smithi173.stdout:Get:16 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-03-23T14:43:03.832 INFO:teuthology.orchestra.run.smithi164.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [697 kB] 2024-03-23T14:43:03.836 INFO:teuthology.orchestra.run.smithi164.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [240 kB] 2024-03-23T14:43:03.838 INFO:teuthology.orchestra.run.smithi164.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [22.1 kB] 2024-03-23T14:43:03.839 INFO:teuthology.orchestra.run.smithi164.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,160 B] 2024-03-23T14:43:03.839 INFO:teuthology.orchestra.run.smithi164.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [42.1 kB] 2024-03-23T14:43:03.839 INFO:teuthology.orchestra.run.smithi164.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.1 kB] 2024-03-23T14:43:03.845 INFO:teuthology.orchestra.run.smithi173.stdout:Get:17 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.1 kB] 2024-03-23T14:43:03.849 INFO:teuthology.orchestra.run.smithi173.stdout:Get:18 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,476 B] 2024-03-23T14:43:03.849 INFO:teuthology.orchestra.run.smithi173.stdout:Get:19 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [260 B] 2024-03-23T14:43:03.859 INFO:teuthology.orchestra.run.smithi164.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [472 B] 2024-03-23T14:43:03.859 INFO:teuthology.orchestra.run.smithi164.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.2 kB] 2024-03-23T14:43:03.860 INFO:teuthology.orchestra.run.smithi164.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.1 kB] 2024-03-23T14:43:03.861 INFO:teuthology.orchestra.run.smithi164.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.0 kB] 2024-03-23T14:43:03.861 INFO:teuthology.orchestra.run.smithi164.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-03-23T14:43:03.861 INFO:teuthology.orchestra.run.smithi164.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [28.4 kB] 2024-03-23T14:43:03.885 INFO:teuthology.orchestra.run.smithi164.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [17.2 kB] 2024-03-23T14:43:03.885 INFO:teuthology.orchestra.run.smithi164.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.2 kB] 2024-03-23T14:43:03.885 INFO:teuthology.orchestra.run.smithi164.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [644 B] 2024-03-23T14:43:03.899 INFO:teuthology.orchestra.run.smithi173.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [109 kB] 2024-03-23T14:43:03.928 INFO:teuthology.orchestra.run.smithi164.stdout:Get:29 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [428 kB] 2024-03-23T14:43:04.036 INFO:teuthology.orchestra.run.smithi173.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [594 kB] 2024-03-23T14:43:04.206 INFO:teuthology.orchestra.run.smithi164.stdout:Get:30 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,286 kB] 2024-03-23T14:43:04.232 INFO:teuthology.orchestra.run.smithi173.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [1,502 kB] 2024-03-23T14:43:04.372 INFO:teuthology.orchestra.run.smithi164.stdout:Get:31 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [229 kB] 2024-03-23T14:43:04.378 INFO:teuthology.orchestra.run.smithi173.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [289 kB] 2024-03-23T14:43:04.384 INFO:teuthology.orchestra.run.smithi164.stdout:Get:32 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [11.4 kB] 2024-03-23T14:43:04.385 INFO:teuthology.orchestra.run.smithi164.stdout:Get:33 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [1,591 kB] 2024-03-23T14:43:04.393 INFO:teuthology.orchestra.run.smithi173.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [16.1 kB] 2024-03-23T14:43:04.393 INFO:teuthology.orchestra.run.smithi173.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [35.3 kB] 2024-03-23T14:43:04.395 INFO:teuthology.orchestra.run.smithi173.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [1,619 kB] 2024-03-23T14:43:04.454 INFO:teuthology.orchestra.run.smithi173.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [271 kB] 2024-03-23T14:43:04.461 INFO:teuthology.orchestra.run.smithi173.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [520 B] 2024-03-23T14:43:04.465 INFO:teuthology.orchestra.run.smithi164.stdout:Get:34 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [33.7 kB] 2024-03-23T14:43:04.466 INFO:teuthology.orchestra.run.smithi164.stdout:Get:35 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [267 kB] 2024-03-23T14:43:04.476 INFO:teuthology.orchestra.run.smithi164.stdout:Get:36 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [520 B] 2024-03-23T14:43:04.477 INFO:teuthology.orchestra.run.smithi164.stdout:Get:37 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [851 kB] 2024-03-23T14:43:04.497 INFO:teuthology.orchestra.run.smithi164.stdout:Get:38 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [599 kB] 2024-03-23T14:43:04.624 INFO:teuthology.orchestra.run.smithi173.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,059 kB] 2024-03-23T14:43:04.678 INFO:teuthology.orchestra.run.smithi164.stdout:Get:39 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [162 kB] 2024-03-23T14:43:05.074 INFO:teuthology.orchestra.run.smithi164.stdout:Get:40 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [16.8 kB] 2024-03-23T14:43:05.080 INFO:teuthology.orchestra.run.smithi164.stdout:Get:41 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-03-23T14:43:05.080 INFO:teuthology.orchestra.run.smithi164.stdout:Get:42 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.1 kB] 2024-03-23T14:43:05.095 INFO:teuthology.orchestra.run.smithi164.stdout:Get:43 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,476 B] 2024-03-23T14:43:05.097 INFO:teuthology.orchestra.run.smithi164.stdout:Get:44 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [260 B] 2024-03-23T14:43:05.250 INFO:teuthology.orchestra.run.smithi173.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [697 kB] 2024-03-23T14:43:05.288 INFO:teuthology.orchestra.run.smithi173.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [240 kB] 2024-03-23T14:43:05.301 INFO:teuthology.orchestra.run.smithi173.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [22.1 kB] 2024-03-23T14:43:05.302 INFO:teuthology.orchestra.run.smithi173.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,160 B] 2024-03-23T14:43:05.303 INFO:teuthology.orchestra.run.smithi173.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [42.1 kB] 2024-03-23T14:43:05.304 INFO:teuthology.orchestra.run.smithi173.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.1 kB] 2024-03-23T14:43:05.305 INFO:teuthology.orchestra.run.smithi173.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [472 B] 2024-03-23T14:43:05.306 INFO:teuthology.orchestra.run.smithi173.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.2 kB] 2024-03-23T14:43:05.332 INFO:teuthology.orchestra.run.smithi173.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.1 kB] 2024-03-23T14:43:05.336 INFO:teuthology.orchestra.run.smithi173.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.0 kB] 2024-03-23T14:43:05.501 INFO:teuthology.orchestra.run.smithi173.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-03-23T14:43:05.583 INFO:teuthology.orchestra.run.smithi173.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [17.2 kB] 2024-03-23T14:43:05.667 INFO:teuthology.orchestra.run.smithi173.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [28.4 kB] 2024-03-23T14:43:05.749 INFO:teuthology.orchestra.run.smithi173.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.2 kB] 2024-03-23T14:43:05.783 INFO:teuthology.orchestra.run.smithi173.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [644 B] 2024-03-23T14:43:09.311 INFO:teuthology.orchestra.run.smithi164.stdout:Fetched 12.5 MB in 2s (6,868 kB/s) 2024-03-23T14:43:10.139 INFO:teuthology.orchestra.run.smithi173.stdout:Fetched 12.5 MB in 3s (4,964 kB/s) 2024-03-23T14:43:10.648 INFO:teuthology.orchestra.run.smithi164.stdout:Reading package lists... 2024-03-23T14:43:10.666 DEBUG:teuthology.orchestra.run.smithi164:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-03-23T14:43:10.716 INFO:teuthology.orchestra.run.smithi164.stdout:Reading package lists... 2024-03-23T14:43:10.896 INFO:teuthology.orchestra.run.smithi164.stdout:Building dependency tree... 2024-03-23T14:43:10.897 INFO:teuthology.orchestra.run.smithi164.stdout:Reading state information... 2024-03-23T14:43:11.054 INFO:teuthology.orchestra.run.smithi164.stdout:The following packages were automatically installed and are no longer required: 2024-03-23T14:43:11.054 INFO:teuthology.orchestra.run.smithi164.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-03-23T14:43:11.054 INFO:teuthology.orchestra.run.smithi164.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-03-23T14:43:11.055 INFO:teuthology.orchestra.run.smithi164.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-03-23T14:43:11.055 INFO:teuthology.orchestra.run.smithi164.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-03-23T14:43:11.055 INFO:teuthology.orchestra.run.smithi164.stdout: python2.7-minimal 2024-03-23T14:43:11.055 INFO:teuthology.orchestra.run.smithi164.stdout:Use 'sudo apt autoremove' to remove them. 2024-03-23T14:43:11.056 INFO:teuthology.orchestra.run.smithi164.stdout:The following additional packages will be installed: 2024-03-23T14:43:11.056 INFO:teuthology.orchestra.run.smithi164.stdout: linux-generic linux-headers-5.15.0-101 linux-headers-5.15.0-101-generic 2024-03-23T14:43:11.056 INFO:teuthology.orchestra.run.smithi164.stdout: linux-headers-generic linux-image-5.15.0-101-generic 2024-03-23T14:43:11.057 INFO:teuthology.orchestra.run.smithi164.stdout: linux-modules-5.15.0-101-generic linux-modules-extra-5.15.0-101-generic 2024-03-23T14:43:11.058 INFO:teuthology.orchestra.run.smithi164.stdout:Suggested packages: 2024-03-23T14:43:11.058 INFO:teuthology.orchestra.run.smithi164.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-03-23T14:43:11.058 INFO:teuthology.orchestra.run.smithi164.stdout:Recommended packages: 2024-03-23T14:43:11.058 INFO:teuthology.orchestra.run.smithi164.stdout: thermald 2024-03-23T14:43:11.121 INFO:teuthology.orchestra.run.smithi164.stdout:The following NEW packages will be installed: 2024-03-23T14:43:11.121 INFO:teuthology.orchestra.run.smithi164.stdout: linux-headers-5.15.0-101 linux-headers-5.15.0-101-generic 2024-03-23T14:43:11.121 INFO:teuthology.orchestra.run.smithi164.stdout: linux-image-5.15.0-101-generic linux-modules-5.15.0-101-generic 2024-03-23T14:43:11.122 INFO:teuthology.orchestra.run.smithi164.stdout: linux-modules-extra-5.15.0-101-generic 2024-03-23T14:43:11.123 INFO:teuthology.orchestra.run.smithi164.stdout:The following packages will be upgraded: 2024-03-23T14:43:11.123 INFO:teuthology.orchestra.run.smithi164.stdout: linux-generic linux-headers-generic linux-image-generic 2024-03-23T14:43:11.302 INFO:teuthology.orchestra.run.smithi164.stdout:3 upgraded, 5 newly installed, 0 to remove and 298 not upgraded. 2024-03-23T14:43:11.302 INFO:teuthology.orchestra.run.smithi164.stdout:Need to get 113 MB of archives. 2024-03-23T14:43:11.302 INFO:teuthology.orchestra.run.smithi164.stdout:After this operation, 582 MB of additional disk space will be used. 2024-03-23T14:43:11.302 INFO:teuthology.orchestra.run.smithi164.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-101-generic amd64 5.15.0-101.111 [22.6 MB] 2024-03-23T14:43:11.545 INFO:teuthology.orchestra.run.smithi173.stdout:Reading package lists... 2024-03-23T14:43:11.561 DEBUG:teuthology.orchestra.run.smithi173:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-03-23T14:43:11.614 INFO:teuthology.orchestra.run.smithi173.stdout:Reading package lists... 2024-03-23T14:43:11.787 INFO:teuthology.orchestra.run.smithi173.stdout:Building dependency tree... 2024-03-23T14:43:11.787 INFO:teuthology.orchestra.run.smithi173.stdout:Reading state information... 2024-03-23T14:43:11.937 INFO:teuthology.orchestra.run.smithi173.stdout:The following packages were automatically installed and are no longer required: 2024-03-23T14:43:11.937 INFO:teuthology.orchestra.run.smithi173.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-03-23T14:43:11.937 INFO:teuthology.orchestra.run.smithi173.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-03-23T14:43:11.938 INFO:teuthology.orchestra.run.smithi173.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-03-23T14:43:11.938 INFO:teuthology.orchestra.run.smithi173.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-03-23T14:43:11.938 INFO:teuthology.orchestra.run.smithi173.stdout: python2.7-minimal 2024-03-23T14:43:11.938 INFO:teuthology.orchestra.run.smithi173.stdout:Use 'sudo apt autoremove' to remove them. 2024-03-23T14:43:11.939 INFO:teuthology.orchestra.run.smithi173.stdout:The following additional packages will be installed: 2024-03-23T14:43:11.939 INFO:teuthology.orchestra.run.smithi173.stdout: linux-generic linux-headers-5.15.0-101 linux-headers-5.15.0-101-generic 2024-03-23T14:43:11.939 INFO:teuthology.orchestra.run.smithi173.stdout: linux-headers-generic linux-image-5.15.0-101-generic 2024-03-23T14:43:11.939 INFO:teuthology.orchestra.run.smithi173.stdout: linux-modules-5.15.0-101-generic linux-modules-extra-5.15.0-101-generic 2024-03-23T14:43:11.940 INFO:teuthology.orchestra.run.smithi173.stdout:Suggested packages: 2024-03-23T14:43:11.940 INFO:teuthology.orchestra.run.smithi173.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-03-23T14:43:11.940 INFO:teuthology.orchestra.run.smithi173.stdout:Recommended packages: 2024-03-23T14:43:11.940 INFO:teuthology.orchestra.run.smithi173.stdout: thermald 2024-03-23T14:43:11.994 INFO:teuthology.orchestra.run.smithi173.stdout:The following NEW packages will be installed: 2024-03-23T14:43:11.994 INFO:teuthology.orchestra.run.smithi173.stdout: linux-headers-5.15.0-101 linux-headers-5.15.0-101-generic 2024-03-23T14:43:11.994 INFO:teuthology.orchestra.run.smithi173.stdout: linux-image-5.15.0-101-generic linux-modules-5.15.0-101-generic 2024-03-23T14:43:11.994 INFO:teuthology.orchestra.run.smithi173.stdout: linux-modules-extra-5.15.0-101-generic 2024-03-23T14:43:11.995 INFO:teuthology.orchestra.run.smithi173.stdout:The following packages will be upgraded: 2024-03-23T14:43:11.995 INFO:teuthology.orchestra.run.smithi173.stdout: linux-generic linux-headers-generic linux-image-generic 2024-03-23T14:43:12.174 INFO:teuthology.orchestra.run.smithi173.stdout:3 upgraded, 5 newly installed, 0 to remove and 298 not upgraded. 2024-03-23T14:43:12.174 INFO:teuthology.orchestra.run.smithi173.stdout:Need to get 113 MB of archives. 2024-03-23T14:43:12.174 INFO:teuthology.orchestra.run.smithi173.stdout:After this operation, 582 MB of additional disk space will be used. 2024-03-23T14:43:12.174 INFO:teuthology.orchestra.run.smithi173.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-101-generic amd64 5.15.0-101.111 [22.6 MB] 2024-03-23T14:43:12.686 INFO:teuthology.orchestra.run.smithi164.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-101-generic amd64 5.15.0-101.111 [11.5 MB] 2024-03-23T14:43:13.032 INFO:teuthology.orchestra.run.smithi164.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-101-generic amd64 5.15.0-101.111 [63.9 MB] 2024-03-23T14:43:13.691 INFO:teuthology.orchestra.run.smithi173.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-101-generic amd64 5.15.0-101.111 [11.5 MB] 2024-03-23T14:43:14.045 INFO:teuthology.orchestra.run.smithi173.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-101-generic amd64 5.15.0-101.111 [63.9 MB] 2024-03-23T14:43:15.174 INFO:teuthology.orchestra.run.smithi164.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.101.98 [1,698 B] 2024-03-23T14:43:15.174 INFO:teuthology.orchestra.run.smithi164.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.101.98 [2,486 B] 2024-03-23T14:43:15.174 INFO:teuthology.orchestra.run.smithi164.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-101 all 5.15.0-101.111 [12.3 MB] 2024-03-23T14:43:15.637 INFO:teuthology.orchestra.run.smithi164.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-101-generic amd64 5.15.0-101.111 [2,876 kB] 2024-03-23T14:43:15.734 INFO:teuthology.orchestra.run.smithi164.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.101.98 [2,332 B] 2024-03-23T14:43:16.144 INFO:teuthology.orchestra.run.smithi164.stdout:Fetched 113 MB in 5s (24.6 MB/s) 2024-03-23T14:43:16.313 INFO:teuthology.orchestra.run.smithi173.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.101.98 [1,698 B] 2024-03-23T14:43:16.314 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package linux-modules-5.15.0-101-generic. 2024-03-23T14:43:16.351 INFO:teuthology.orchestra.run.smithi173.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.101.98 [2,486 B] 2024-03-23T14:43:16.359 INFO:teuthology.orchestra.run.smithi173.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-101 all 5.15.0-101.111 [12.3 MB] 2024-03-23T14:43:17.680 INFO:teuthology.orchestra.run.smithi173.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-101-generic amd64 5.15.0-101.111 [2,876 kB] 2024-03-23T14:43:17.817 INFO:teuthology.orchestra.run.smithi173.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.101.98 [2,332 B] 2024-03-23T14:43:18.245 INFO:teuthology.orchestra.run.smithi173.stdout:Fetched 113 MB in 6s (19.5 MB/s) 2024-03-23T14:43:18.278 INFO:teuthology.orchestra.run.smithi164.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-03-23T14:43:18.283 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../0-linux-modules-5.15.0-101-generic_5.15.0-101.111_amd64.deb ... 2024-03-23T14:43:18.361 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking linux-modules-5.15.0-101-generic (5.15.0-101.111) ... 2024-03-23T14:43:18.416 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package linux-modules-5.15.0-101-generic. 2024-03-23T14:43:20.426 INFO:teuthology.orchestra.run.smithi173.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-03-23T14:43:20.430 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../0-linux-modules-5.15.0-101-generic_5.15.0-101.111_amd64.deb ... 2024-03-23T14:43:20.488 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking linux-modules-5.15.0-101-generic (5.15.0-101.111) ... 2024-03-23T14:43:20.811 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package linux-image-5.15.0-101-generic. 2024-03-23T14:43:20.832 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../1-linux-image-5.15.0-101-generic_5.15.0-101.111_amd64.deb ... 2024-03-23T14:43:20.913 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking linux-image-5.15.0-101-generic (5.15.0-101.111) ... 2024-03-23T14:43:21.431 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-101-generic. 2024-03-23T14:43:21.452 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-101-generic_5.15.0-101.111_amd64.deb ... 2024-03-23T14:43:21.489 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking linux-modules-extra-5.15.0-101-generic (5.15.0-101.111) ... 2024-03-23T14:43:22.647 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package linux-image-5.15.0-101-generic. 2024-03-23T14:43:22.668 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../1-linux-image-5.15.0-101-generic_5.15.0-101.111_amd64.deb ... 2024-03-23T14:43:22.745 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking linux-image-5.15.0-101-generic (5.15.0-101.111) ... 2024-03-23T14:43:23.225 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-101-generic. 2024-03-23T14:43:23.245 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-101-generic_5.15.0-101.111_amd64.deb ... 2024-03-23T14:43:23.291 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking linux-modules-extra-5.15.0-101-generic (5.15.0-101.111) ... 2024-03-23T14:43:27.858 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../3-linux-generic_5.15.0.101.98_amd64.deb ... 2024-03-23T14:43:27.983 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking linux-generic (5.15.0.101.98) over (5.15.0.56.54) ... 2024-03-23T14:43:28.315 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.101.98_amd64.deb ... 2024-03-23T14:43:28.436 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking linux-image-generic (5.15.0.101.98) over (5.15.0.56.54) ... 2024-03-23T14:43:28.756 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package linux-headers-5.15.0-101. 2024-03-23T14:43:28.778 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../5-linux-headers-5.15.0-101_5.15.0-101.111_all.deb ... 2024-03-23T14:43:28.814 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking linux-headers-5.15.0-101 (5.15.0-101.111) ... 2024-03-23T14:43:30.048 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../3-linux-generic_5.15.0.101.98_amd64.deb ... 2024-03-23T14:43:30.176 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking linux-generic (5.15.0.101.98) over (5.15.0.56.54) ... 2024-03-23T14:43:30.492 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.101.98_amd64.deb ... 2024-03-23T14:43:30.612 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking linux-image-generic (5.15.0.101.98) over (5.15.0.56.54) ... 2024-03-23T14:43:30.899 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package linux-headers-5.15.0-101. 2024-03-23T14:43:30.921 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../5-linux-headers-5.15.0-101_5.15.0-101.111_all.deb ... 2024-03-23T14:43:30.965 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking linux-headers-5.15.0-101 (5.15.0-101.111) ... 2024-03-23T14:43:34.869 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package linux-headers-5.15.0-101-generic. 2024-03-23T14:43:34.906 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../6-linux-headers-5.15.0-101-generic_5.15.0-101.111_amd64.deb ... 2024-03-23T14:43:34.943 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking linux-headers-5.15.0-101-generic (5.15.0-101.111) ... 2024-03-23T14:43:36.397 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package linux-headers-5.15.0-101-generic. 2024-03-23T14:43:36.434 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../6-linux-headers-5.15.0-101-generic_5.15.0-101.111_amd64.deb ... 2024-03-23T14:43:36.472 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking linux-headers-5.15.0-101-generic (5.15.0-101.111) ... 2024-03-23T14:43:37.604 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.101.98_amd64.deb ... 2024-03-23T14:43:37.729 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking linux-headers-generic (5.15.0.101.98) over (5.15.0.56.54) ... 2024-03-23T14:43:38.079 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up linux-headers-5.15.0-101 (5.15.0-101.111) ... 2024-03-23T14:43:38.201 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up linux-headers-5.15.0-101-generic (5.15.0-101.111) ... 2024-03-23T14:43:38.327 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up linux-headers-generic (5.15.0.101.98) ... 2024-03-23T14:43:38.455 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up linux-image-5.15.0-101-generic (5.15.0-101.111) ... 2024-03-23T14:43:39.760 INFO:teuthology.orchestra.run.smithi164.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-03-23T14:43:39.760 INFO:teuthology.orchestra.run.smithi164.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-03-23T14:43:39.760 INFO:teuthology.orchestra.run.smithi164.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-101-generic 2024-03-23T14:43:39.760 INFO:teuthology.orchestra.run.smithi164.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-101-generic 2024-03-23T14:43:39.935 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up linux-modules-extra-5.15.0-101-generic (5.15.0-101.111) ... 2024-03-23T14:43:40.268 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.101.98_amd64.deb ... 2024-03-23T14:43:40.382 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking linux-headers-generic (5.15.0.101.98) over (5.15.0.56.54) ... 2024-03-23T14:43:40.724 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up linux-headers-5.15.0-101 (5.15.0-101.111) ... 2024-03-23T14:43:40.828 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up linux-headers-5.15.0-101-generic (5.15.0-101.111) ... 2024-03-23T14:43:40.930 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up linux-headers-generic (5.15.0.101.98) ... 2024-03-23T14:43:41.050 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up linux-image-5.15.0-101-generic (5.15.0-101.111) ... 2024-03-23T14:43:41.319 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up linux-modules-5.15.0-101-generic (5.15.0-101.111) ... 2024-03-23T14:43:42.389 INFO:teuthology.orchestra.run.smithi173.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-03-23T14:43:42.389 INFO:teuthology.orchestra.run.smithi173.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-03-23T14:43:42.389 INFO:teuthology.orchestra.run.smithi173.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-101-generic 2024-03-23T14:43:42.389 INFO:teuthology.orchestra.run.smithi173.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-101-generic 2024-03-23T14:43:42.544 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up linux-modules-extra-5.15.0-101-generic (5.15.0-101.111) ... 2024-03-23T14:43:42.746 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up linux-image-generic (5.15.0.101.98) ... 2024-03-23T14:43:42.871 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up linux-generic (5.15.0.101.98) ... 2024-03-23T14:43:42.998 INFO:teuthology.orchestra.run.smithi164.stdout:Processing triggers for linux-image-5.15.0-101-generic (5.15.0-101.111) ... 2024-03-23T14:43:43.054 INFO:teuthology.orchestra.run.smithi164.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-03-23T14:43:43.054 INFO:teuthology.orchestra.run.smithi164.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-101-generic 2024-03-23T14:43:43.936 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up linux-modules-5.15.0-101-generic (5.15.0-101.111) ... 2024-03-23T14:43:45.579 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up linux-image-generic (5.15.0.101.98) ... 2024-03-23T14:43:45.680 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up linux-generic (5.15.0.101.98) ... 2024-03-23T14:43:45.806 INFO:teuthology.orchestra.run.smithi173.stdout:Processing triggers for linux-image-5.15.0-101-generic (5.15.0-101.111) ... 2024-03-23T14:43:45.884 INFO:teuthology.orchestra.run.smithi173.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-03-23T14:43:45.884 INFO:teuthology.orchestra.run.smithi173.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-101-generic 2024-03-23T14:43:57.419 INFO:teuthology.orchestra.run.smithi164.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-03-23T14:43:57.419 INFO:teuthology.orchestra.run.smithi164.stdout:Sourcing file `/etc/default/grub' 2024-03-23T14:43:57.433 INFO:teuthology.orchestra.run.smithi164.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-03-23T14:43:57.465 INFO:teuthology.orchestra.run.smithi164.stdout:Generating grub configuration file ... 2024-03-23T14:43:57.782 INFO:teuthology.orchestra.run.smithi164.stdout:Found linux image: /boot/vmlinuz-5.15.0-101-generic 2024-03-23T14:43:57.811 INFO:teuthology.orchestra.run.smithi164.stdout:Found initrd image: /boot/initrd.img-5.15.0-101-generic 2024-03-23T14:43:58.148 INFO:teuthology.orchestra.run.smithi164.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-03-23T14:43:58.154 INFO:teuthology.orchestra.run.smithi164.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-03-23T14:43:58.222 INFO:teuthology.orchestra.run.smithi164.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-03-23T14:43:58.228 INFO:teuthology.orchestra.run.smithi164.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-03-23T14:43:58.391 INFO:teuthology.orchestra.run.smithi164.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-03-23T14:43:58.391 INFO:teuthology.orchestra.run.smithi164.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-03-23T14:43:58.391 INFO:teuthology.orchestra.run.smithi164.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-03-23T14:43:58.443 INFO:teuthology.orchestra.run.smithi164.stdout:done 2024-03-23T14:43:58.936 DEBUG:teuthology.orchestra.run.smithi164:> dpkg -s linux-image-generic 2024-03-23T14:43:58.961 INFO:teuthology.orchestra.run.smithi164.stdout:Package: linux-image-generic 2024-03-23T14:43:58.961 INFO:teuthology.orchestra.run.smithi164.stdout:Status: install ok installed 2024-03-23T14:43:58.961 INFO:teuthology.orchestra.run.smithi164.stdout:Priority: optional 2024-03-23T14:43:58.961 INFO:teuthology.orchestra.run.smithi164.stdout:Section: kernel 2024-03-23T14:43:58.961 INFO:teuthology.orchestra.run.smithi164.stdout:Installed-Size: 21 2024-03-23T14:43:58.961 INFO:teuthology.orchestra.run.smithi164.stdout:Maintainer: Ubuntu Kernel Team 2024-03-23T14:43:58.962 INFO:teuthology.orchestra.run.smithi164.stdout:Architecture: amd64 2024-03-23T14:43:58.962 INFO:teuthology.orchestra.run.smithi164.stdout:Source: linux-meta 2024-03-23T14:43:58.962 INFO:teuthology.orchestra.run.smithi164.stdout:Version: 5.15.0.101.98 2024-03-23T14:43:58.962 INFO:teuthology.orchestra.run.smithi164.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-101), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-03-23T14:43:58.962 INFO:teuthology.orchestra.run.smithi164.stdout:Depends: linux-image-5.15.0-101-generic, linux-modules-extra-5.15.0-101-generic, linux-firmware, intel-microcode, amd64-microcode 2024-03-23T14:43:58.962 INFO:teuthology.orchestra.run.smithi164.stdout:Recommends: thermald 2024-03-23T14:43:58.962 INFO:teuthology.orchestra.run.smithi164.stdout:Description: Generic Linux kernel image 2024-03-23T14:43:58.962 INFO:teuthology.orchestra.run.smithi164.stdout: This package will always depend on the latest generic kernel image 2024-03-23T14:43:58.962 INFO:teuthology.orchestra.run.smithi164.stdout: available. 2024-03-23T14:43:58.962 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-101-generic 2024-03-23T14:43:58.963 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-03-23T14:43:58.963 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-03-23T14:43:58.963 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-03-23T14:43:58.963 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi164.front.sepia.ceph.com, path=None, version=distro) 2024-03-23T14:43:58.963 DEBUG:teuthology.orchestra.run.smithi164:> sudo apt-get clean 2024-03-23T14:43:59.134 DEBUG:teuthology.orchestra.run.smithi164:> sudo apt-get update 2024-03-23T14:43:59.385 INFO:teuthology.orchestra.run.smithi164.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-03-23T14:43:59.396 INFO:teuthology.orchestra.run.smithi164.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-03-23T14:43:59.478 INFO:teuthology.orchestra.run.smithi164.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-03-23T14:43:59.560 INFO:teuthology.orchestra.run.smithi164.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-03-23T14:43:59.825 INFO:teuthology.orchestra.run.smithi173.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-03-23T14:43:59.825 INFO:teuthology.orchestra.run.smithi173.stdout:Sourcing file `/etc/default/grub' 2024-03-23T14:43:59.840 INFO:teuthology.orchestra.run.smithi173.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-03-23T14:43:59.871 INFO:teuthology.orchestra.run.smithi173.stdout:Generating grub configuration file ... 2024-03-23T14:44:00.129 INFO:teuthology.orchestra.run.smithi173.stdout:Found linux image: /boot/vmlinuz-5.15.0-101-generic 2024-03-23T14:44:00.143 INFO:teuthology.orchestra.run.smithi173.stdout:Found initrd image: /boot/initrd.img-5.15.0-101-generic 2024-03-23T14:44:00.416 INFO:teuthology.orchestra.run.smithi173.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-03-23T14:44:00.421 INFO:teuthology.orchestra.run.smithi173.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-03-23T14:44:00.485 INFO:teuthology.orchestra.run.smithi173.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-03-23T14:44:00.490 INFO:teuthology.orchestra.run.smithi173.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-03-23T14:44:00.632 INFO:teuthology.orchestra.run.smithi173.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-03-23T14:44:00.632 INFO:teuthology.orchestra.run.smithi173.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-03-23T14:44:00.632 INFO:teuthology.orchestra.run.smithi173.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-03-23T14:44:00.683 INFO:teuthology.orchestra.run.smithi173.stdout:done 2024-03-23T14:44:00.725 INFO:teuthology.orchestra.run.smithi164.stdout:Reading package lists... 2024-03-23T14:44:00.743 DEBUG:teuthology.orchestra.run.smithi164:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-03-23T14:44:00.805 INFO:teuthology.orchestra.run.smithi164.stdout:Reading package lists... 2024-03-23T14:44:01.007 INFO:teuthology.orchestra.run.smithi164.stdout:Building dependency tree... 2024-03-23T14:44:01.007 INFO:teuthology.orchestra.run.smithi164.stdout:Reading state information... 2024-03-23T14:44:01.126 DEBUG:teuthology.orchestra.run.smithi173:> dpkg -s linux-image-generic 2024-03-23T14:44:01.150 INFO:teuthology.orchestra.run.smithi173.stdout:Package: linux-image-generic 2024-03-23T14:44:01.150 INFO:teuthology.orchestra.run.smithi173.stdout:Status: install ok installed 2024-03-23T14:44:01.150 INFO:teuthology.orchestra.run.smithi173.stdout:Priority: optional 2024-03-23T14:44:01.150 INFO:teuthology.orchestra.run.smithi173.stdout:Section: kernel 2024-03-23T14:44:01.150 INFO:teuthology.orchestra.run.smithi173.stdout:Installed-Size: 21 2024-03-23T14:44:01.150 INFO:teuthology.orchestra.run.smithi173.stdout:Maintainer: Ubuntu Kernel Team 2024-03-23T14:44:01.150 INFO:teuthology.orchestra.run.smithi173.stdout:Architecture: amd64 2024-03-23T14:44:01.150 INFO:teuthology.orchestra.run.smithi173.stdout:Source: linux-meta 2024-03-23T14:44:01.150 INFO:teuthology.orchestra.run.smithi173.stdout:Version: 5.15.0.101.98 2024-03-23T14:44:01.150 INFO:teuthology.orchestra.run.smithi173.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-101), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-03-23T14:44:01.151 INFO:teuthology.orchestra.run.smithi173.stdout:Depends: linux-image-5.15.0-101-generic, linux-modules-extra-5.15.0-101-generic, linux-firmware, intel-microcode, amd64-microcode 2024-03-23T14:44:01.151 INFO:teuthology.orchestra.run.smithi173.stdout:Recommends: thermald 2024-03-23T14:44:01.151 INFO:teuthology.orchestra.run.smithi173.stdout:Description: Generic Linux kernel image 2024-03-23T14:44:01.151 INFO:teuthology.orchestra.run.smithi173.stdout: This package will always depend on the latest generic kernel image 2024-03-23T14:44:01.151 INFO:teuthology.orchestra.run.smithi173.stdout: available. 2024-03-23T14:44:01.151 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-101-generic 2024-03-23T14:44:01.151 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-03-23T14:44:01.151 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-03-23T14:44:01.151 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-03-23T14:44:01.151 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi173.front.sepia.ceph.com, path=None, version=distro) 2024-03-23T14:44:01.152 DEBUG:teuthology.orchestra.run.smithi173:> sudo apt-get clean 2024-03-23T14:44:01.183 INFO:teuthology.orchestra.run.smithi164.stdout:linux-image-generic is already the newest version (5.15.0.101.98). 2024-03-23T14:44:01.183 INFO:teuthology.orchestra.run.smithi164.stdout:The following packages were automatically installed and are no longer required: 2024-03-23T14:44:01.183 INFO:teuthology.orchestra.run.smithi164.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-03-23T14:44:01.184 INFO:teuthology.orchestra.run.smithi164.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-03-23T14:44:01.184 INFO:teuthology.orchestra.run.smithi164.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-03-23T14:44:01.184 INFO:teuthology.orchestra.run.smithi164.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-03-23T14:44:01.184 INFO:teuthology.orchestra.run.smithi164.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-03-23T14:44:01.184 INFO:teuthology.orchestra.run.smithi164.stdout:Use 'sudo apt autoremove' to remove them. 2024-03-23T14:44:01.224 INFO:teuthology.orchestra.run.smithi164.stdout:0 upgraded, 0 newly installed, 0 to remove and 298 not upgraded. 2024-03-23T14:44:01.225 DEBUG:teuthology.orchestra.run.smithi164:> dpkg -s linux-image-generic 2024-03-23T14:44:01.241 INFO:teuthology.orchestra.run.smithi164.stdout:Package: linux-image-generic 2024-03-23T14:44:01.241 INFO:teuthology.orchestra.run.smithi164.stdout:Status: install ok installed 2024-03-23T14:44:01.242 INFO:teuthology.orchestra.run.smithi164.stdout:Priority: optional 2024-03-23T14:44:01.242 INFO:teuthology.orchestra.run.smithi164.stdout:Section: kernel 2024-03-23T14:44:01.242 INFO:teuthology.orchestra.run.smithi164.stdout:Installed-Size: 21 2024-03-23T14:44:01.242 INFO:teuthology.orchestra.run.smithi164.stdout:Maintainer: Ubuntu Kernel Team 2024-03-23T14:44:01.242 INFO:teuthology.orchestra.run.smithi164.stdout:Architecture: amd64 2024-03-23T14:44:01.242 INFO:teuthology.orchestra.run.smithi164.stdout:Source: linux-meta 2024-03-23T14:44:01.242 INFO:teuthology.orchestra.run.smithi164.stdout:Version: 5.15.0.101.98 2024-03-23T14:44:01.242 INFO:teuthology.orchestra.run.smithi164.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-101), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-03-23T14:44:01.242 INFO:teuthology.orchestra.run.smithi164.stdout:Depends: linux-image-5.15.0-101-generic, linux-modules-extra-5.15.0-101-generic, linux-firmware, intel-microcode, amd64-microcode 2024-03-23T14:44:01.242 INFO:teuthology.orchestra.run.smithi164.stdout:Recommends: thermald 2024-03-23T14:44:01.242 INFO:teuthology.orchestra.run.smithi164.stdout:Description: Generic Linux kernel image 2024-03-23T14:44:01.242 INFO:teuthology.orchestra.run.smithi164.stdout: This package will always depend on the latest generic kernel image 2024-03-23T14:44:01.242 INFO:teuthology.orchestra.run.smithi164.stdout: available. 2024-03-23T14:44:01.243 DEBUG:teuthology.orchestra.run.smithi164:> mktemp 2024-03-23T14:44:01.288 INFO:teuthology.orchestra.run.smithi164.stdout:/tmp/tmp.9WPHCBKyMj 2024-03-23T14:44:01.289 DEBUG:teuthology.orchestra.run.smithi164:> sudo cp /boot/grub/grub.cfg /tmp/tmp.9WPHCBKyMj 2024-03-23T14:44:01.300 DEBUG:teuthology.orchestra.run.smithi173:> sudo apt-get update 2024-03-23T14:44:01.345 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod 0666 /tmp/tmp.9WPHCBKyMj 2024-03-23T14:44:01.427 INFO:teuthology.orchestra.run.smithi173.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-03-23T14:44:01.437 INFO:teuthology.orchestra.run.smithi173.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-03-23T14:44:01.459 DEBUG:teuthology.orchestra.remote:smithi164:/tmp/tmp.9WPHCBKyMj is 10KB 2024-03-23T14:44:01.461 INFO:teuthology.orchestra.run.smithi173.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-03-23T14:44:01.472 DEBUG:teuthology.orchestra.run.smithi164:> rm -fr /tmp/tmp.9WPHCBKyMj 2024-03-23T14:44:01.478 DEBUG:teuthology.orchestra.run.smithi164:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-03-23T14:44:01.532 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-23T14:44:01.532 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-03-23T14:44:01.532 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-03-23T14:44:01.536 INFO:teuthology.orchestra.run.smithi173.stdout:Hit:4 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-03-23T14:44:01.599 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-101-generic 2024-03-23T14:44:01.599 DEBUG:teuthology.orchestra.run.smithi164:> sudo update-grub 2024-03-23T14:44:02.749 INFO:teuthology.orchestra.run.smithi173.stdout:Reading package lists... 2024-03-23T14:44:02.766 DEBUG:teuthology.orchestra.run.smithi173:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-03-23T14:44:02.818 INFO:teuthology.orchestra.run.smithi173.stdout:Reading package lists... 2024-03-23T14:44:02.854 INFO:teuthology.orchestra.run.smithi164.stderr:Sourcing file `/etc/default/grub' 2024-03-23T14:44:02.855 INFO:teuthology.orchestra.run.smithi164.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-03-23T14:44:02.857 INFO:teuthology.orchestra.run.smithi164.stderr:Generating grub configuration file ... 2024-03-23T14:44:02.993 INFO:teuthology.orchestra.run.smithi173.stdout:Building dependency tree... 2024-03-23T14:44:02.993 INFO:teuthology.orchestra.run.smithi173.stdout:Reading state information... 2024-03-23T14:44:03.105 INFO:teuthology.orchestra.run.smithi164.stderr:Found linux image: /boot/vmlinuz-5.15.0-101-generic 2024-03-23T14:44:03.111 INFO:teuthology.orchestra.run.smithi164.stderr:Found initrd image: /boot/initrd.img-5.15.0-101-generic 2024-03-23T14:44:03.149 INFO:teuthology.orchestra.run.smithi173.stdout:linux-image-generic is already the newest version (5.15.0.101.98). 2024-03-23T14:44:03.150 INFO:teuthology.orchestra.run.smithi173.stdout:The following packages were automatically installed and are no longer required: 2024-03-23T14:44:03.150 INFO:teuthology.orchestra.run.smithi173.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-03-23T14:44:03.150 INFO:teuthology.orchestra.run.smithi173.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-03-23T14:44:03.150 INFO:teuthology.orchestra.run.smithi173.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-03-23T14:44:03.150 INFO:teuthology.orchestra.run.smithi173.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-03-23T14:44:03.150 INFO:teuthology.orchestra.run.smithi173.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-03-23T14:44:03.151 INFO:teuthology.orchestra.run.smithi173.stdout:Use 'sudo apt autoremove' to remove them. 2024-03-23T14:44:03.188 INFO:teuthology.orchestra.run.smithi173.stdout:0 upgraded, 0 newly installed, 0 to remove and 298 not upgraded. 2024-03-23T14:44:03.190 DEBUG:teuthology.orchestra.run.smithi173:> dpkg -s linux-image-generic 2024-03-23T14:44:03.249 INFO:teuthology.orchestra.run.smithi173.stdout:Package: linux-image-generic 2024-03-23T14:44:03.249 INFO:teuthology.orchestra.run.smithi173.stdout:Status: install ok installed 2024-03-23T14:44:03.249 INFO:teuthology.orchestra.run.smithi173.stdout:Priority: optional 2024-03-23T14:44:03.249 INFO:teuthology.orchestra.run.smithi173.stdout:Section: kernel 2024-03-23T14:44:03.249 INFO:teuthology.orchestra.run.smithi173.stdout:Installed-Size: 21 2024-03-23T14:44:03.249 INFO:teuthology.orchestra.run.smithi173.stdout:Maintainer: Ubuntu Kernel Team 2024-03-23T14:44:03.250 INFO:teuthology.orchestra.run.smithi173.stdout:Architecture: amd64 2024-03-23T14:44:03.250 INFO:teuthology.orchestra.run.smithi173.stdout:Source: linux-meta 2024-03-23T14:44:03.250 INFO:teuthology.orchestra.run.smithi173.stdout:Version: 5.15.0.101.98 2024-03-23T14:44:03.250 INFO:teuthology.orchestra.run.smithi173.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-101), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-03-23T14:44:03.250 INFO:teuthology.orchestra.run.smithi173.stdout:Depends: linux-image-5.15.0-101-generic, linux-modules-extra-5.15.0-101-generic, linux-firmware, intel-microcode, amd64-microcode 2024-03-23T14:44:03.250 INFO:teuthology.orchestra.run.smithi173.stdout:Recommends: thermald 2024-03-23T14:44:03.250 INFO:teuthology.orchestra.run.smithi173.stdout:Description: Generic Linux kernel image 2024-03-23T14:44:03.250 INFO:teuthology.orchestra.run.smithi173.stdout: This package will always depend on the latest generic kernel image 2024-03-23T14:44:03.250 INFO:teuthology.orchestra.run.smithi173.stdout: available. 2024-03-23T14:44:03.250 DEBUG:teuthology.orchestra.run.smithi173:> mktemp 2024-03-23T14:44:03.295 INFO:teuthology.orchestra.run.smithi173.stdout:/tmp/tmp.5vGh7I0K2d 2024-03-23T14:44:03.295 DEBUG:teuthology.orchestra.run.smithi173:> sudo cp /boot/grub/grub.cfg /tmp/tmp.5vGh7I0K2d 2024-03-23T14:44:03.351 DEBUG:teuthology.orchestra.run.smithi173:> sudo chmod 0666 /tmp/tmp.5vGh7I0K2d 2024-03-23T14:44:03.452 INFO:teuthology.orchestra.run.smithi164.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-03-23T14:44:03.458 INFO:teuthology.orchestra.run.smithi164.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-03-23T14:44:03.472 DEBUG:teuthology.orchestra.remote:smithi173:/tmp/tmp.5vGh7I0K2d is 10KB 2024-03-23T14:44:03.484 DEBUG:teuthology.orchestra.run.smithi173:> rm -fr /tmp/tmp.5vGh7I0K2d 2024-03-23T14:44:03.489 DEBUG:teuthology.orchestra.run.smithi173:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-03-23T14:44:03.499 INFO:teuthology.orchestra.run.smithi164.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-03-23T14:44:03.505 INFO:teuthology.orchestra.run.smithi164.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-03-23T14:44:03.546 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-03-23T14:44:03.546 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-03-23T14:44:03.546 DEBUG:teuthology.orchestra.run.smithi173:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-03-23T14:44:03.609 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-101-generic 2024-03-23T14:44:03.609 DEBUG:teuthology.orchestra.run.smithi173:> sudo update-grub 2024-03-23T14:44:03.626 INFO:teuthology.orchestra.run.smithi164.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-03-23T14:44:03.626 INFO:teuthology.orchestra.run.smithi164.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-03-23T14:44:03.626 INFO:teuthology.orchestra.run.smithi164.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-03-23T14:44:03.642 INFO:teuthology.orchestra.run.smithi164.stderr:done 2024-03-23T14:44:03.652 DEBUG:teuthology.orchestra.run.smithi164:> sudo shutdown -r now 2024-03-23T14:44:04.756 INFO:teuthology.orchestra.run.smithi173.stderr:Sourcing file `/etc/default/grub' 2024-03-23T14:44:04.757 INFO:teuthology.orchestra.run.smithi173.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-03-23T14:44:04.760 INFO:teuthology.orchestra.run.smithi173.stderr:Generating grub configuration file ... 2024-03-23T14:44:04.934 INFO:teuthology.orchestra.run.smithi173.stderr:Found linux image: /boot/vmlinuz-5.15.0-101-generic 2024-03-23T14:44:04.941 INFO:teuthology.orchestra.run.smithi173.stderr:Found initrd image: /boot/initrd.img-5.15.0-101-generic 2024-03-23T14:44:05.177 INFO:teuthology.orchestra.run.smithi173.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-03-23T14:44:05.183 INFO:teuthology.orchestra.run.smithi173.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-03-23T14:44:05.226 INFO:teuthology.orchestra.run.smithi173.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-03-23T14:44:05.232 INFO:teuthology.orchestra.run.smithi173.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-03-23T14:44:05.382 INFO:teuthology.orchestra.run.smithi173.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-03-23T14:44:05.382 INFO:teuthology.orchestra.run.smithi173.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-03-23T14:44:05.382 INFO:teuthology.orchestra.run.smithi173.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-03-23T14:44:05.408 INFO:teuthology.orchestra.run.smithi173.stderr:done 2024-03-23T14:44:05.416 DEBUG:teuthology.orchestra.run.smithi173:> sudo shutdown -r now 2024-03-23T14:44:33.677 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-03-23T14:44:33.678 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi164.front.sepia.ceph.com' 2024-03-23T14:44:33.678 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-23T14:44:35.421 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-03-23T14:44:35.421 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi173.front.sepia.ceph.com' 2024-03-23T14:44:35.421 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi173.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-23T14:44:52.126 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.164 2024-03-23T14:44:53.918 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.173 2024-03-23T14:45:01.135 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi164.front.sepia.ceph.com' 2024-03-23T14:45:01.135 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-23T14:45:02.921 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi173.front.sepia.ceph.com' 2024-03-23T14:45:02.921 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi173.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-23T14:45:04.190 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.164 2024-03-23T14:45:05.982 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.173 2024-03-23T14:45:16.198 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi164.front.sepia.ceph.com' 2024-03-23T14:45:16.199 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-23T14:45:17.983 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi173.front.sepia.ceph.com' 2024-03-23T14:45:17.984 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi173.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-23T14:45:34.622 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.164 2024-03-23T14:45:49.625 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi164.front.sepia.ceph.com' 2024-03-23T14:45:49.625 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-23T14:45:49.628 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.164 2024-03-23T14:45:49.891 DEBUG:teuthology.orchestra.run.smithi173:> true 2024-03-23T14:45:50.448 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi173.front.sepia.ceph.com' 2024-03-23T14:45:50.448 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.15.0-101-generic"... 2024-03-23T14:45:50.448 DEBUG:teuthology.orchestra.run.smithi173:> uname -r 2024-03-23T14:45:50.495 INFO:teuthology.orchestra.run.smithi173.stdout:5.15.0-101-generic 2024-03-23T14:45:50.495 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-101-generic vs 5.15.0-101-generic 2024-03-23T14:45:50.495 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-23T14:45:50.495 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-03-23T14:45:51.497 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-03-23T14:45:51.498 DEBUG:teuthology.orchestra.run.smithi173:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-03-23T14:45:51.564 INFO:teuthology.orchestra.run.smithi173.stdout:ttyS1 2024-03-23T14:45:51.592 DEBUG:teuthology.parallel:result is None 2024-03-23T14:46:07.629 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi164.front.sepia.ceph.com' 2024-03-23T14:46:07.629 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-23T14:46:07.867 DEBUG:teuthology.orchestra.run.smithi164:> true 2024-03-23T14:46:08.445 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi164.front.sepia.ceph.com' 2024-03-23T14:46:08.445 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.15.0-101-generic"... 2024-03-23T14:46:08.445 DEBUG:teuthology.orchestra.run.smithi164:> uname -r 2024-03-23T14:46:08.493 INFO:teuthology.orchestra.run.smithi164.stdout:5.15.0-101-generic 2024-03-23T14:46:08.493 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-101-generic vs 5.15.0-101-generic 2024-03-23T14:46:08.493 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-23T14:46:08.493 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-03-23T14:46:09.494 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-03-23T14:46:09.495 DEBUG:teuthology.orchestra.run.smithi164:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-03-23T14:46:09.575 INFO:teuthology.orchestra.run.smithi164.stdout:ttyS1 2024-03-23T14:46:09.595 DEBUG:teuthology.parallel:result is None 2024-03-23T14:46:09.595 INFO:teuthology.run_tasks:Running task internal.base... 2024-03-23T14:46:09.602 INFO:teuthology.task.internal:Creating test directory... 2024-03-23T14:46:09.603 DEBUG:teuthology.orchestra.run.smithi164:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-23T14:46:09.607 DEBUG:teuthology.orchestra.run.smithi173:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-23T14:46:09.613 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-03-23T14:46:09.623 INFO:teuthology.run_tasks:Running task internal.archive... 2024-03-23T14:46:09.634 INFO:teuthology.task.internal:Creating archive directory... 2024-03-23T14:46:09.635 DEBUG:teuthology.orchestra.run.smithi164:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-23T14:46:09.655 DEBUG:teuthology.orchestra.run.smithi173:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-23T14:46:09.753 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-03-23T14:46:09.760 INFO:teuthology.task.internal:Enabling coredump saving... 2024-03-23T14:46:09.760 DEBUG:teuthology.orchestra.run.smithi164:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-23T14:46:09.762 DEBUG:teuthology.orchestra.run.smithi173:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-23T14:46:09.780 INFO:teuthology.orchestra.run.smithi164.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-23T14:46:09.781 INFO:teuthology.orchestra.run.smithi173.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-23T14:46:09.790 INFO:teuthology.orchestra.run.smithi164.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-23T14:46:09.791 INFO:teuthology.orchestra.run.smithi173.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-23T14:46:09.792 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-03-23T14:46:09.799 INFO:teuthology.task.internal:Configuring sudo... 2024-03-23T14:46:09.799 DEBUG:teuthology.orchestra.run.smithi164:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-23T14:46:09.834 DEBUG:teuthology.orchestra.run.smithi173:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-23T14:46:09.849 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-03-23T14:46:09.858 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-03-23T14:46:09.859 DEBUG:teuthology.orchestra.run.smithi164:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-23T14:46:09.894 DEBUG:teuthology.orchestra.run.smithi173:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-23T14:46:09.899 DEBUG:teuthology.orchestra.run.smithi164:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-23T14:46:09.945 DEBUG:teuthology.orchestra.run.smithi164:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-23T14:46:09.995 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-23T14:46:09.996 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-23T14:46:10.056 DEBUG:teuthology.orchestra.run.smithi173:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-23T14:46:10.062 DEBUG:teuthology.orchestra.run.smithi173:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-23T14:46:10.112 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-03-23T14:46:10.112 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-23T14:46:10.174 DEBUG:teuthology.orchestra.run.smithi164:> sudo service rsyslog restart 2024-03-23T14:46:10.176 DEBUG:teuthology.orchestra.run.smithi173:> sudo service rsyslog restart 2024-03-23T14:46:10.247 INFO:teuthology.run_tasks:Running task internal.timer... 2024-03-23T14:46:10.257 INFO:teuthology.task.internal:Starting timer... 2024-03-23T14:46:10.257 INFO:teuthology.run_tasks:Running task pcp... 2024-03-23T14:46:10.268 INFO:teuthology.run_tasks:Running task selinux... 2024-03-23T14:46:10.276 DEBUG:teuthology.task.selinux:Excluding smithi164: OS 'ubuntu' does not support SELinux 2024-03-23T14:46:10.276 DEBUG:teuthology.task.selinux:Excluding smithi173: OS 'ubuntu' does not support SELinux 2024-03-23T14:46:10.276 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-03-23T14:46:10.276 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-03-23T14:46:10.276 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-03-23T14:46:10.276 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-03-23T14:46:10.287 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-03-23T14:46:10.307 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-03-23T14:46:10.308 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi164.front.sepia.ceph.com,smithi173.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-03-23T14:53:15.178 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi164.front.sepia.ceph.com'), Remote(name='ubuntu@smithi173.front.sepia.ceph.com')] 2024-03-23T14:53:15.179 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi164.front.sepia.ceph.com' 2024-03-23T14:53:15.180 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-23T14:53:15.263 DEBUG:teuthology.orchestra.run.smithi164:> true 2024-03-23T14:53:15.343 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi164.front.sepia.ceph.com' 2024-03-23T14:53:15.343 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi173.front.sepia.ceph.com' 2024-03-23T14:53:15.344 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi173.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-23T14:53:15.424 DEBUG:teuthology.orchestra.run.smithi173:> true 2024-03-23T14:53:15.502 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi173.front.sepia.ceph.com' 2024-03-23T14:53:15.503 INFO:teuthology.run_tasks:Running task clock... 2024-03-23T14:53:15.514 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-03-23T14:53:15.515 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-23T14:53:15.515 DEBUG:teuthology.orchestra.run.smithi164:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-23T14:53:15.517 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-23T14:53:15.517 DEBUG:teuthology.orchestra.run.smithi173:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-23T14:53:15.545 INFO:teuthology.orchestra.run.smithi164.stdout:23 Mar 14:53:15 ntpd[17162]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-03-23T14:53:15.545 INFO:teuthology.orchestra.run.smithi164.stdout:23 Mar 14:53:15 ntpd[17162]: Command line: ntpd -gq 2024-03-23T14:53:15.545 INFO:teuthology.orchestra.run.smithi164.stdout:23 Mar 14:53:15 ntpd[17162]: ---------------------------------------------------- 2024-03-23T14:53:15.545 INFO:teuthology.orchestra.run.smithi164.stdout:23 Mar 14:53:15 ntpd[17162]: ntp-4 is maintained by Network Time Foundation, 2024-03-23T14:53:15.545 INFO:teuthology.orchestra.run.smithi164.stdout:23 Mar 14:53:15 ntpd[17162]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-03-23T14:53:15.545 INFO:teuthology.orchestra.run.smithi164.stdout:23 Mar 14:53:15 ntpd[17162]: corporation. Support and training for ntp-4 are 2024-03-23T14:53:15.545 INFO:teuthology.orchestra.run.smithi164.stdout:23 Mar 14:53:15 ntpd[17162]: available at https://www.nwtime.org/support 2024-03-23T14:53:15.545 INFO:teuthology.orchestra.run.smithi164.stdout:23 Mar 14:53:15 ntpd[17162]: ---------------------------------------------------- 2024-03-23T14:53:15.546 INFO:teuthology.orchestra.run.smithi164.stdout:23 Mar 14:53:15 ntpd[17162]: proto: precision = 0.061 usec (-24) 2024-03-23T14:53:15.546 INFO:teuthology.orchestra.run.smithi164.stdout:23 Mar 14:53:15 ntpd[17162]: basedate set to 2022-02-04 2024-03-23T14:53:15.546 INFO:teuthology.orchestra.run.smithi164.stdout:23 Mar 14:53:15 ntpd[17162]: gps base set to 2022-02-06 (week 2196) 2024-03-23T14:53:15.546 INFO:teuthology.orchestra.run.smithi164.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-03-23T14:53:15.547 INFO:teuthology.orchestra.run.smithi164.stdout:23 Mar 14:53:15 ntpd[17162]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-03-23T14:53:15.547 INFO:teuthology.orchestra.run.smithi164.stdout:23 Mar 14:53:15 ntpd[17162]: restrict ::: KOD does nothing without LIMITED. 2024-03-23T14:53:15.547 INFO:teuthology.orchestra.run.smithi164.stderr:restrict ::: KOD does nothing without LIMITED. 2024-03-23T14:53:15.548 INFO:teuthology.orchestra.run.smithi164.stdout:23 Mar 14:53:15 ntpd[17162]: Listen and drop on 0 v6wildcard [::]:123 2024-03-23T14:53:15.548 INFO:teuthology.orchestra.run.smithi164.stdout:23 Mar 14:53:15 ntpd[17162]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-03-23T14:53:15.548 INFO:teuthology.orchestra.run.smithi164.stdout:23 Mar 14:53:15 ntpd[17162]: Listen normally on 2 lo 127.0.0.1:123 2024-03-23T14:53:15.548 INFO:teuthology.orchestra.run.smithi164.stdout:23 Mar 14:53:15 ntpd[17162]: Listen normally on 3 enp3s0f1 172.21.15.164:123 2024-03-23T14:53:15.548 INFO:teuthology.orchestra.run.smithi164.stdout:23 Mar 14:53:15 ntpd[17162]: Listen normally on 4 lo [::1]:123 2024-03-23T14:53:15.548 INFO:teuthology.orchestra.run.smithi164.stdout:23 Mar 14:53:15 ntpd[17162]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:73f3%5]:123 2024-03-23T14:53:15.548 INFO:teuthology.orchestra.run.smithi164.stdout:23 Mar 14:53:15 ntpd[17162]: Listening on routing socket on fd #22 for interface updates 2024-03-23T14:53:15.572 INFO:teuthology.orchestra.run.smithi173.stdout:23 Mar 14:53:15 ntpd[17191]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-03-23T14:53:15.572 INFO:teuthology.orchestra.run.smithi173.stdout:23 Mar 14:53:15 ntpd[17191]: Command line: ntpd -gq 2024-03-23T14:53:15.572 INFO:teuthology.orchestra.run.smithi173.stdout:23 Mar 14:53:15 ntpd[17191]: ---------------------------------------------------- 2024-03-23T14:53:15.572 INFO:teuthology.orchestra.run.smithi173.stdout:23 Mar 14:53:15 ntpd[17191]: ntp-4 is maintained by Network Time Foundation, 2024-03-23T14:53:15.572 INFO:teuthology.orchestra.run.smithi173.stdout:23 Mar 14:53:15 ntpd[17191]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-03-23T14:53:15.572 INFO:teuthology.orchestra.run.smithi173.stdout:23 Mar 14:53:15 ntpd[17191]: corporation. Support and training for ntp-4 are 2024-03-23T14:53:15.573 INFO:teuthology.orchestra.run.smithi173.stdout:23 Mar 14:53:15 ntpd[17191]: available at https://www.nwtime.org/support 2024-03-23T14:53:15.573 INFO:teuthology.orchestra.run.smithi173.stdout:23 Mar 14:53:15 ntpd[17191]: ---------------------------------------------------- 2024-03-23T14:53:15.573 INFO:teuthology.orchestra.run.smithi173.stdout:23 Mar 14:53:15 ntpd[17191]: proto: precision = 0.056 usec (-24) 2024-03-23T14:53:15.573 INFO:teuthology.orchestra.run.smithi173.stdout:23 Mar 14:53:15 ntpd[17191]: basedate set to 2022-02-04 2024-03-23T14:53:15.573 INFO:teuthology.orchestra.run.smithi173.stdout:23 Mar 14:53:15 ntpd[17191]: gps base set to 2022-02-06 (week 2196) 2024-03-23T14:53:15.573 INFO:teuthology.orchestra.run.smithi173.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-03-23T14:53:15.574 INFO:teuthology.orchestra.run.smithi173.stdout:23 Mar 14:53:15 ntpd[17191]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-03-23T14:53:15.574 INFO:teuthology.orchestra.run.smithi173.stdout:23 Mar 14:53:15 ntpd[17191]: restrict ::: KOD does nothing without LIMITED. 2024-03-23T14:53:15.574 INFO:teuthology.orchestra.run.smithi173.stderr:restrict ::: KOD does nothing without LIMITED. 2024-03-23T14:53:15.575 INFO:teuthology.orchestra.run.smithi173.stdout:23 Mar 14:53:15 ntpd[17191]: Listen and drop on 0 v6wildcard [::]:123 2024-03-23T14:53:15.575 INFO:teuthology.orchestra.run.smithi173.stdout:23 Mar 14:53:15 ntpd[17191]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-03-23T14:53:15.575 INFO:teuthology.orchestra.run.smithi173.stdout:23 Mar 14:53:15 ntpd[17191]: Listen normally on 2 lo 127.0.0.1:123 2024-03-23T14:53:15.575 INFO:teuthology.orchestra.run.smithi173.stdout:23 Mar 14:53:15 ntpd[17191]: Listen normally on 3 enp3s0f1 172.21.15.173:123 2024-03-23T14:53:15.575 INFO:teuthology.orchestra.run.smithi173.stdout:23 Mar 14:53:15 ntpd[17191]: Listen normally on 4 lo [::1]:123 2024-03-23T14:53:15.575 INFO:teuthology.orchestra.run.smithi173.stdout:23 Mar 14:53:15 ntpd[17191]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:7259%5]:123 2024-03-23T14:53:15.576 INFO:teuthology.orchestra.run.smithi173.stdout:23 Mar 14:53:15 ntpd[17191]: Listening on routing socket on fd #22 for interface updates 2024-03-23T14:53:16.546 INFO:teuthology.orchestra.run.smithi164.stderr:23 Mar 14:53:16 ntpd[17162]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-23T14:53:16.546 INFO:teuthology.orchestra.run.smithi164.stderr:23 Mar 14:53:16 ntpd[17162]: can't open /var/log/ntpstats/rawstats.20240323: Permission denied 2024-03-23T14:53:16.546 INFO:teuthology.orchestra.run.smithi164.stderr:23 Mar 14:53:16 ntpd[17162]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-23T14:53:16.546 INFO:teuthology.orchestra.run.smithi164.stderr:23 Mar 14:53:16 ntpd[17162]: can't open /var/log/ntpstats/rawstats.20240323: Permission denied 2024-03-23T14:53:16.546 INFO:teuthology.orchestra.run.smithi164.stderr:23 Mar 14:53:16 ntpd[17162]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-23T14:53:16.546 INFO:teuthology.orchestra.run.smithi164.stderr:23 Mar 14:53:16 ntpd[17162]: can't open /var/log/ntpstats/peerstats.20240323: Permission denied 2024-03-23T14:53:16.573 INFO:teuthology.orchestra.run.smithi173.stderr:23 Mar 14:53:16 ntpd[17191]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-23T14:53:16.573 INFO:teuthology.orchestra.run.smithi173.stderr:23 Mar 14:53:16 ntpd[17191]: can't open /var/log/ntpstats/rawstats.20240323: Permission denied 2024-03-23T14:53:16.573 INFO:teuthology.orchestra.run.smithi173.stderr:23 Mar 14:53:16 ntpd[17191]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-23T14:53:16.573 INFO:teuthology.orchestra.run.smithi173.stderr:23 Mar 14:53:16 ntpd[17191]: can't open /var/log/ntpstats/rawstats.20240323: Permission denied 2024-03-23T14:53:16.573 INFO:teuthology.orchestra.run.smithi173.stderr:23 Mar 14:53:16 ntpd[17191]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-23T14:53:16.573 INFO:teuthology.orchestra.run.smithi173.stderr:23 Mar 14:53:16 ntpd[17191]: can't open /var/log/ntpstats/peerstats.20240323: Permission denied 2024-03-23T14:53:18.546 INFO:teuthology.orchestra.run.smithi164.stderr:23 Mar 14:53:18 ntpd[17162]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-23T14:53:18.546 INFO:teuthology.orchestra.run.smithi164.stderr:23 Mar 14:53:18 ntpd[17162]: can't open /var/log/ntpstats/rawstats.20240323: Permission denied 2024-03-23T14:53:18.546 INFO:teuthology.orchestra.run.smithi164.stderr:23 Mar 14:53:18 ntpd[17162]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-23T14:53:18.547 INFO:teuthology.orchestra.run.smithi164.stderr:23 Mar 14:53:18 ntpd[17162]: can't open /var/log/ntpstats/rawstats.20240323: Permission denied 2024-03-23T14:53:18.547 INFO:teuthology.orchestra.run.smithi164.stderr:23 Mar 14:53:18 ntpd[17162]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-23T14:53:18.547 INFO:teuthology.orchestra.run.smithi164.stderr:23 Mar 14:53:18 ntpd[17162]: can't open /var/log/ntpstats/rawstats.20240323: Permission denied 2024-03-23T14:53:18.547 INFO:teuthology.orchestra.run.smithi164.stderr:23 Mar 14:53:18 ntpd[17162]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-23T14:53:18.547 INFO:teuthology.orchestra.run.smithi164.stderr:23 Mar 14:53:18 ntpd[17162]: can't open /var/log/ntpstats/peerstats.20240323: Permission denied 2024-03-23T14:53:18.547 INFO:teuthology.orchestra.run.smithi164.stderr:23 Mar 14:53:18 ntpd[17162]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-23T14:53:18.547 INFO:teuthology.orchestra.run.smithi164.stderr:23 Mar 14:53:18 ntpd[17162]: can't open /var/log/ntpstats/rawstats.20240323: Permission denied 2024-03-23T14:53:18.547 INFO:teuthology.orchestra.run.smithi164.stderr:23 Mar 14:53:18 ntpd[17162]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-23T14:53:18.547 INFO:teuthology.orchestra.run.smithi164.stderr:23 Mar 14:53:18 ntpd[17162]: can't open /var/log/ntpstats/peerstats.20240323: Permission denied 2024-03-23T14:53:18.573 INFO:teuthology.orchestra.run.smithi173.stderr:23 Mar 14:53:18 ntpd[17191]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-23T14:53:18.573 INFO:teuthology.orchestra.run.smithi173.stderr:23 Mar 14:53:18 ntpd[17191]: can't open /var/log/ntpstats/rawstats.20240323: Permission denied 2024-03-23T14:53:18.573 INFO:teuthology.orchestra.run.smithi173.stderr:23 Mar 14:53:18 ntpd[17191]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-23T14:53:18.573 INFO:teuthology.orchestra.run.smithi173.stderr:23 Mar 14:53:18 ntpd[17191]: can't open /var/log/ntpstats/rawstats.20240323: Permission denied 2024-03-23T14:53:18.573 INFO:teuthology.orchestra.run.smithi173.stderr:23 Mar 14:53:18 ntpd[17191]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-23T14:53:18.573 INFO:teuthology.orchestra.run.smithi173.stderr:23 Mar 14:53:18 ntpd[17191]: can't open /var/log/ntpstats/rawstats.20240323: Permission denied 2024-03-23T14:53:18.573 INFO:teuthology.orchestra.run.smithi173.stderr:23 Mar 14:53:18 ntpd[17191]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-23T14:53:18.573 INFO:teuthology.orchestra.run.smithi173.stderr:23 Mar 14:53:18 ntpd[17191]: can't open /var/log/ntpstats/peerstats.20240323: Permission denied 2024-03-23T14:53:19.546 INFO:teuthology.orchestra.run.smithi164.stderr:23 Mar 14:53:19 ntpd[17162]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-23T14:53:19.546 INFO:teuthology.orchestra.run.smithi164.stderr:23 Mar 14:53:19 ntpd[17162]: can't open /var/log/ntpstats/rawstats.20240323: Permission denied 2024-03-23T14:53:19.546 INFO:teuthology.orchestra.run.smithi164.stderr:23 Mar 14:53:19 ntpd[17162]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-23T14:53:19.546 INFO:teuthology.orchestra.run.smithi164.stderr:23 Mar 14:53:19 ntpd[17162]: can't open /var/log/ntpstats/rawstats.20240323: Permission denied 2024-03-23T14:53:19.546 INFO:teuthology.orchestra.run.smithi164.stderr:23 Mar 14:53:19 ntpd[17162]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-23T14:53:19.546 INFO:teuthology.orchestra.run.smithi164.stderr:23 Mar 14:53:19 ntpd[17162]: can't open /var/log/ntpstats/peerstats.20240323: Permission denied 2024-03-23T14:53:19.573 INFO:teuthology.orchestra.run.smithi173.stderr:23 Mar 14:53:19 ntpd[17191]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-23T14:53:19.573 INFO:teuthology.orchestra.run.smithi173.stderr:23 Mar 14:53:19 ntpd[17191]: can't open /var/log/ntpstats/rawstats.20240323: Permission denied 2024-03-23T14:53:19.573 INFO:teuthology.orchestra.run.smithi173.stderr:23 Mar 14:53:19 ntpd[17191]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-23T14:53:19.573 INFO:teuthology.orchestra.run.smithi173.stderr:23 Mar 14:53:19 ntpd[17191]: can't open /var/log/ntpstats/rawstats.20240323: Permission denied 2024-03-23T14:53:19.573 INFO:teuthology.orchestra.run.smithi173.stderr:23 Mar 14:53:19 ntpd[17191]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-23T14:53:19.573 INFO:teuthology.orchestra.run.smithi173.stderr:23 Mar 14:53:19 ntpd[17191]: can't open /var/log/ntpstats/peerstats.20240323: Permission denied 2024-03-23T14:53:20.546 INFO:teuthology.orchestra.run.smithi164.stderr:23 Mar 14:53:20 ntpd[17162]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-23T14:53:20.547 INFO:teuthology.orchestra.run.smithi164.stderr:23 Mar 14:53:20 ntpd[17162]: can't open /var/log/ntpstats/rawstats.20240323: Permission denied 2024-03-23T14:53:20.547 INFO:teuthology.orchestra.run.smithi164.stderr:23 Mar 14:53:20 ntpd[17162]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-23T14:53:20.547 INFO:teuthology.orchestra.run.smithi164.stderr:23 Mar 14:53:20 ntpd[17162]: can't open /var/log/ntpstats/rawstats.20240323: Permission denied 2024-03-23T14:53:20.547 INFO:teuthology.orchestra.run.smithi164.stderr:23 Mar 14:53:20 ntpd[17162]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-23T14:53:20.547 INFO:teuthology.orchestra.run.smithi164.stderr:23 Mar 14:53:20 ntpd[17162]: can't open /var/log/ntpstats/rawstats.20240323: Permission denied 2024-03-23T14:53:20.547 INFO:teuthology.orchestra.run.smithi164.stderr:23 Mar 14:53:20 ntpd[17162]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-23T14:53:20.547 INFO:teuthology.orchestra.run.smithi164.stderr:23 Mar 14:53:20 ntpd[17162]: can't open /var/log/ntpstats/rawstats.20240323: Permission denied 2024-03-23T14:53:20.547 INFO:teuthology.orchestra.run.smithi164.stderr:23 Mar 14:53:20 ntpd[17162]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-23T14:53:20.547 INFO:teuthology.orchestra.run.smithi164.stderr:23 Mar 14:53:20 ntpd[17162]: can't open /var/log/ntpstats/peerstats.20240323: Permission denied 2024-03-23T14:53:20.547 INFO:teuthology.orchestra.run.smithi164.stderr:23 Mar 14:53:20 ntpd[17162]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-23T14:53:20.547 INFO:teuthology.orchestra.run.smithi164.stderr:23 Mar 14:53:20 ntpd[17162]: can't open /var/log/ntpstats/rawstats.20240323: Permission denied 2024-03-23T14:53:20.547 INFO:teuthology.orchestra.run.smithi164.stderr:23 Mar 14:53:20 ntpd[17162]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-23T14:53:20.547 INFO:teuthology.orchestra.run.smithi164.stderr:23 Mar 14:53:20 ntpd[17162]: can't open /var/log/ntpstats/peerstats.20240323: Permission denied 2024-03-23T14:53:20.573 INFO:teuthology.orchestra.run.smithi173.stderr:23 Mar 14:53:20 ntpd[17191]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-23T14:53:20.573 INFO:teuthology.orchestra.run.smithi173.stderr:23 Mar 14:53:20 ntpd[17191]: can't open /var/log/ntpstats/rawstats.20240323: Permission denied 2024-03-23T14:53:20.573 INFO:teuthology.orchestra.run.smithi173.stderr:23 Mar 14:53:20 ntpd[17191]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-23T14:53:20.573 INFO:teuthology.orchestra.run.smithi173.stderr:23 Mar 14:53:20 ntpd[17191]: can't open /var/log/ntpstats/rawstats.20240323: Permission denied 2024-03-23T14:53:20.573 INFO:teuthology.orchestra.run.smithi173.stderr:23 Mar 14:53:20 ntpd[17191]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-23T14:53:20.573 INFO:teuthology.orchestra.run.smithi173.stderr:23 Mar 14:53:20 ntpd[17191]: can't open /var/log/ntpstats/rawstats.20240323: Permission denied 2024-03-23T14:53:20.573 INFO:teuthology.orchestra.run.smithi173.stderr:23 Mar 14:53:20 ntpd[17191]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-23T14:53:20.574 INFO:teuthology.orchestra.run.smithi173.stderr:23 Mar 14:53:20 ntpd[17191]: can't open /var/log/ntpstats/peerstats.20240323: Permission denied 2024-03-23T14:53:20.574 INFO:teuthology.orchestra.run.smithi173.stderr:23 Mar 14:53:20 ntpd[17191]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-23T14:53:20.574 INFO:teuthology.orchestra.run.smithi173.stderr:23 Mar 14:53:20 ntpd[17191]: can't open /var/log/ntpstats/rawstats.20240323: Permission denied 2024-03-23T14:53:20.574 INFO:teuthology.orchestra.run.smithi173.stderr:23 Mar 14:53:20 ntpd[17191]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-23T14:53:20.574 INFO:teuthology.orchestra.run.smithi173.stderr:23 Mar 14:53:20 ntpd[17191]: can't open /var/log/ntpstats/peerstats.20240323: Permission denied 2024-03-23T14:53:21.546 INFO:teuthology.orchestra.run.smithi164.stderr:23 Mar 14:53:21 ntpd[17162]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-23T14:53:21.546 INFO:teuthology.orchestra.run.smithi164.stderr:23 Mar 14:53:21 ntpd[17162]: can't open /var/log/ntpstats/rawstats.20240323: Permission denied 2024-03-23T14:53:21.546 INFO:teuthology.orchestra.run.smithi164.stderr:23 Mar 14:53:21 ntpd[17162]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-23T14:53:21.546 INFO:teuthology.orchestra.run.smithi164.stderr:23 Mar 14:53:21 ntpd[17162]: can't open /var/log/ntpstats/rawstats.20240323: Permission denied 2024-03-23T14:53:21.546 INFO:teuthology.orchestra.run.smithi164.stderr:23 Mar 14:53:21 ntpd[17162]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-23T14:53:21.546 INFO:teuthology.orchestra.run.smithi164.stderr:23 Mar 14:53:21 ntpd[17162]: can't open /var/log/ntpstats/peerstats.20240323: Permission denied 2024-03-23T14:53:21.573 INFO:teuthology.orchestra.run.smithi173.stderr:23 Mar 14:53:21 ntpd[17191]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-23T14:53:21.573 INFO:teuthology.orchestra.run.smithi173.stderr:23 Mar 14:53:21 ntpd[17191]: can't open /var/log/ntpstats/rawstats.20240323: Permission denied 2024-03-23T14:53:21.573 INFO:teuthology.orchestra.run.smithi173.stderr:23 Mar 14:53:21 ntpd[17191]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-23T14:53:21.573 INFO:teuthology.orchestra.run.smithi173.stderr:23 Mar 14:53:21 ntpd[17191]: can't open /var/log/ntpstats/rawstats.20240323: Permission denied 2024-03-23T14:53:21.573 INFO:teuthology.orchestra.run.smithi173.stderr:23 Mar 14:53:21 ntpd[17191]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-23T14:53:21.573 INFO:teuthology.orchestra.run.smithi173.stderr:23 Mar 14:53:21 ntpd[17191]: can't open /var/log/ntpstats/peerstats.20240323: Permission denied 2024-03-23T14:53:22.546 INFO:teuthology.orchestra.run.smithi164.stderr:23 Mar 14:53:22 ntpd[17162]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-23T14:53:22.546 INFO:teuthology.orchestra.run.smithi164.stderr:23 Mar 14:53:22 ntpd[17162]: can't open /var/log/ntpstats/rawstats.20240323: Permission denied 2024-03-23T14:53:22.546 INFO:teuthology.orchestra.run.smithi164.stderr:23 Mar 14:53:22 ntpd[17162]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-23T14:53:22.547 INFO:teuthology.orchestra.run.smithi164.stderr:23 Mar 14:53:22 ntpd[17162]: can't open /var/log/ntpstats/rawstats.20240323: Permission denied 2024-03-23T14:53:22.547 INFO:teuthology.orchestra.run.smithi164.stderr:23 Mar 14:53:22 ntpd[17162]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-23T14:53:22.547 INFO:teuthology.orchestra.run.smithi164.stderr:23 Mar 14:53:22 ntpd[17162]: can't open /var/log/ntpstats/rawstats.20240323: Permission denied 2024-03-23T14:53:22.547 INFO:teuthology.orchestra.run.smithi164.stderr:23 Mar 14:53:22 ntpd[17162]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-23T14:53:22.547 INFO:teuthology.orchestra.run.smithi164.stderr:23 Mar 14:53:22 ntpd[17162]: can't open /var/log/ntpstats/peerstats.20240323: Permission denied 2024-03-23T14:53:22.547 INFO:teuthology.orchestra.run.smithi164.stderr:23 Mar 14:53:22 ntpd[17162]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-23T14:53:22.547 INFO:teuthology.orchestra.run.smithi164.stderr:23 Mar 14:53:22 ntpd[17162]: can't open /var/log/ntpstats/rawstats.20240323: Permission denied 2024-03-23T14:53:22.547 INFO:teuthology.orchestra.run.smithi164.stderr:23 Mar 14:53:22 ntpd[17162]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-23T14:53:22.547 INFO:teuthology.orchestra.run.smithi164.stderr:23 Mar 14:53:22 ntpd[17162]: can't open /var/log/ntpstats/peerstats.20240323: Permission denied 2024-03-23T14:53:22.547 INFO:teuthology.orchestra.run.smithi164.stdout:23 Mar 14:53:22 ntpd[17162]: ntpd: time slew -0.002097 s 2024-03-23T14:53:22.548 INFO:teuthology.orchestra.run.smithi164.stdout:ntpd: time slew -0.002097s 2024-03-23T14:53:22.548 INFO:teuthology.orchestra.run.smithi164.stderr:23 Mar 14:53:22 ntpd[17162]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-03-23T14:53:22.548 INFO:teuthology.orchestra.run.smithi164.stderr:23 Mar 14:53:22 ntpd[17162]: can't open /var/log/ntpstats/loopstats.20240323: Permission denied 2024-03-23T14:53:22.573 INFO:teuthology.orchestra.run.smithi173.stderr:23 Mar 14:53:22 ntpd[17191]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-23T14:53:22.573 INFO:teuthology.orchestra.run.smithi173.stderr:23 Mar 14:53:22 ntpd[17191]: can't open /var/log/ntpstats/rawstats.20240323: Permission denied 2024-03-23T14:53:22.573 INFO:teuthology.orchestra.run.smithi173.stderr:23 Mar 14:53:22 ntpd[17191]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-23T14:53:22.573 INFO:teuthology.orchestra.run.smithi173.stderr:23 Mar 14:53:22 ntpd[17191]: can't open /var/log/ntpstats/rawstats.20240323: Permission denied 2024-03-23T14:53:22.573 INFO:teuthology.orchestra.run.smithi173.stderr:23 Mar 14:53:22 ntpd[17191]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-23T14:53:22.574 INFO:teuthology.orchestra.run.smithi173.stderr:23 Mar 14:53:22 ntpd[17191]: can't open /var/log/ntpstats/rawstats.20240323: Permission denied 2024-03-23T14:53:22.574 INFO:teuthology.orchestra.run.smithi173.stderr:23 Mar 14:53:22 ntpd[17191]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-23T14:53:22.574 INFO:teuthology.orchestra.run.smithi173.stderr:23 Mar 14:53:22 ntpd[17191]: can't open /var/log/ntpstats/peerstats.20240323: Permission denied 2024-03-23T14:53:22.574 INFO:teuthology.orchestra.run.smithi173.stdout:23 Mar 14:53:22 ntpd[17191]: ntpd: time slew +0.000380 s 2024-03-23T14:53:22.574 INFO:teuthology.orchestra.run.smithi173.stdout:ntpd: time slew +0.000380s 2024-03-23T14:53:22.575 INFO:teuthology.orchestra.run.smithi173.stderr:23 Mar 14:53:22 ntpd[17191]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-03-23T14:53:22.575 INFO:teuthology.orchestra.run.smithi173.stderr:23 Mar 14:53:22 ntpd[17191]: can't open /var/log/ntpstats/loopstats.20240323: Permission denied 2024-03-23T14:53:22.605 INFO:teuthology.orchestra.run.smithi164.stdout: remote refid st t when poll reach delay offset jitter 2024-03-23T14:53:22.605 INFO:teuthology.orchestra.run.smithi164.stdout:============================================================================== 2024-03-23T14:53:22.605 INFO:teuthology.orchestra.run.smithi164.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-23T14:53:22.605 INFO:teuthology.orchestra.run.smithi164.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-23T14:53:22.605 INFO:teuthology.orchestra.run.smithi164.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-23T14:53:22.605 INFO:teuthology.orchestra.run.smithi164.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-23T14:53:22.635 INFO:teuthology.orchestra.run.smithi173.stdout: remote refid st t when poll reach delay offset jitter 2024-03-23T14:53:22.635 INFO:teuthology.orchestra.run.smithi173.stdout:============================================================================== 2024-03-23T14:53:22.635 INFO:teuthology.orchestra.run.smithi173.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-23T14:53:22.635 INFO:teuthology.orchestra.run.smithi173.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-23T14:53:22.635 INFO:teuthology.orchestra.run.smithi173.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-23T14:53:22.635 INFO:teuthology.orchestra.run.smithi173.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-23T14:53:22.635 INFO:teuthology.run_tasks:Running task install... 2024-03-23T14:53:22.644 DEBUG:teuthology.task.install:project ceph 2024-03-23T14:53:22.645 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '21b3bd672163092b4e542f772797fcba02d3d4de'}} 2024-03-23T14:53:22.645 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '21b3bd672163092b4e542f772797fcba02d3d4de'} 2024-03-23T14:53:22.645 INFO:teuthology.task.install:Using flavor: default 2024-03-23T14:53:22.652 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-03-23T14:53:22.652 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-03-23T14:53:22.652 DEBUG:teuthology.orchestra.run.smithi164:> sudo apt-key list | grep Ceph 2024-03-23T14:53:22.652 DEBUG:teuthology.orchestra.run.smithi173:> sudo apt-key list | grep Ceph 2024-03-23T14:53:22.709 INFO:teuthology.orchestra.run.smithi164.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-03-23T14:53:22.735 INFO:teuthology.orchestra.run.smithi164.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-03-23T14:53:22.735 INFO:teuthology.orchestra.run.smithi164.stdout:uid [ unknown] Ceph.com (release key) 2024-03-23T14:53:22.737 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-03-23T14:53:22.737 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2024-03-23T14:53:22.737 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=21b3bd672163092b4e542f772797fcba02d3d4de 2024-03-23T14:53:22.741 INFO:teuthology.orchestra.run.smithi173.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-03-23T14:53:22.763 INFO:teuthology.orchestra.run.smithi173.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-03-23T14:53:22.763 INFO:teuthology.orchestra.run.smithi173.stdout:uid [ unknown] Ceph.com (release key) 2024-03-23T14:53:22.764 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-03-23T14:53:22.764 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2024-03-23T14:53:22.764 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=21b3bd672163092b4e542f772797fcba02d3d4de 2024-03-23T14:53:22.960 INFO:teuthology.task.install.deb:Pulling from https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240323.003144-squid/21b3bd672163092b4e542f772797fcba02d3d4de/ubuntu/jammy/flavors/default/ 2024-03-23T14:53:22.961 INFO:teuthology.task.install.deb:Package version is 19.0.0-1855-g21b3bd67-1jammy 2024-03-23T14:53:22.981 INFO:teuthology.task.install.deb:Pulling from https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240323.003144-squid/21b3bd672163092b4e542f772797fcba02d3d4de/ubuntu/jammy/flavors/default/ 2024-03-23T14:53:22.981 INFO:teuthology.task.install.deb:Package version is 19.0.0-1855-g21b3bd67-1jammy 2024-03-23T14:53:23.070 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-23T14:53:23.070 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-03-23T14:53:23.084 DEBUG:teuthology.orchestra.run.smithi164:> sudo apt-get update 2024-03-23T14:53:23.089 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-03-23T14:53:23.089 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-03-23T14:53:23.103 DEBUG:teuthology.orchestra.run.smithi173:> sudo apt-get update 2024-03-23T14:53:23.281 INFO:teuthology.orchestra.run.smithi164.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-03-23T14:53:23.282 INFO:teuthology.orchestra.run.smithi164.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-03-23T14:53:23.282 INFO:teuthology.orchestra.run.smithi173.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-03-23T14:53:23.295 INFO:teuthology.orchestra.run.smithi173.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-03-23T14:53:23.296 INFO:teuthology.orchestra.run.smithi164.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-03-23T14:53:23.315 INFO:teuthology.orchestra.run.smithi173.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-03-23T14:53:23.317 INFO:teuthology.orchestra.run.smithi164.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-03-23T14:53:23.323 INFO:teuthology.orchestra.run.smithi173.stdout:Ign:4 https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240323.003144-squid/21b3bd672163092b4e542f772797fcba02d3d4de/ubuntu/jammy/flavors/default jammy InRelease 2024-03-23T14:53:23.344 INFO:teuthology.orchestra.run.smithi173.stdout:Get:5 https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240323.003144-squid/21b3bd672163092b4e542f772797fcba02d3d4de/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-03-23T14:53:23.366 INFO:teuthology.orchestra.run.smithi173.stdout:Ign:6 https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240323.003144-squid/21b3bd672163092b4e542f772797fcba02d3d4de/ubuntu/jammy/flavors/default jammy Release.gpg 2024-03-23T14:53:23.370 INFO:teuthology.orchestra.run.smithi164.stdout:Ign:5 https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240323.003144-squid/21b3bd672163092b4e542f772797fcba02d3d4de/ubuntu/jammy/flavors/default jammy InRelease 2024-03-23T14:53:23.387 INFO:teuthology.orchestra.run.smithi173.stdout:Get:7 https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240323.003144-squid/21b3bd672163092b4e542f772797fcba02d3d4de/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,424 B] 2024-03-23T14:53:23.394 INFO:teuthology.orchestra.run.smithi173.stdout:Hit:8 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-03-23T14:53:23.407 INFO:teuthology.orchestra.run.smithi164.stdout:Get:6 https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240323.003144-squid/21b3bd672163092b4e542f772797fcba02d3d4de/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-03-23T14:53:23.414 INFO:teuthology.orchestra.run.smithi173.stdout:Get:9 https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240323.003144-squid/21b3bd672163092b4e542f772797fcba02d3d4de/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.8 kB] 2024-03-23T14:53:23.444 INFO:teuthology.orchestra.run.smithi164.stdout:Ign:7 https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240323.003144-squid/21b3bd672163092b4e542f772797fcba02d3d4de/ubuntu/jammy/flavors/default jammy Release.gpg 2024-03-23T14:53:23.480 INFO:teuthology.orchestra.run.smithi164.stdout:Get:8 https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240323.003144-squid/21b3bd672163092b4e542f772797fcba02d3d4de/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.8 kB] 2024-03-23T14:53:23.557 INFO:teuthology.orchestra.run.smithi164.stdout:Get:9 https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240323.003144-squid/21b3bd672163092b4e542f772797fcba02d3d4de/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,424 B] 2024-03-23T14:53:23.637 INFO:teuthology.orchestra.run.smithi164.stdout:Fetched 28.9 kB in 0s (79.1 kB/s) 2024-03-23T14:53:23.662 INFO:teuthology.orchestra.run.smithi173.stdout:Fetched 28.9 kB in 0s (77.9 kB/s) 2024-03-23T14:53:24.592 INFO:teuthology.orchestra.run.smithi164.stdout:Reading package lists... 2024-03-23T14:53:24.608 DEBUG:teuthology.orchestra.run.smithi164:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.0.0-1855-g21b3bd67-1jammy cephadm=19.0.0-1855-g21b3bd67-1jammy ceph-mds=19.0.0-1855-g21b3bd67-1jammy ceph-mgr=19.0.0-1855-g21b3bd67-1jammy ceph-common=19.0.0-1855-g21b3bd67-1jammy ceph-fuse=19.0.0-1855-g21b3bd67-1jammy ceph-test=19.0.0-1855-g21b3bd67-1jammy ceph-volume=19.0.0-1855-g21b3bd67-1jammy radosgw=19.0.0-1855-g21b3bd67-1jammy python3-rados=19.0.0-1855-g21b3bd67-1jammy python3-rgw=19.0.0-1855-g21b3bd67-1jammy python3-cephfs=19.0.0-1855-g21b3bd67-1jammy python3-rbd=19.0.0-1855-g21b3bd67-1jammy libcephfs2=19.0.0-1855-g21b3bd67-1jammy libcephfs-dev=19.0.0-1855-g21b3bd67-1jammy librados2=19.0.0-1855-g21b3bd67-1jammy librbd1=19.0.0-1855-g21b3bd67-1jammy rbd-fuse=19.0.0-1855-g21b3bd67-1jammy python3-cephfs=19.0.0-1855-g21b3bd67-1jammy cephfs-shell=19.0.0-1855-g21b3bd67-1jammy cephfs-top=19.0.0-1855-g21b3bd67-1jammy cephfs-mirror=19.0.0-1855-g21b3bd67-1jammy 2024-03-23T14:53:24.644 INFO:teuthology.orchestra.run.smithi173.stdout:Reading package lists... 2024-03-23T14:53:24.660 DEBUG:teuthology.orchestra.run.smithi173:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.0.0-1855-g21b3bd67-1jammy cephadm=19.0.0-1855-g21b3bd67-1jammy ceph-mds=19.0.0-1855-g21b3bd67-1jammy ceph-mgr=19.0.0-1855-g21b3bd67-1jammy ceph-common=19.0.0-1855-g21b3bd67-1jammy ceph-fuse=19.0.0-1855-g21b3bd67-1jammy ceph-test=19.0.0-1855-g21b3bd67-1jammy ceph-volume=19.0.0-1855-g21b3bd67-1jammy radosgw=19.0.0-1855-g21b3bd67-1jammy python3-rados=19.0.0-1855-g21b3bd67-1jammy python3-rgw=19.0.0-1855-g21b3bd67-1jammy python3-cephfs=19.0.0-1855-g21b3bd67-1jammy python3-rbd=19.0.0-1855-g21b3bd67-1jammy libcephfs2=19.0.0-1855-g21b3bd67-1jammy libcephfs-dev=19.0.0-1855-g21b3bd67-1jammy librados2=19.0.0-1855-g21b3bd67-1jammy librbd1=19.0.0-1855-g21b3bd67-1jammy rbd-fuse=19.0.0-1855-g21b3bd67-1jammy python3-cephfs=19.0.0-1855-g21b3bd67-1jammy cephfs-shell=19.0.0-1855-g21b3bd67-1jammy cephfs-top=19.0.0-1855-g21b3bd67-1jammy cephfs-mirror=19.0.0-1855-g21b3bd67-1jammy 2024-03-23T14:53:24.673 INFO:teuthology.orchestra.run.smithi164.stdout:Reading package lists... 2024-03-23T14:53:24.719 INFO:teuthology.orchestra.run.smithi173.stdout:Reading package lists... 2024-03-23T14:53:24.869 INFO:teuthology.orchestra.run.smithi164.stdout:Building dependency tree... 2024-03-23T14:53:24.869 INFO:teuthology.orchestra.run.smithi164.stdout:Reading state information... 2024-03-23T14:53:24.911 INFO:teuthology.orchestra.run.smithi173.stdout:Building dependency tree... 2024-03-23T14:53:24.912 INFO:teuthology.orchestra.run.smithi173.stdout:Reading state information... 2024-03-23T14:53:25.053 INFO:teuthology.orchestra.run.smithi164.stdout:The following packages were automatically installed and are no longer required: 2024-03-23T14:53:25.053 INFO:teuthology.orchestra.run.smithi164.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-03-23T14:53:25.053 INFO:teuthology.orchestra.run.smithi164.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-03-23T14:53:25.054 INFO:teuthology.orchestra.run.smithi164.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-03-23T14:53:25.054 INFO:teuthology.orchestra.run.smithi164.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-03-23T14:53:25.054 INFO:teuthology.orchestra.run.smithi164.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-03-23T14:53:25.054 INFO:teuthology.orchestra.run.smithi164.stdout:Use 'sudo apt autoremove' to remove them. 2024-03-23T14:53:25.054 INFO:teuthology.orchestra.run.smithi164.stdout:The following additional packages will be installed: 2024-03-23T14:53:25.055 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-03-23T14:53:25.055 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-03-23T14:53:25.055 INFO:teuthology.orchestra.run.smithi164.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-03-23T14:53:25.055 INFO:teuthology.orchestra.run.smithi164.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-03-23T14:53:25.055 INFO:teuthology.orchestra.run.smithi164.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-03-23T14:53:25.055 INFO:teuthology.orchestra.run.smithi164.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-03-23T14:53:25.056 INFO:teuthology.orchestra.run.smithi164.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-03-23T14:53:25.056 INFO:teuthology.orchestra.run.smithi164.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-03-23T14:53:25.056 INFO:teuthology.orchestra.run.smithi164.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-03-23T14:53:25.056 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-google-auth 2024-03-23T14:53:25.056 INFO:teuthology.orchestra.run.smithi164.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2024-03-23T14:53:25.057 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-03-23T14:53:25.057 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-03-23T14:53:25.057 INFO:teuthology.orchestra.run.smithi164.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-03-23T14:53:25.057 INFO:teuthology.orchestra.run.smithi164.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-03-23T14:53:25.057 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-03-23T14:53:25.057 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-03-23T14:53:25.058 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2024-03-23T14:53:25.058 INFO:teuthology.orchestra.run.smithi164.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2024-03-23T14:53:25.058 INFO:teuthology.orchestra.run.smithi164.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-03-23T14:53:25.058 INFO:teuthology.orchestra.run.smithi164.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2024-03-23T14:53:25.058 INFO:teuthology.orchestra.run.smithi164.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2024-03-23T14:53:25.059 INFO:teuthology.orchestra.run.smithi164.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2024-03-23T14:53:25.059 INFO:teuthology.orchestra.run.smithi164.stdout: qttranslations5-l10n socat unzip xmlstarlet zip 2024-03-23T14:53:25.059 INFO:teuthology.orchestra.run.smithi164.stdout:Suggested packages: 2024-03-23T14:53:25.059 INFO:teuthology.orchestra.run.smithi164.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-03-23T14:53:25.059 INFO:teuthology.orchestra.run.smithi164.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-03-23T14:53:25.059 INFO:teuthology.orchestra.run.smithi164.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-03-23T14:53:25.060 INFO:teuthology.orchestra.run.smithi164.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-03-23T14:53:25.060 INFO:teuthology.orchestra.run.smithi164.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-03-23T14:53:25.060 INFO:teuthology.orchestra.run.smithi164.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-03-23T14:53:25.060 INFO:teuthology.orchestra.run.smithi164.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-03-23T14:53:25.060 INFO:teuthology.orchestra.run.smithi164.stdout:Recommended packages: 2024-03-23T14:53:25.060 INFO:teuthology.orchestra.run.smithi164.stdout: btrfs-tools 2024-03-23T14:53:25.092 INFO:teuthology.orchestra.run.smithi173.stdout:The following packages were automatically installed and are no longer required: 2024-03-23T14:53:25.092 INFO:teuthology.orchestra.run.smithi173.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-03-23T14:53:25.092 INFO:teuthology.orchestra.run.smithi173.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-03-23T14:53:25.092 INFO:teuthology.orchestra.run.smithi173.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-03-23T14:53:25.093 INFO:teuthology.orchestra.run.smithi173.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-03-23T14:53:25.094 INFO:teuthology.orchestra.run.smithi173.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-03-23T14:53:25.095 INFO:teuthology.orchestra.run.smithi173.stdout:Use 'sudo apt autoremove' to remove them. 2024-03-23T14:53:25.095 INFO:teuthology.orchestra.run.smithi173.stdout:The following additional packages will be installed: 2024-03-23T14:53:25.095 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-03-23T14:53:25.095 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-03-23T14:53:25.095 INFO:teuthology.orchestra.run.smithi173.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-03-23T14:53:25.095 INFO:teuthology.orchestra.run.smithi173.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-03-23T14:53:25.095 INFO:teuthology.orchestra.run.smithi173.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-03-23T14:53:25.095 INFO:teuthology.orchestra.run.smithi173.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-03-23T14:53:25.095 INFO:teuthology.orchestra.run.smithi173.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-03-23T14:53:25.095 INFO:teuthology.orchestra.run.smithi173.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-03-23T14:53:25.095 INFO:teuthology.orchestra.run.smithi173.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-03-23T14:53:25.095 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-google-auth 2024-03-23T14:53:25.096 INFO:teuthology.orchestra.run.smithi173.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2024-03-23T14:53:25.096 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-03-23T14:53:25.096 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-03-23T14:53:25.096 INFO:teuthology.orchestra.run.smithi173.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-03-23T14:53:25.096 INFO:teuthology.orchestra.run.smithi173.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-03-23T14:53:25.096 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-03-23T14:53:25.096 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-03-23T14:53:25.096 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2024-03-23T14:53:25.096 INFO:teuthology.orchestra.run.smithi173.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2024-03-23T14:53:25.097 INFO:teuthology.orchestra.run.smithi173.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-03-23T14:53:25.097 INFO:teuthology.orchestra.run.smithi173.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2024-03-23T14:53:25.097 INFO:teuthology.orchestra.run.smithi173.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2024-03-23T14:53:25.097 INFO:teuthology.orchestra.run.smithi173.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2024-03-23T14:53:25.097 INFO:teuthology.orchestra.run.smithi173.stdout: qttranslations5-l10n socat unzip xmlstarlet zip 2024-03-23T14:53:25.098 INFO:teuthology.orchestra.run.smithi173.stdout:Suggested packages: 2024-03-23T14:53:25.098 INFO:teuthology.orchestra.run.smithi173.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-03-23T14:53:25.098 INFO:teuthology.orchestra.run.smithi173.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-03-23T14:53:25.099 INFO:teuthology.orchestra.run.smithi173.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-03-23T14:53:25.099 INFO:teuthology.orchestra.run.smithi173.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-03-23T14:53:25.099 INFO:teuthology.orchestra.run.smithi173.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-03-23T14:53:25.099 INFO:teuthology.orchestra.run.smithi173.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-03-23T14:53:25.099 INFO:teuthology.orchestra.run.smithi173.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-03-23T14:53:25.100 INFO:teuthology.orchestra.run.smithi173.stdout:Recommended packages: 2024-03-23T14:53:25.100 INFO:teuthology.orchestra.run.smithi173.stdout: btrfs-tools 2024-03-23T14:53:25.161 INFO:teuthology.orchestra.run.smithi164.stdout:The following NEW packages will be installed: 2024-03-23T14:53:25.161 INFO:teuthology.orchestra.run.smithi164.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-03-23T14:53:25.161 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-03-23T14:53:25.161 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-03-23T14:53:25.162 INFO:teuthology.orchestra.run.smithi164.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2024-03-23T14:53:25.163 INFO:teuthology.orchestra.run.smithi164.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-03-23T14:53:25.163 INFO:teuthology.orchestra.run.smithi164.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2024-03-23T14:53:25.163 INFO:teuthology.orchestra.run.smithi164.stdout: libradosstriper1 librbd1 librdkafka1 libreadline-dev librgw2 2024-03-23T14:53:25.163 INFO:teuthology.orchestra.run.smithi164.stdout: libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 2024-03-23T14:53:25.163 INFO:teuthology.orchestra.run.smithi164.stdout: luarocks nvme-cli pkg-config python-asyncssh-doc python-babel-localedata 2024-03-23T14:53:25.163 INFO:teuthology.orchestra.run.smithi164.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-03-23T14:53:25.163 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-03-23T14:53:25.163 INFO:teuthology.orchestra.run.smithi164.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2024-03-23T14:53:25.163 INFO:teuthology.orchestra.run.smithi164.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2024-03-23T14:53:25.163 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-03-23T14:53:25.163 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-03-23T14:53:25.163 INFO:teuthology.orchestra.run.smithi164.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-03-23T14:53:25.163 INFO:teuthology.orchestra.run.smithi164.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-03-23T14:53:25.164 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-03-23T14:53:25.164 INFO:teuthology.orchestra.run.smithi164.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-03-23T14:53:25.164 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2024-03-23T14:53:25.164 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2024-03-23T14:53:25.164 INFO:teuthology.orchestra.run.smithi164.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-03-23T14:53:25.164 INFO:teuthology.orchestra.run.smithi164.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-03-23T14:53:25.164 INFO:teuthology.orchestra.run.smithi164.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-03-23T14:53:25.164 INFO:teuthology.orchestra.run.smithi164.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-03-23T14:53:25.164 INFO:teuthology.orchestra.run.smithi164.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-03-23T14:53:25.164 INFO:teuthology.orchestra.run.smithi164.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-03-23T14:53:25.164 INFO:teuthology.orchestra.run.smithi164.stdout: socat unzip xmlstarlet zip 2024-03-23T14:53:25.202 INFO:teuthology.orchestra.run.smithi173.stdout:The following NEW packages will be installed: 2024-03-23T14:53:25.203 INFO:teuthology.orchestra.run.smithi173.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-03-23T14:53:25.203 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-03-23T14:53:25.203 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-03-23T14:53:25.203 INFO:teuthology.orchestra.run.smithi173.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2024-03-23T14:53:25.204 INFO:teuthology.orchestra.run.smithi173.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-03-23T14:53:25.204 INFO:teuthology.orchestra.run.smithi173.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2024-03-23T14:53:25.204 INFO:teuthology.orchestra.run.smithi173.stdout: libradosstriper1 librbd1 librdkafka1 libreadline-dev librgw2 2024-03-23T14:53:25.204 INFO:teuthology.orchestra.run.smithi173.stdout: libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 2024-03-23T14:53:25.205 INFO:teuthology.orchestra.run.smithi173.stdout: luarocks nvme-cli pkg-config python-asyncssh-doc python-babel-localedata 2024-03-23T14:53:25.205 INFO:teuthology.orchestra.run.smithi173.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-03-23T14:53:25.205 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-03-23T14:53:25.205 INFO:teuthology.orchestra.run.smithi173.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2024-03-23T14:53:25.205 INFO:teuthology.orchestra.run.smithi173.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2024-03-23T14:53:25.205 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-03-23T14:53:25.205 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-03-23T14:53:25.205 INFO:teuthology.orchestra.run.smithi173.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-03-23T14:53:25.205 INFO:teuthology.orchestra.run.smithi173.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-03-23T14:53:25.205 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-03-23T14:53:25.205 INFO:teuthology.orchestra.run.smithi173.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-03-23T14:53:25.205 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2024-03-23T14:53:25.205 INFO:teuthology.orchestra.run.smithi173.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2024-03-23T14:53:25.206 INFO:teuthology.orchestra.run.smithi173.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-03-23T14:53:25.206 INFO:teuthology.orchestra.run.smithi173.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-03-23T14:53:25.206 INFO:teuthology.orchestra.run.smithi173.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-03-23T14:53:25.206 INFO:teuthology.orchestra.run.smithi173.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-03-23T14:53:25.206 INFO:teuthology.orchestra.run.smithi173.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-03-23T14:53:25.206 INFO:teuthology.orchestra.run.smithi173.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-03-23T14:53:25.206 INFO:teuthology.orchestra.run.smithi173.stdout: socat unzip xmlstarlet zip 2024-03-23T14:53:25.288 INFO:teuthology.orchestra.run.smithi164.stdout:0 upgraded, 124 newly installed, 0 to remove and 285 not upgraded. 2024-03-23T14:53:25.288 INFO:teuthology.orchestra.run.smithi164.stdout:Need to get 178 MB of archives. 2024-03-23T14:53:25.288 INFO:teuthology.orchestra.run.smithi164.stdout:After this operation, 736 MB of additional disk space will be used. 2024-03-23T14:53:25.288 INFO:teuthology.orchestra.run.smithi164.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2024-03-23T14:53:25.306 INFO:teuthology.orchestra.run.smithi164.stdout:Get:2 https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240323.003144-squid/21b3bd672163092b4e542f772797fcba02d3d4de/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.0.0-1855-g21b3bd67-1jammy [3,545 kB] 2024-03-23T14:53:25.338 INFO:teuthology.orchestra.run.smithi164.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-03-23T14:53:25.343 INFO:teuthology.orchestra.run.smithi173.stdout:0 upgraded, 124 newly installed, 0 to remove and 285 not upgraded. 2024-03-23T14:53:25.343 INFO:teuthology.orchestra.run.smithi173.stdout:Need to get 178 MB of archives. 2024-03-23T14:53:25.344 INFO:teuthology.orchestra.run.smithi173.stdout:After this operation, 736 MB of additional disk space will be used. 2024-03-23T14:53:25.344 INFO:teuthology.orchestra.run.smithi173.stdout:Get:1 https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240323.003144-squid/21b3bd672163092b4e542f772797fcba02d3d4de/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.0.0-1855-g21b3bd67-1jammy [3,545 kB] 2024-03-23T14:53:25.366 INFO:teuthology.orchestra.run.smithi164.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2024-03-23T14:53:25.392 INFO:teuthology.orchestra.run.smithi173.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2024-03-23T14:53:25.475 INFO:teuthology.orchestra.run.smithi173.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-03-23T14:53:25.489 INFO:teuthology.orchestra.run.smithi164.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-03-23T14:53:25.520 INFO:teuthology.orchestra.run.smithi173.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2024-03-23T14:53:25.552 INFO:teuthology.orchestra.run.smithi164.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-03-23T14:53:25.561 INFO:teuthology.orchestra.run.smithi164.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-03-23T14:53:25.597 INFO:teuthology.orchestra.run.smithi164.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-03-23T14:53:25.653 INFO:teuthology.orchestra.run.smithi164.stdout:Get:9 https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240323.003144-squid/21b3bd672163092b4e542f772797fcba02d3d4de/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.0.0-1855-g21b3bd67-1jammy [3,210 kB] 2024-03-23T14:53:25.716 INFO:teuthology.orchestra.run.smithi173.stdout:Get:5 https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240323.003144-squid/21b3bd672163092b4e542f772797fcba02d3d4de/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.0.0-1855-g21b3bd67-1jammy [3,210 kB] 2024-03-23T14:53:25.717 INFO:teuthology.orchestra.run.smithi164.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-03-23T14:53:25.723 INFO:teuthology.orchestra.run.smithi173.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-03-23T14:53:25.724 INFO:teuthology.orchestra.run.smithi164.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-03-23T14:53:25.746 INFO:teuthology.orchestra.run.smithi164.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-03-23T14:53:25.752 INFO:teuthology.orchestra.run.smithi164.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-03-23T14:53:25.753 INFO:teuthology.orchestra.run.smithi164.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-03-23T14:53:25.770 INFO:teuthology.orchestra.run.smithi164.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-03-23T14:53:25.773 INFO:teuthology.orchestra.run.smithi164.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-03-23T14:53:25.818 INFO:teuthology.orchestra.run.smithi173.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-03-23T14:53:25.832 INFO:teuthology.orchestra.run.smithi173.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-03-23T14:53:25.844 INFO:teuthology.orchestra.run.smithi164.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-03-23T14:53:25.846 INFO:teuthology.orchestra.run.smithi164.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-03-23T14:53:25.847 INFO:teuthology.orchestra.run.smithi164.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-03-23T14:53:25.851 INFO:teuthology.orchestra.run.smithi164.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-03-23T14:53:25.855 INFO:teuthology.orchestra.run.smithi164.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-03-23T14:53:25.859 INFO:teuthology.orchestra.run.smithi164.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-03-23T14:53:25.860 INFO:teuthology.orchestra.run.smithi164.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-03-23T14:53:25.860 INFO:teuthology.orchestra.run.smithi164.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-03-23T14:53:25.862 INFO:teuthology.orchestra.run.smithi164.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-03-23T14:53:25.864 INFO:teuthology.orchestra.run.smithi164.stdout:Get:26 https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240323.003144-squid/21b3bd672163092b4e542f772797fcba02d3d4de/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.0.0-1855-g21b3bd67-1jammy [711 kB] 2024-03-23T14:53:25.884 INFO:teuthology.orchestra.run.smithi173.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-03-23T14:53:25.900 INFO:teuthology.orchestra.run.smithi164.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-03-23T14:53:25.901 INFO:teuthology.orchestra.run.smithi164.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-03-23T14:53:25.901 INFO:teuthology.orchestra.run.smithi164.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-03-23T14:53:25.902 INFO:teuthology.orchestra.run.smithi164.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-03-23T14:53:25.903 INFO:teuthology.orchestra.run.smithi164.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-03-23T14:53:25.904 INFO:teuthology.orchestra.run.smithi164.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-03-23T14:53:25.909 INFO:teuthology.orchestra.run.smithi164.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-03-23T14:53:25.910 INFO:teuthology.orchestra.run.smithi164.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-03-23T14:53:25.911 INFO:teuthology.orchestra.run.smithi164.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-03-23T14:53:25.918 INFO:teuthology.orchestra.run.smithi164.stdout:Get:36 https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240323.003144-squid/21b3bd672163092b4e542f772797fcba02d3d4de/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.0.0-1855-g21b3bd67-1jammy [356 kB] 2024-03-23T14:53:25.934 INFO:teuthology.orchestra.run.smithi164.stdout:Get:37 https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240323.003144-squid/21b3bd672163092b4e542f772797fcba02d3d4de/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.0.0-1855-g21b3bd67-1jammy [32.8 kB] 2024-03-23T14:53:25.934 INFO:teuthology.orchestra.run.smithi164.stdout:Get:38 https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240323.003144-squid/21b3bd672163092b4e542f772797fcba02d3d4de/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.0.0-1855-g21b3bd67-1jammy [180 kB] 2024-03-23T14:53:25.941 INFO:teuthology.orchestra.run.smithi164.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-03-23T14:53:25.953 INFO:teuthology.orchestra.run.smithi164.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-03-23T14:53:25.954 INFO:teuthology.orchestra.run.smithi164.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-03-23T14:53:25.954 INFO:teuthology.orchestra.run.smithi164.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2024-03-23T14:53:25.957 INFO:teuthology.orchestra.run.smithi164.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-03-23T14:53:25.958 INFO:teuthology.orchestra.run.smithi164.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-03-23T14:53:25.959 INFO:teuthology.orchestra.run.smithi173.stdout:Get:10 https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240323.003144-squid/21b3bd672163092b4e542f772797fcba02d3d4de/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.0.0-1855-g21b3bd67-1jammy [711 kB] 2024-03-23T14:53:25.959 INFO:teuthology.orchestra.run.smithi164.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-03-23T14:53:25.968 INFO:teuthology.orchestra.run.smithi164.stdout:Get:46 https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240323.003144-squid/21b3bd672163092b4e542f772797fcba02d3d4de/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.0.0-1855-g21b3bd67-1jammy [64.8 kB] 2024-03-23T14:53:25.970 INFO:teuthology.orchestra.run.smithi164.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-03-23T14:53:25.970 INFO:teuthology.orchestra.run.smithi164.stdout:Get:48 https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240323.003144-squid/21b3bd672163092b4e542f772797fcba02d3d4de/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.0.0-1855-g21b3bd67-1jammy [330 kB] 2024-03-23T14:53:25.970 INFO:teuthology.orchestra.run.smithi164.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-03-23T14:53:25.983 INFO:teuthology.orchestra.run.smithi164.stdout:Get:50 https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240323.003144-squid/21b3bd672163092b4e542f772797fcba02d3d4de/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.0.0-1855-g21b3bd67-1jammy [6,574 kB] 2024-03-23T14:53:25.985 INFO:teuthology.orchestra.run.smithi164.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-03-23T14:53:26.012 INFO:teuthology.orchestra.run.smithi164.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-03-23T14:53:26.014 INFO:teuthology.orchestra.run.smithi164.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-03-23T14:53:26.016 INFO:teuthology.orchestra.run.smithi173.stdout:Get:11 https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240323.003144-squid/21b3bd672163092b4e542f772797fcba02d3d4de/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.0.0-1855-g21b3bd67-1jammy [356 kB] 2024-03-23T14:53:26.018 INFO:teuthology.orchestra.run.smithi164.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2024-03-23T14:53:26.019 INFO:teuthology.orchestra.run.smithi164.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-03-23T14:53:26.022 INFO:teuthology.orchestra.run.smithi164.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-03-23T14:53:26.024 INFO:teuthology.orchestra.run.smithi164.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2024-03-23T14:53:26.028 INFO:teuthology.orchestra.run.smithi164.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-03-23T14:53:26.029 INFO:teuthology.orchestra.run.smithi164.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-03-23T14:53:26.031 INFO:teuthology.orchestra.run.smithi164.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-03-23T14:53:26.058 INFO:teuthology.orchestra.run.smithi173.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-03-23T14:53:26.065 INFO:teuthology.orchestra.run.smithi173.stdout:Get:13 https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240323.003144-squid/21b3bd672163092b4e542f772797fcba02d3d4de/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.0.0-1855-g21b3bd67-1jammy [32.8 kB] 2024-03-23T14:53:26.066 INFO:teuthology.orchestra.run.smithi173.stdout:Get:14 https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240323.003144-squid/21b3bd672163092b4e542f772797fcba02d3d4de/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.0.0-1855-g21b3bd67-1jammy [180 kB] 2024-03-23T14:53:26.067 INFO:teuthology.orchestra.run.smithi173.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-03-23T14:53:26.070 INFO:teuthology.orchestra.run.smithi164.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-03-23T14:53:26.072 INFO:teuthology.orchestra.run.smithi173.stdout:Get:16 https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240323.003144-squid/21b3bd672163092b4e542f772797fcba02d3d4de/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.0.0-1855-g21b3bd67-1jammy [64.8 kB] 2024-03-23T14:53:26.074 INFO:teuthology.orchestra.run.smithi173.stdout:Get:17 https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240323.003144-squid/21b3bd672163092b4e542f772797fcba02d3d4de/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.0.0-1855-g21b3bd67-1jammy [330 kB] 2024-03-23T14:53:26.087 INFO:teuthology.orchestra.run.smithi173.stdout:Get:18 https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240323.003144-squid/21b3bd672163092b4e542f772797fcba02d3d4de/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.0.0-1855-g21b3bd67-1jammy [6,574 kB] 2024-03-23T14:53:26.094 INFO:teuthology.orchestra.run.smithi173.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-03-23T14:53:26.102 INFO:teuthology.orchestra.run.smithi173.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-03-23T14:53:26.173 INFO:teuthology.orchestra.run.smithi164.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-03-23T14:53:26.175 INFO:teuthology.orchestra.run.smithi164.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.1 [108 kB] 2024-03-23T14:53:26.177 INFO:teuthology.orchestra.run.smithi164.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-03-23T14:53:26.178 INFO:teuthology.orchestra.run.smithi164.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-03-23T14:53:26.180 INFO:teuthology.orchestra.run.smithi164.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-03-23T14:53:26.267 INFO:teuthology.orchestra.run.smithi173.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-03-23T14:53:26.287 INFO:teuthology.orchestra.run.smithi164.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-03-23T14:53:26.292 INFO:teuthology.orchestra.run.smithi164.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-03-23T14:53:26.293 INFO:teuthology.orchestra.run.smithi164.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-03-23T14:53:26.321 INFO:teuthology.orchestra.run.smithi164.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-03-23T14:53:26.322 INFO:teuthology.orchestra.run.smithi164.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-03-23T14:53:26.322 INFO:teuthology.orchestra.run.smithi164.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-03-23T14:53:26.323 INFO:teuthology.orchestra.run.smithi164.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-03-23T14:53:26.324 INFO:teuthology.orchestra.run.smithi164.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-03-23T14:53:26.325 INFO:teuthology.orchestra.run.smithi164.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-03-23T14:53:26.337 INFO:teuthology.orchestra.run.smithi164.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-03-23T14:53:26.338 INFO:teuthology.orchestra.run.smithi164.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-03-23T14:53:26.342 INFO:teuthology.orchestra.run.smithi164.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-03-23T14:53:26.368 INFO:teuthology.orchestra.run.smithi164.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-03-23T14:53:26.369 INFO:teuthology.orchestra.run.smithi164.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-03-23T14:53:26.370 INFO:teuthology.orchestra.run.smithi164.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-03-23T14:53:26.375 INFO:teuthology.orchestra.run.smithi164.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-03-23T14:53:26.378 INFO:teuthology.orchestra.run.smithi164.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2024-03-23T14:53:26.379 INFO:teuthology.orchestra.run.smithi164.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-03-23T14:53:26.381 INFO:teuthology.orchestra.run.smithi164.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-03-23T14:53:26.402 INFO:teuthology.orchestra.run.smithi164.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2024-03-23T14:53:26.408 INFO:teuthology.orchestra.run.smithi164.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-03-23T14:53:26.408 INFO:teuthology.orchestra.run.smithi164.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-03-23T14:53:26.431 INFO:teuthology.orchestra.run.smithi164.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-03-23T14:53:26.431 INFO:teuthology.orchestra.run.smithi164.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-03-23T14:53:26.432 INFO:teuthology.orchestra.run.smithi164.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-03-23T14:53:26.432 INFO:teuthology.orchestra.run.smithi164.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-03-23T14:53:26.434 INFO:teuthology.orchestra.run.smithi164.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-03-23T14:53:26.435 INFO:teuthology.orchestra.run.smithi164.stdout:Get:94 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-03-23T14:53:26.447 INFO:teuthology.orchestra.run.smithi164.stdout:Get:95 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-03-23T14:53:26.469 INFO:teuthology.orchestra.run.smithi164.stdout:Get:96 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-03-23T14:53:26.469 INFO:teuthology.orchestra.run.smithi164.stdout:Get:97 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-03-23T14:53:26.472 INFO:teuthology.orchestra.run.smithi164.stdout:Get:98 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-03-23T14:53:26.502 INFO:teuthology.orchestra.run.smithi164.stdout:Get:99 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-03-23T14:53:26.503 INFO:teuthology.orchestra.run.smithi164.stdout:Get:100 https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240323.003144-squid/21b3bd672163092b4e542f772797fcba02d3d4de/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.0.0-1855-g21b3bd67-1jammy [112 kB] 2024-03-23T14:53:26.542 INFO:teuthology.orchestra.run.smithi164.stdout:Get:101 https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240323.003144-squid/21b3bd672163092b4e542f772797fcba02d3d4de/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.0.0-1855-g21b3bd67-1jammy [465 kB] 2024-03-23T14:53:26.554 INFO:teuthology.orchestra.run.smithi164.stdout:Get:102 https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240323.003144-squid/21b3bd672163092b4e542f772797fcba02d3d4de/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.0.0-1855-g21b3bd67-1jammy [25.1 MB] 2024-03-23T14:53:26.644 INFO:teuthology.orchestra.run.smithi173.stdout:Get:22 https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240323.003144-squid/21b3bd672163092b4e542f772797fcba02d3d4de/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.0.0-1855-g21b3bd67-1jammy [112 kB] 2024-03-23T14:53:26.647 INFO:teuthology.orchestra.run.smithi173.stdout:Get:23 https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240323.003144-squid/21b3bd672163092b4e542f772797fcba02d3d4de/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.0.0-1855-g21b3bd67-1jammy [465 kB] 2024-03-23T14:53:26.705 INFO:teuthology.orchestra.run.smithi173.stdout:Get:24 https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240323.003144-squid/21b3bd672163092b4e542f772797fcba02d3d4de/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.0.0-1855-g21b3bd67-1jammy [25.1 MB] 2024-03-23T14:53:26.832 INFO:teuthology.orchestra.run.smithi173.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-03-23T14:53:26.847 INFO:teuthology.orchestra.run.smithi173.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-03-23T14:53:26.863 INFO:teuthology.orchestra.run.smithi173.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-03-23T14:53:26.872 INFO:teuthology.orchestra.run.smithi173.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-03-23T14:53:26.872 INFO:teuthology.orchestra.run.smithi173.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-03-23T14:53:26.889 INFO:teuthology.orchestra.run.smithi173.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-03-23T14:53:26.927 INFO:teuthology.orchestra.run.smithi173.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-03-23T14:53:26.939 INFO:teuthology.orchestra.run.smithi173.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-03-23T14:53:26.944 INFO:teuthology.orchestra.run.smithi173.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-03-23T14:53:26.944 INFO:teuthology.orchestra.run.smithi173.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-03-23T14:53:27.115 INFO:teuthology.orchestra.run.smithi173.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-03-23T14:53:27.198 INFO:teuthology.orchestra.run.smithi173.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-03-23T14:53:27.221 INFO:teuthology.orchestra.run.smithi173.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-03-23T14:53:27.279 INFO:teuthology.orchestra.run.smithi173.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-03-23T14:53:27.322 INFO:teuthology.orchestra.run.smithi173.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-03-23T14:53:27.344 INFO:teuthology.orchestra.run.smithi173.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-03-23T14:53:27.372 INFO:teuthology.orchestra.run.smithi173.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-03-23T14:53:27.556 INFO:teuthology.orchestra.run.smithi173.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-03-23T14:53:27.572 INFO:teuthology.orchestra.run.smithi173.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-03-23T14:53:27.579 INFO:teuthology.orchestra.run.smithi173.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-03-23T14:53:27.584 INFO:teuthology.orchestra.run.smithi173.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-03-23T14:53:27.769 INFO:teuthology.orchestra.run.smithi173.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-03-23T14:53:27.852 INFO:teuthology.orchestra.run.smithi173.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-03-23T14:53:27.868 INFO:teuthology.orchestra.run.smithi173.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2024-03-23T14:53:28.053 INFO:teuthology.orchestra.run.smithi173.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-03-23T14:53:28.104 INFO:teuthology.orchestra.run.smithi173.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-03-23T14:53:28.118 INFO:teuthology.orchestra.run.smithi173.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-03-23T14:53:28.276 INFO:teuthology.orchestra.run.smithi173.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-03-23T14:53:28.277 INFO:teuthology.orchestra.run.smithi173.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-03-23T14:53:28.282 INFO:teuthology.orchestra.run.smithi173.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-03-23T14:53:28.287 INFO:teuthology.orchestra.run.smithi173.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-03-23T14:53:28.302 INFO:teuthology.orchestra.run.smithi173.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-03-23T14:53:28.486 INFO:teuthology.orchestra.run.smithi173.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2024-03-23T14:53:28.533 INFO:teuthology.orchestra.run.smithi164.stdout:Get:103 https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240323.003144-squid/21b3bd672163092b4e542f772797fcba02d3d4de/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.0.0-1855-g21b3bd67-1jammy [5,076 kB] 2024-03-23T14:53:28.749 INFO:teuthology.orchestra.run.smithi173.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-03-23T14:53:28.806 INFO:teuthology.orchestra.run.smithi164.stdout:Get:104 https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240323.003144-squid/21b3bd672163092b4e542f772797fcba02d3d4de/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.0.0-1855-g21b3bd67-1jammy [244 kB] 2024-03-23T14:53:28.813 INFO:teuthology.orchestra.run.smithi164.stdout:Get:105 https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240323.003144-squid/21b3bd672163092b4e542f772797fcba02d3d4de/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.0.0-1855-g21b3bd67-1jammy [124 kB] 2024-03-23T14:53:28.817 INFO:teuthology.orchestra.run.smithi164.stdout:Get:106 https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240323.003144-squid/21b3bd672163092b4e542f772797fcba02d3d4de/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.0.0-1855-g21b3bd67-1jammy [1,524 kB] 2024-03-23T14:53:28.876 INFO:teuthology.orchestra.run.smithi173.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-03-23T14:53:28.877 INFO:teuthology.orchestra.run.smithi173.stdout:Get:60 https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240323.003144-squid/21b3bd672163092b4e542f772797fcba02d3d4de/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.0.0-1855-g21b3bd67-1jammy [5,076 kB] 2024-03-23T14:53:28.910 INFO:teuthology.orchestra.run.smithi173.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2024-03-23T14:53:28.912 INFO:teuthology.orchestra.run.smithi164.stdout:Get:107 https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240323.003144-squid/21b3bd672163092b4e542f772797fcba02d3d4de/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.0.0-1855-g21b3bd67-1jammy [6,246 kB] 2024-03-23T14:53:28.992 INFO:teuthology.orchestra.run.smithi173.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-03-23T14:53:29.004 INFO:teuthology.orchestra.run.smithi173.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-03-23T14:53:29.017 INFO:teuthology.orchestra.run.smithi173.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-03-23T14:53:29.058 INFO:teuthology.orchestra.run.smithi173.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-03-23T14:53:29.236 INFO:teuthology.orchestra.run.smithi173.stdout:Get:66 https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240323.003144-squid/21b3bd672163092b4e542f772797fcba02d3d4de/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.0.0-1855-g21b3bd67-1jammy [244 kB] 2024-03-23T14:53:29.243 INFO:teuthology.orchestra.run.smithi173.stdout:Get:67 https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240323.003144-squid/21b3bd672163092b4e542f772797fcba02d3d4de/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.0.0-1855-g21b3bd67-1jammy [124 kB] 2024-03-23T14:53:29.321 INFO:teuthology.orchestra.run.smithi173.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-03-23T14:53:29.323 INFO:teuthology.orchestra.run.smithi173.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.1 [108 kB] 2024-03-23T14:53:29.325 INFO:teuthology.orchestra.run.smithi173.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-03-23T14:53:29.327 INFO:teuthology.orchestra.run.smithi173.stdout:Get:71 https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240323.003144-squid/21b3bd672163092b4e542f772797fcba02d3d4de/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.0.0-1855-g21b3bd67-1jammy [1,524 kB] 2024-03-23T14:53:29.358 INFO:teuthology.orchestra.run.smithi164.stdout:Get:108 https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240323.003144-squid/21b3bd672163092b4e542f772797fcba02d3d4de/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.0.0-1855-g21b3bd67-1jammy [22.7 MB] 2024-03-23T14:53:29.391 INFO:teuthology.orchestra.run.smithi173.stdout:Get:72 https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240323.003144-squid/21b3bd672163092b4e542f772797fcba02d3d4de/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.0.0-1855-g21b3bd67-1jammy [6,246 kB] 2024-03-23T14:53:29.490 INFO:teuthology.orchestra.run.smithi173.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-03-23T14:53:29.816 INFO:teuthology.orchestra.run.smithi173.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-03-23T14:53:30.013 INFO:teuthology.orchestra.run.smithi173.stdout:Get:75 https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240323.003144-squid/21b3bd672163092b4e542f772797fcba02d3d4de/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.0.0-1855-g21b3bd67-1jammy [22.7 MB] 2024-03-23T14:53:30.226 INFO:teuthology.orchestra.run.smithi173.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-03-23T14:53:30.236 INFO:teuthology.orchestra.run.smithi173.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-03-23T14:53:30.237 INFO:teuthology.orchestra.run.smithi173.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-03-23T14:53:30.308 INFO:teuthology.orchestra.run.smithi173.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-03-23T14:53:30.309 INFO:teuthology.orchestra.run.smithi173.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-03-23T14:53:30.310 INFO:teuthology.orchestra.run.smithi173.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-03-23T14:53:30.311 INFO:teuthology.orchestra.run.smithi173.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-03-23T14:53:30.312 INFO:teuthology.orchestra.run.smithi173.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-03-23T14:53:30.479 INFO:teuthology.orchestra.run.smithi173.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-03-23T14:53:30.576 INFO:teuthology.orchestra.run.smithi173.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-03-23T14:53:30.670 INFO:teuthology.orchestra.run.smithi173.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-03-23T14:53:30.909 INFO:teuthology.orchestra.run.smithi164.stdout:Get:109 https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240323.003144-squid/21b3bd672163092b4e542f772797fcba02d3d4de/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.0.0-1855-g21b3bd67-1jammy [14.0 kB] 2024-03-23T14:53:30.910 INFO:teuthology.orchestra.run.smithi164.stdout:Get:110 https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240323.003144-squid/21b3bd672163092b4e542f772797fcba02d3d4de/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.0.0-1855-g21b3bd67-1jammy [868 kB] 2024-03-23T14:53:30.916 INFO:teuthology.orchestra.run.smithi173.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-03-23T14:53:30.926 INFO:teuthology.orchestra.run.smithi164.stdout:Get:111 https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240323.003144-squid/21b3bd672163092b4e542f772797fcba02d3d4de/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.0.0-1855-g21b3bd67-1jammy [2,427 kB] 2024-03-23T14:53:30.959 INFO:teuthology.orchestra.run.smithi173.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-03-23T14:53:30.984 INFO:teuthology.orchestra.run.smithi173.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-03-23T14:53:30.992 INFO:teuthology.orchestra.run.smithi173.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-03-23T14:53:31.019 INFO:teuthology.orchestra.run.smithi164.stdout:Get:112 https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240323.003144-squid/21b3bd672163092b4e542f772797fcba02d3d4de/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.0.0-1855-g21b3bd67-1jammy [776 kB] 2024-03-23T14:53:31.036 INFO:teuthology.orchestra.run.smithi173.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-03-23T14:53:31.061 INFO:teuthology.orchestra.run.smithi173.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2024-03-23T14:53:31.064 INFO:teuthology.orchestra.run.smithi173.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-03-23T14:53:31.070 INFO:teuthology.orchestra.run.smithi173.stdout:Get:94 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-03-23T14:53:31.075 INFO:teuthology.orchestra.run.smithi164.stdout:Get:113 https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240323.003144-squid/21b3bd672163092b4e542f772797fcba02d3d4de/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.0.0-1855-g21b3bd67-1jammy [144 kB] 2024-03-23T14:53:31.082 INFO:teuthology.orchestra.run.smithi164.stdout:Get:114 https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240323.003144-squid/21b3bd672163092b4e542f772797fcba02d3d4de/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.0.0-1855-g21b3bd67-1jammy [1,923 kB] 2024-03-23T14:53:31.194 INFO:teuthology.orchestra.run.smithi164.stdout:Get:115 https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240323.003144-squid/21b3bd672163092b4e542f772797fcba02d3d4de/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.0.0-1855-g21b3bd67-1jammy [8,625 kB] 2024-03-23T14:53:31.237 INFO:teuthology.orchestra.run.smithi173.stdout:Get:95 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2024-03-23T14:53:31.726 INFO:teuthology.orchestra.run.smithi164.stdout:Get:116 https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240323.003144-squid/21b3bd672163092b4e542f772797fcba02d3d4de/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.0.0-1855-g21b3bd67-1jammy [14.2 kB] 2024-03-23T14:53:31.726 INFO:teuthology.orchestra.run.smithi164.stdout:Get:117 https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240323.003144-squid/21b3bd672163092b4e542f772797fcba02d3d4de/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.0.0-1855-g21b3bd67-1jammy [49.9 MB] 2024-03-23T14:53:31.777 INFO:teuthology.orchestra.run.smithi173.stdout:Get:96 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-03-23T14:53:31.782 INFO:teuthology.orchestra.run.smithi173.stdout:Get:97 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-03-23T14:53:31.821 INFO:teuthology.orchestra.run.smithi173.stdout:Get:98 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-03-23T14:53:31.822 INFO:teuthology.orchestra.run.smithi173.stdout:Get:99 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-03-23T14:53:31.828 INFO:teuthology.orchestra.run.smithi173.stdout:Get:100 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-03-23T14:53:31.830 INFO:teuthology.orchestra.run.smithi173.stdout:Get:101 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-03-23T14:53:31.871 INFO:teuthology.orchestra.run.smithi173.stdout:Get:102 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-03-23T14:53:31.880 INFO:teuthology.orchestra.run.smithi173.stdout:Get:103 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-03-23T14:53:31.953 INFO:teuthology.orchestra.run.smithi173.stdout:Get:104 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-03-23T14:53:31.954 INFO:teuthology.orchestra.run.smithi173.stdout:Get:105 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-03-23T14:53:32.119 INFO:teuthology.orchestra.run.smithi173.stdout:Get:106 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-03-23T14:53:32.206 INFO:teuthology.orchestra.run.smithi173.stdout:Get:107 https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240323.003144-squid/21b3bd672163092b4e542f772797fcba02d3d4de/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.0.0-1855-g21b3bd67-1jammy [14.0 kB] 2024-03-23T14:53:32.207 INFO:teuthology.orchestra.run.smithi173.stdout:Get:108 https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240323.003144-squid/21b3bd672163092b4e542f772797fcba02d3d4de/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.0.0-1855-g21b3bd67-1jammy [868 kB] 2024-03-23T14:53:32.224 INFO:teuthology.orchestra.run.smithi173.stdout:Get:109 https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240323.003144-squid/21b3bd672163092b4e542f772797fcba02d3d4de/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.0.0-1855-g21b3bd67-1jammy [2,427 kB] 2024-03-23T14:53:32.396 INFO:teuthology.orchestra.run.smithi173.stdout:Get:110 https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240323.003144-squid/21b3bd672163092b4e542f772797fcba02d3d4de/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.0.0-1855-g21b3bd67-1jammy [776 kB] 2024-03-23T14:53:32.459 INFO:teuthology.orchestra.run.smithi173.stdout:Get:111 https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240323.003144-squid/21b3bd672163092b4e542f772797fcba02d3d4de/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.0.0-1855-g21b3bd67-1jammy [144 kB] 2024-03-23T14:53:32.463 INFO:teuthology.orchestra.run.smithi173.stdout:Get:112 https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240323.003144-squid/21b3bd672163092b4e542f772797fcba02d3d4de/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.0.0-1855-g21b3bd67-1jammy [1,923 kB] 2024-03-23T14:53:32.563 INFO:teuthology.orchestra.run.smithi173.stdout:Get:113 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-03-23T14:53:32.579 INFO:teuthology.orchestra.run.smithi173.stdout:Get:114 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-03-23T14:53:32.621 INFO:teuthology.orchestra.run.smithi173.stdout:Get:115 https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240323.003144-squid/21b3bd672163092b4e542f772797fcba02d3d4de/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.0.0-1855-g21b3bd67-1jammy [8,625 kB] 2024-03-23T14:53:33.327 INFO:teuthology.orchestra.run.smithi173.stdout:Get:116 https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240323.003144-squid/21b3bd672163092b4e542f772797fcba02d3d4de/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.0.0-1855-g21b3bd67-1jammy [14.2 kB] 2024-03-23T14:53:33.328 INFO:teuthology.orchestra.run.smithi173.stdout:Get:117 https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240323.003144-squid/21b3bd672163092b4e542f772797fcba02d3d4de/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.0.0-1855-g21b3bd67-1jammy [49.9 MB] 2024-03-23T14:53:34.717 INFO:teuthology.orchestra.run.smithi164.stdout:Get:118 https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240323.003144-squid/21b3bd672163092b4e542f772797fcba02d3d4de/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.0.0-1855-g21b3bd67-1jammy [121 kB] 2024-03-23T14:53:34.719 INFO:teuthology.orchestra.run.smithi164.stdout:Get:119 https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240323.003144-squid/21b3bd672163092b4e542f772797fcba02d3d4de/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 19.0.0-1855-g21b3bd67-1jammy [227 kB] 2024-03-23T14:53:34.724 INFO:teuthology.orchestra.run.smithi164.stdout:Get:120 https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240323.003144-squid/21b3bd672163092b4e542f772797fcba02d3d4de/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 19.0.0-1855-g21b3bd67-1jammy [34.0 kB] 2024-03-23T14:53:34.725 INFO:teuthology.orchestra.run.smithi164.stdout:Get:121 https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240323.003144-squid/21b3bd672163092b4e542f772797fcba02d3d4de/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 19.0.0-1855-g21b3bd67-1jammy [23.4 kB] 2024-03-23T14:53:34.725 INFO:teuthology.orchestra.run.smithi164.stdout:Get:122 https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240323.003144-squid/21b3bd672163092b4e542f772797fcba02d3d4de/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.0.0-1855-g21b3bd67-1jammy [38.9 kB] 2024-03-23T14:53:34.727 INFO:teuthology.orchestra.run.smithi164.stdout:Get:123 https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240323.003144-squid/21b3bd672163092b4e542f772797fcba02d3d4de/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.0.0-1855-g21b3bd67-1jammy [13.0 MB] 2024-03-23T14:53:35.267 INFO:teuthology.orchestra.run.smithi164.stdout:Get:124 https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240323.003144-squid/21b3bd672163092b4e542f772797fcba02d3d4de/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.0.0-1855-g21b3bd67-1jammy [91.4 kB] 2024-03-23T14:53:35.882 INFO:teuthology.orchestra.run.smithi164.stdout:Fetched 178 MB in 10s (17.6 MB/s) 2024-03-23T14:53:36.021 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-pyperclip. 2024-03-23T14:53:36.120 INFO:teuthology.orchestra.run.smithi164.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165997 files and directories currently installed.) 2024-03-23T14:53:36.126 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2024-03-23T14:53:36.169 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2024-03-23T14:53:36.353 INFO:teuthology.orchestra.run.smithi173.stdout:Get:118 https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240323.003144-squid/21b3bd672163092b4e542f772797fcba02d3d4de/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.0.0-1855-g21b3bd67-1jammy [121 kB] 2024-03-23T14:53:36.356 INFO:teuthology.orchestra.run.smithi173.stdout:Get:119 https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240323.003144-squid/21b3bd672163092b4e542f772797fcba02d3d4de/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 19.0.0-1855-g21b3bd67-1jammy [227 kB] 2024-03-23T14:53:36.360 INFO:teuthology.orchestra.run.smithi173.stdout:Get:120 https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240323.003144-squid/21b3bd672163092b4e542f772797fcba02d3d4de/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 19.0.0-1855-g21b3bd67-1jammy [34.0 kB] 2024-03-23T14:53:36.361 INFO:teuthology.orchestra.run.smithi173.stdout:Get:121 https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240323.003144-squid/21b3bd672163092b4e542f772797fcba02d3d4de/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 19.0.0-1855-g21b3bd67-1jammy [23.4 kB] 2024-03-23T14:53:36.362 INFO:teuthology.orchestra.run.smithi173.stdout:Get:122 https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240323.003144-squid/21b3bd672163092b4e542f772797fcba02d3d4de/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.0.0-1855-g21b3bd67-1jammy [38.9 kB] 2024-03-23T14:53:36.363 INFO:teuthology.orchestra.run.smithi173.stdout:Get:123 https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240323.003144-squid/21b3bd672163092b4e542f772797fcba02d3d4de/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.0.0-1855-g21b3bd67-1jammy [13.0 MB] 2024-03-23T14:53:36.447 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-wcwidth. 2024-03-23T14:53:36.474 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-03-23T14:53:36.514 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-03-23T14:53:36.808 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-cmd2. 2024-03-23T14:53:36.837 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2024-03-23T14:53:36.867 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2024-03-23T14:53:37.006 INFO:teuthology.orchestra.run.smithi173.stdout:Get:124 https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240323.003144-squid/21b3bd672163092b4e542f772797fcba02d3d4de/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.0.0-1855-g21b3bd67-1jammy [91.4 kB] 2024-03-23T14:53:37.170 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-03-23T14:53:37.196 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../003-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-03-23T14:53:37.262 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-03-23T14:53:37.590 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-03-23T14:53:37.616 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-03-23T14:53:37.622 INFO:teuthology.orchestra.run.smithi173.stdout:Fetched 178 MB in 12s (15.1 MB/s) 2024-03-23T14:53:37.656 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-03-23T14:53:37.756 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-pyperclip. 2024-03-23T14:53:37.843 INFO:teuthology.orchestra.run.smithi173.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165997 files and directories currently installed.) 2024-03-23T14:53:37.848 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2024-03-23T14:53:37.888 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2024-03-23T14:53:37.976 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-03-23T14:53:38.002 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../005-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-03-23T14:53:38.043 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-03-23T14:53:38.183 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-wcwidth. 2024-03-23T14:53:38.210 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-03-23T14:53:38.250 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-03-23T14:53:38.446 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-03-23T14:53:38.473 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../006-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-03-23T14:53:38.538 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-23T14:53:38.569 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-cmd2. 2024-03-23T14:53:38.596 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2024-03-23T14:53:38.635 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2024-03-23T14:53:38.999 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-03-23T14:53:39.025 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../007-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-03-23T14:53:39.074 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-23T14:53:39.172 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-03-23T14:53:39.199 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../003-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-03-23T14:53:39.280 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-03-23T14:53:39.411 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-03-23T14:53:39.437 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../008-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-03-23T14:53:39.477 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-23T14:53:39.642 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-03-23T14:53:39.669 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-03-23T14:53:39.709 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-03-23T14:53:39.805 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-03-23T14:53:39.832 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../009-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-03-23T14:53:39.872 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-03-23T14:53:40.053 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-03-23T14:53:40.080 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../005-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-03-23T14:53:40.120 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-03-23T14:53:40.175 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package librados2. 2024-03-23T14:53:40.202 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../010-librados2_19.0.0-1855-g21b3bd67-1jammy_amd64.deb ... 2024-03-23T14:53:40.258 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking librados2 (19.0.0-1855-g21b3bd67-1jammy) ... 2024-03-23T14:53:40.678 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package librbd1. 2024-03-23T14:53:40.682 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-03-23T14:53:40.704 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../011-librbd1_19.0.0-1855-g21b3bd67-1jammy_amd64.deb ... 2024-03-23T14:53:40.709 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../006-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-03-23T14:53:40.736 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking librbd1 (19.0.0-1855-g21b3bd67-1jammy) ... 2024-03-23T14:53:40.764 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-23T14:53:41.138 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package libcephfs2. 2024-03-23T14:53:41.165 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../012-libcephfs2_19.0.0-1855-g21b3bd67-1jammy_amd64.deb ... 2024-03-23T14:53:41.201 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-03-23T14:53:41.205 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking libcephfs2 (19.0.0-1855-g21b3bd67-1jammy) ... 2024-03-23T14:53:41.228 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../007-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-03-23T14:53:41.268 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-23T14:53:41.483 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-rados. 2024-03-23T14:53:41.509 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../013-python3-rados_19.0.0-1855-g21b3bd67-1jammy_amd64.deb ... 2024-03-23T14:53:41.549 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-rados (19.0.0-1855-g21b3bd67-1jammy) ... 2024-03-23T14:53:41.621 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-03-23T14:53:41.648 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../008-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-03-23T14:53:41.696 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-23T14:53:41.852 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-03-23T14:53:41.878 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../014-python3-ceph-argparse_19.0.0-1855-g21b3bd67-1jammy_all.deb ... 2024-03-23T14:53:41.919 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-ceph-argparse (19.0.0-1855-g21b3bd67-1jammy) ... 2024-03-23T14:53:42.124 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-03-23T14:53:42.146 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-cephfs. 2024-03-23T14:53:42.150 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../009-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-03-23T14:53:42.162 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../015-python3-cephfs_19.0.0-1855-g21b3bd67-1jammy_amd64.deb ... 2024-03-23T14:53:42.196 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-cephfs (19.0.0-1855-g21b3bd67-1jammy) ... 2024-03-23T14:53:42.199 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-03-23T14:53:42.482 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-ceph-common. 2024-03-23T14:53:42.508 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../016-python3-ceph-common_19.0.0-1855-g21b3bd67-1jammy_all.deb ... 2024-03-23T14:53:42.535 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package librados2. 2024-03-23T14:53:42.549 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-ceph-common (19.0.0-1855-g21b3bd67-1jammy) ... 2024-03-23T14:53:42.561 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../010-librados2_19.0.0-1855-g21b3bd67-1jammy_amd64.deb ... 2024-03-23T14:53:42.601 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking librados2 (19.0.0-1855-g21b3bd67-1jammy) ... 2024-03-23T14:53:42.860 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-prettytable. 2024-03-23T14:53:42.887 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../017-python3-prettytable_2.5.0-2_all.deb ... 2024-03-23T14:53:42.927 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-03-23T14:53:43.046 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package librbd1. 2024-03-23T14:53:43.072 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../011-librbd1_19.0.0-1855-g21b3bd67-1jammy_amd64.deb ... 2024-03-23T14:53:43.104 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking librbd1 (19.0.0-1855-g21b3bd67-1jammy) ... 2024-03-23T14:53:43.180 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-rbd. 2024-03-23T14:53:43.195 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../018-python3-rbd_19.0.0-1855-g21b3bd67-1jammy_amd64.deb ... 2024-03-23T14:53:43.238 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-rbd (19.0.0-1855-g21b3bd67-1jammy) ... 2024-03-23T14:53:43.532 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-03-23T14:53:43.559 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../019-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-03-23T14:53:43.574 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package libcephfs2. 2024-03-23T14:53:43.591 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-03-23T14:53:43.600 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../012-libcephfs2_19.0.0-1855-g21b3bd67-1jammy_amd64.deb ... 2024-03-23T14:53:43.640 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking libcephfs2 (19.0.0-1855-g21b3bd67-1jammy) ... 2024-03-23T14:53:43.926 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-rados. 2024-03-23T14:53:43.944 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../013-python3-rados_19.0.0-1855-g21b3bd67-1jammy_amd64.deb ... 2024-03-23T14:53:43.984 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-rados (19.0.0-1855-g21b3bd67-1jammy) ... 2024-03-23T14:53:44.062 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-03-23T14:53:44.075 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../020-libreadline-dev_8.1.2-1_amd64.deb ... 2024-03-23T14:53:44.118 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-03-23T14:53:44.396 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-03-23T14:53:44.423 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../021-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-03-23T14:53:44.429 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-03-23T14:53:44.455 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../014-python3-ceph-argparse_19.0.0-1855-g21b3bd67-1jammy_all.deb ... 2024-03-23T14:53:44.463 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-03-23T14:53:44.495 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-ceph-argparse (19.0.0-1855-g21b3bd67-1jammy) ... 2024-03-23T14:53:44.722 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-cephfs. 2024-03-23T14:53:44.738 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../015-python3-cephfs_19.0.0-1855-g21b3bd67-1jammy_amd64.deb ... 2024-03-23T14:53:44.772 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-cephfs (19.0.0-1855-g21b3bd67-1jammy) ... 2024-03-23T14:53:44.774 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package lua5.1. 2024-03-23T14:53:44.790 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../022-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-03-23T14:53:44.824 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-03-23T14:53:45.100 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-ceph-common. 2024-03-23T14:53:45.127 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../016-python3-ceph-common_19.0.0-1855-g21b3bd67-1jammy_all.deb ... 2024-03-23T14:53:45.128 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package lua-any. 2024-03-23T14:53:45.143 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../023-lua-any_27ubuntu1_all.deb ... 2024-03-23T14:53:45.167 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-ceph-common (19.0.0-1855-g21b3bd67-1jammy) ... 2024-03-23T14:53:45.186 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking lua-any (27ubuntu1) ... 2024-03-23T14:53:45.430 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package zip. 2024-03-23T14:53:45.457 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../024-zip_3.0-12build2_amd64.deb ... 2024-03-23T14:53:45.497 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking zip (3.0-12build2) ... 2024-03-23T14:53:45.503 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-prettytable. 2024-03-23T14:53:45.529 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../017-python3-prettytable_2.5.0-2_all.deb ... 2024-03-23T14:53:45.570 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-03-23T14:53:45.833 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package unzip. 2024-03-23T14:53:45.839 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-rbd. 2024-03-23T14:53:45.854 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../018-python3-rbd_19.0.0-1855-g21b3bd67-1jammy_amd64.deb ... 2024-03-23T14:53:45.860 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../025-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-03-23T14:53:45.897 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-rbd (19.0.0-1855-g21b3bd67-1jammy) ... 2024-03-23T14:53:45.908 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-03-23T14:53:46.217 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-03-23T14:53:46.243 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../019-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-03-23T14:53:46.245 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package luarocks. 2024-03-23T14:53:46.269 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../026-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-03-23T14:53:46.283 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-03-23T14:53:46.304 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-03-23T14:53:46.536 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-03-23T14:53:46.551 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../020-libreadline-dev_8.1.2-1_amd64.deb ... 2024-03-23T14:53:46.594 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-03-23T14:53:46.664 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package librgw2. 2024-03-23T14:53:46.690 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../027-librgw2_19.0.0-1855-g21b3bd67-1jammy_amd64.deb ... 2024-03-23T14:53:46.730 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking librgw2 (19.0.0-1855-g21b3bd67-1jammy) ... 2024-03-23T14:53:46.914 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-03-23T14:53:46.940 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../021-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-03-23T14:53:46.980 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-03-23T14:53:47.233 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-rgw. 2024-03-23T14:53:47.259 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../028-python3-rgw_19.0.0-1855-g21b3bd67-1jammy_amd64.deb ... 2024-03-23T14:53:47.299 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-rgw (19.0.0-1855-g21b3bd67-1jammy) ... 2024-03-23T14:53:47.317 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package lua5.1. 2024-03-23T14:53:47.343 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../022-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-03-23T14:53:47.383 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-03-23T14:53:47.635 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package liboath0:amd64. 2024-03-23T14:53:47.662 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../029-liboath0_2.6.7-3build1_amd64.deb ... 2024-03-23T14:53:47.686 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package lua-any. 2024-03-23T14:53:47.701 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-03-23T14:53:47.713 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../023-lua-any_27ubuntu1_all.deb ... 2024-03-23T14:53:47.753 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking lua-any (27ubuntu1) ... 2024-03-23T14:53:47.955 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package zip. 2024-03-23T14:53:47.970 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../024-zip_3.0-12build2_amd64.deb ... 2024-03-23T14:53:48.013 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking zip (3.0-12build2) ... 2024-03-23T14:53:48.030 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package libradosstriper1. 2024-03-23T14:53:48.056 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../030-libradosstriper1_19.0.0-1855-g21b3bd67-1jammy_amd64.deb ... 2024-03-23T14:53:48.096 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking libradosstriper1 (19.0.0-1855-g21b3bd67-1jammy) ... 2024-03-23T14:53:48.317 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package unzip. 2024-03-23T14:53:48.343 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../025-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-03-23T14:53:48.383 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-03-23T14:53:48.424 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package ceph-common. 2024-03-23T14:53:48.451 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../031-ceph-common_19.0.0-1855-g21b3bd67-1jammy_amd64.deb ... 2024-03-23T14:53:48.482 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking ceph-common (19.0.0-1855-g21b3bd67-1jammy) ... 2024-03-23T14:53:48.804 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package luarocks. 2024-03-23T14:53:48.831 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../026-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-03-23T14:53:48.870 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-03-23T14:53:49.228 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package librgw2. 2024-03-23T14:53:49.254 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../027-librgw2_19.0.0-1855-g21b3bd67-1jammy_amd64.deb ... 2024-03-23T14:53:49.390 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking librgw2 (19.0.0-1855-g21b3bd67-1jammy) ... 2024-03-23T14:53:49.829 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package ceph-base. 2024-03-23T14:53:49.856 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../032-ceph-base_19.0.0-1855-g21b3bd67-1jammy_amd64.deb ... 2024-03-23T14:53:49.905 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-rgw. 2024-03-23T14:53:49.906 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking ceph-base (19.0.0-1855-g21b3bd67-1jammy) ... 2024-03-23T14:53:49.932 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../028-python3-rgw_19.0.0-1855-g21b3bd67-1jammy_amd64.deb ... 2024-03-23T14:53:49.972 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-rgw (19.0.0-1855-g21b3bd67-1jammy) ... 2024-03-23T14:53:50.300 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package liboath0:amd64. 2024-03-23T14:53:50.326 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../029-liboath0_2.6.7-3build1_amd64.deb ... 2024-03-23T14:53:50.366 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-03-23T14:53:50.450 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-03-23T14:53:50.476 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../033-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-03-23T14:53:50.516 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-03-23T14:53:50.677 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package libradosstriper1. 2024-03-23T14:53:50.703 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../030-libradosstriper1_19.0.0-1855-g21b3bd67-1jammy_amd64.deb ... 2024-03-23T14:53:50.744 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking libradosstriper1 (19.0.0-1855-g21b3bd67-1jammy) ... 2024-03-23T14:53:50.828 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-cheroot. 2024-03-23T14:53:50.854 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../034-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-03-23T14:53:50.894 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-03-23T14:53:51.122 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package ceph-common. 2024-03-23T14:53:51.149 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../031-ceph-common_19.0.0-1855-g21b3bd67-1jammy_amd64.deb ... 2024-03-23T14:53:51.180 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking ceph-common (19.0.0-1855-g21b3bd67-1jammy) ... 2024-03-23T14:53:51.214 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-03-23T14:53:51.240 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../035-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-03-23T14:53:51.280 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-03-23T14:53:51.550 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-jaraco.text. 2024-03-23T14:53:51.565 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../036-python3-jaraco.text_3.6.0-2_all.deb ... 2024-03-23T14:53:51.600 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-03-23T14:53:51.886 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-03-23T14:53:51.912 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../037-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-03-23T14:53:51.953 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-03-23T14:53:52.239 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-tempora. 2024-03-23T14:53:52.266 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../038-python3-tempora_4.1.2-1_all.deb ... 2024-03-23T14:53:52.305 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-03-23T14:53:52.486 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package ceph-base. 2024-03-23T14:53:52.512 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../032-ceph-base_19.0.0-1855-g21b3bd67-1jammy_amd64.deb ... 2024-03-23T14:53:52.561 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking ceph-base (19.0.0-1855-g21b3bd67-1jammy) ... 2024-03-23T14:53:52.608 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-portend. 2024-03-23T14:53:52.635 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../039-python3-portend_3.0.0-1_all.deb ... 2024-03-23T14:53:52.674 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-03-23T14:53:52.962 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-03-23T14:53:52.988 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../040-python3-zc.lockfile_2.0-1_all.deb ... 2024-03-23T14:53:53.027 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-03-23T14:53:53.189 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-03-23T14:53:53.205 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../033-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-03-23T14:53:53.239 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-03-23T14:53:53.322 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-cherrypy3. 2024-03-23T14:53:53.349 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../041-python3-cherrypy3_18.6.1-4_all.deb ... 2024-03-23T14:53:53.388 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-03-23T14:53:53.542 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-cheroot. 2024-03-23T14:53:53.569 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../034-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-03-23T14:53:53.609 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-03-23T14:53:53.708 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-natsort. 2024-03-23T14:53:53.735 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../042-python3-natsort_8.0.2-1_all.deb ... 2024-03-23T14:53:53.766 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-03-23T14:53:53.937 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-03-23T14:53:53.958 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../035-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-03-23T14:53:54.003 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-03-23T14:53:54.036 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-logutils. 2024-03-23T14:53:54.063 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../043-python3-logutils_0.3.3-8_all.deb ... 2024-03-23T14:53:54.108 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-03-23T14:53:54.306 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-jaraco.text. 2024-03-23T14:53:54.333 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../036-python3-jaraco.text_3.6.0-2_all.deb ... 2024-03-23T14:53:54.372 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-03-23T14:53:54.573 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-markupsafe. 2024-03-23T14:53:54.599 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../044-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-03-23T14:53:54.639 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-03-23T14:53:54.651 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-03-23T14:53:54.677 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../037-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-03-23T14:53:54.709 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-03-23T14:53:54.942 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-mako. 2024-03-23T14:53:54.969 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../045-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-03-23T14:53:54.995 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-tempora. 2024-03-23T14:53:55.008 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-03-23T14:53:55.022 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../038-python3-tempora_4.1.2-1_all.deb ... 2024-03-23T14:53:55.062 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-03-23T14:53:55.345 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-simplegeneric. 2024-03-23T14:53:55.372 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../046-python3-simplegeneric_0.8.1-3_all.deb ... 2024-03-23T14:53:55.381 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-portend. 2024-03-23T14:53:55.408 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../039-python3-portend_3.0.0-1_all.deb ... 2024-03-23T14:53:55.412 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-03-23T14:53:55.447 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-03-23T14:53:55.714 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-singledispatch. 2024-03-23T14:53:55.741 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../047-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-03-23T14:53:55.750 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-03-23T14:53:55.777 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../040-python3-zc.lockfile_2.0-1_all.deb ... 2024-03-23T14:53:55.781 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-03-23T14:53:55.817 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-03-23T14:53:56.042 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-webob. 2024-03-23T14:53:56.057 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../048-python3-webob_1%3a1.8.6-1.1_all.deb ... 2024-03-23T14:53:56.078 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-cherrypy3. 2024-03-23T14:53:56.092 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2024-03-23T14:53:56.105 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../041-python3-cherrypy3_18.6.1-4_all.deb ... 2024-03-23T14:53:56.136 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-03-23T14:53:56.438 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-waitress. 2024-03-23T14:53:56.465 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../049-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-03-23T14:53:56.481 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-natsort. 2024-03-23T14:53:56.498 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../042-python3-natsort_8.0.2-1_all.deb ... 2024-03-23T14:53:56.506 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-03-23T14:53:56.539 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-03-23T14:53:56.814 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-tempita. 2024-03-23T14:53:56.841 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../050-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-03-23T14:53:56.851 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-logutils. 2024-03-23T14:53:56.878 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../043-python3-logutils_0.3.3-8_all.deb ... 2024-03-23T14:53:56.881 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-03-23T14:53:56.918 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-03-23T14:53:57.184 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-paste. 2024-03-23T14:53:57.195 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-markupsafe. 2024-03-23T14:53:57.210 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../051-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-03-23T14:53:57.222 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../044-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-03-23T14:53:57.250 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-03-23T14:53:57.254 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-03-23T14:53:57.553 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-03-23T14:53:57.556 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-mako. 2024-03-23T14:53:57.576 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../052-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-03-23T14:53:57.583 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../045-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-03-23T14:53:57.620 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-03-23T14:53:57.623 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-03-23T14:53:57.923 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-pastedeploy. 2024-03-23T14:53:57.949 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../053-python3-pastedeploy_2.1.1-1_all.deb ... 2024-03-23T14:53:57.962 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-simplegeneric. 2024-03-23T14:53:57.987 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../046-python3-simplegeneric_0.8.1-3_all.deb ... 2024-03-23T14:53:57.989 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-03-23T14:53:58.026 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-03-23T14:53:58.292 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-webtest. 2024-03-23T14:53:58.319 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../054-python3-webtest_2.0.35-1_all.deb ... 2024-03-23T14:53:58.329 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-singledispatch. 2024-03-23T14:53:58.356 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../047-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-03-23T14:53:58.358 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-03-23T14:53:58.395 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-03-23T14:53:59.363 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-pecan. 2024-03-23T14:53:59.374 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-webob. 2024-03-23T14:53:59.389 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../055-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-03-23T14:53:59.390 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../048-python3-webob_1%3a1.8.6-1.1_all.deb ... 2024-03-23T14:53:59.420 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-03-23T14:53:59.423 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2024-03-23T14:53:59.733 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-certifi. 2024-03-23T14:53:59.749 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../056-python3-certifi_2020.6.20-1_all.deb ... 2024-03-23T14:53:59.783 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-03-23T14:53:59.993 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-waitress. 2024-03-23T14:54:00.020 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../049-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-03-23T14:54:00.063 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-03-23T14:54:00.119 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-idna. 2024-03-23T14:54:00.146 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../057-python3-idna_3.3-1_all.deb ... 2024-03-23T14:54:00.194 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-idna (3.3-1) ... 2024-03-23T14:54:00.414 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-tempita. 2024-03-23T14:54:00.439 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../050-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-03-23T14:54:00.488 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-03-23T14:54:00.497 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-urllib3. 2024-03-23T14:54:00.524 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../058-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-03-23T14:54:00.564 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-03-23T14:54:00.857 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-paste. 2024-03-23T14:54:00.883 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../051-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-03-23T14:54:00.900 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-requests. 2024-03-23T14:54:00.927 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../059-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-03-23T14:54:00.932 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-03-23T14:54:00.967 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-03-23T14:54:01.278 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-werkzeug. 2024-03-23T14:54:01.305 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../060-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2024-03-23T14:54:01.310 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-03-23T14:54:01.325 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../052-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-03-23T14:54:01.353 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-03-23T14:54:01.376 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-03-23T14:54:01.647 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-03-23T14:54:01.675 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../061-ceph-mgr-modules-core_19.0.0-1855-g21b3bd67-1jammy_all.deb ... 2024-03-23T14:54:01.714 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking ceph-mgr-modules-core (19.0.0-1855-g21b3bd67-1jammy) ... 2024-03-23T14:54:01.746 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-pastedeploy. 2024-03-23T14:54:01.772 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../053-python3-pastedeploy_2.1.1-1_all.deb ... 2024-03-23T14:54:01.821 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-03-23T14:54:02.060 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-03-23T14:54:02.087 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../062-libsqlite3-mod-ceph_19.0.0-1855-g21b3bd67-1jammy_amd64.deb ... 2024-03-23T14:54:02.127 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking libsqlite3-mod-ceph (19.0.0-1855-g21b3bd67-1jammy) ... 2024-03-23T14:54:02.190 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-webtest. 2024-03-23T14:54:02.217 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../054-python3-webtest_2.0.35-1_all.deb ... 2024-03-23T14:54:02.266 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-03-23T14:54:02.438 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-bcrypt. 2024-03-23T14:54:02.465 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../063-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-03-23T14:54:02.504 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-03-23T14:54:02.585 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-pecan. 2024-03-23T14:54:02.612 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../055-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-03-23T14:54:02.651 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-03-23T14:54:02.807 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-openssl. 2024-03-23T14:54:02.834 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../064-python3-openssl_21.0.0-1_all.deb ... 2024-03-23T14:54:02.874 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-03-23T14:54:03.015 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-certifi. 2024-03-23T14:54:03.041 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../056-python3-certifi_2020.6.20-1_all.deb ... 2024-03-23T14:54:03.081 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-03-23T14:54:03.168 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package ceph-mgr. 2024-03-23T14:54:03.196 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../065-ceph-mgr_19.0.0-1855-g21b3bd67-1jammy_amd64.deb ... 2024-03-23T14:54:03.226 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking ceph-mgr (19.0.0-1855-g21b3bd67-1jammy) ... 2024-03-23T14:54:03.367 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-idna. 2024-03-23T14:54:03.383 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../057-python3-idna_3.3-1_all.deb ... 2024-03-23T14:54:03.417 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-idna (3.3-1) ... 2024-03-23T14:54:03.630 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package ceph-mon. 2024-03-23T14:54:03.657 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../066-ceph-mon_19.0.0-1855-g21b3bd67-1jammy_amd64.deb ... 2024-03-23T14:54:03.696 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking ceph-mon (19.0.0-1855-g21b3bd67-1jammy) ... 2024-03-23T14:54:03.712 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-urllib3. 2024-03-23T14:54:03.738 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../058-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-03-23T14:54:03.778 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-03-23T14:54:04.081 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-requests. 2024-03-23T14:54:04.108 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../059-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-03-23T14:54:04.148 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-03-23T14:54:04.275 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package ceph-osd. 2024-03-23T14:54:04.301 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../067-ceph-osd_19.0.0-1855-g21b3bd67-1jammy_amd64.deb ... 2024-03-23T14:54:04.344 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking ceph-osd (19.0.0-1855-g21b3bd67-1jammy) ... 2024-03-23T14:54:04.442 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-werkzeug. 2024-03-23T14:54:04.469 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../060-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2024-03-23T14:54:04.509 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-03-23T14:54:04.786 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-03-23T14:54:04.802 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../061-ceph-mgr-modules-core_19.0.0-1855-g21b3bd67-1jammy_all.deb ... 2024-03-23T14:54:04.845 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking ceph-mgr-modules-core (19.0.0-1855-g21b3bd67-1jammy) ... 2024-03-23T14:54:05.250 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-03-23T14:54:05.277 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../062-libsqlite3-mod-ceph_19.0.0-1855-g21b3bd67-1jammy_amd64.deb ... 2024-03-23T14:54:05.316 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking libsqlite3-mod-ceph (19.0.0-1855-g21b3bd67-1jammy) ... 2024-03-23T14:54:05.419 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package ceph. 2024-03-23T14:54:05.434 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../068-ceph_19.0.0-1855-g21b3bd67-1jammy_amd64.deb ... 2024-03-23T14:54:05.477 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking ceph (19.0.0-1855-g21b3bd67-1jammy) ... 2024-03-23T14:54:05.602 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-bcrypt. 2024-03-23T14:54:05.629 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../063-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-03-23T14:54:05.660 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-03-23T14:54:05.797 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package ceph-fuse. 2024-03-23T14:54:05.823 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../069-ceph-fuse_19.0.0-1855-g21b3bd67-1jammy_amd64.deb ... 2024-03-23T14:54:05.863 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking ceph-fuse (19.0.0-1855-g21b3bd67-1jammy) ... 2024-03-23T14:54:05.921 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-openssl. 2024-03-23T14:54:05.949 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../064-python3-openssl_21.0.0-1_all.deb ... 2024-03-23T14:54:05.988 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-03-23T14:54:06.191 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package ceph-mds. 2024-03-23T14:54:06.218 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../070-ceph-mds_19.0.0-1855-g21b3bd67-1jammy_amd64.deb ... 2024-03-23T14:54:06.249 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking ceph-mds (19.0.0-1855-g21b3bd67-1jammy) ... 2024-03-23T14:54:06.324 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package ceph-mgr. 2024-03-23T14:54:06.351 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../065-ceph-mgr_19.0.0-1855-g21b3bd67-1jammy_amd64.deb ... 2024-03-23T14:54:06.399 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking ceph-mgr (19.0.0-1855-g21b3bd67-1jammy) ... 2024-03-23T14:54:06.644 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package cephadm. 2024-03-23T14:54:06.671 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../071-cephadm_19.0.0-1855-g21b3bd67-1jammy_amd64.deb ... 2024-03-23T14:54:06.710 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking cephadm (19.0.0-1855-g21b3bd67-1jammy) ... 2024-03-23T14:54:06.794 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package ceph-mon. 2024-03-23T14:54:06.821 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../066-ceph-mon_19.0.0-1855-g21b3bd67-1jammy_amd64.deb ... 2024-03-23T14:54:06.860 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking ceph-mon (19.0.0-1855-g21b3bd67-1jammy) ... 2024-03-23T14:54:07.030 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-asyncssh. 2024-03-23T14:54:07.057 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../072-python3-asyncssh_2.5.0-1_all.deb ... 2024-03-23T14:54:07.096 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-03-23T14:54:07.389 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package ceph-osd. 2024-03-23T14:54:07.416 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../067-ceph-osd_19.0.0-1855-g21b3bd67-1jammy_amd64.deb ... 2024-03-23T14:54:07.441 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python-babel-localedata. 2024-03-23T14:54:07.447 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking ceph-osd (19.0.0-1855-g21b3bd67-1jammy) ... 2024-03-23T14:54:07.457 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../073-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-03-23T14:54:07.499 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-03-23T14:54:08.429 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-babel. 2024-03-23T14:54:08.445 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../074-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-03-23T14:54:08.478 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-03-23T14:54:08.534 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package ceph. 2024-03-23T14:54:08.549 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../068-ceph_19.0.0-1855-g21b3bd67-1jammy_amd64.deb ... 2024-03-23T14:54:08.592 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking ceph (19.0.0-1855-g21b3bd67-1jammy) ... 2024-03-23T14:54:08.773 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-jinja2. 2024-03-23T14:54:08.800 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../075-python3-jinja2_3.0.3-1ubuntu0.1_all.deb ... 2024-03-23T14:54:08.831 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-03-23T14:54:08.894 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package ceph-fuse. 2024-03-23T14:54:08.921 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../069-ceph-fuse_19.0.0-1855-g21b3bd67-1jammy_amd64.deb ... 2024-03-23T14:54:08.961 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking ceph-fuse (19.0.0-1855-g21b3bd67-1jammy) ... 2024-03-23T14:54:09.126 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-03-23T14:54:09.153 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../076-ceph-mgr-cephadm_19.0.0-1855-g21b3bd67-1jammy_all.deb ... 2024-03-23T14:54:09.184 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking ceph-mgr-cephadm (19.0.0-1855-g21b3bd67-1jammy) ... 2024-03-23T14:54:09.339 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package ceph-mds. 2024-03-23T14:54:09.366 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../070-ceph-mds_19.0.0-1855-g21b3bd67-1jammy_amd64.deb ... 2024-03-23T14:54:09.405 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking ceph-mds (19.0.0-1855-g21b3bd67-1jammy) ... 2024-03-23T14:54:09.529 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-repoze.lru. 2024-03-23T14:54:09.556 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../077-python3-repoze.lru_0.7-2_all.deb ... 2024-03-23T14:54:09.587 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-03-23T14:54:09.793 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package cephadm. 2024-03-23T14:54:09.821 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../071-cephadm_19.0.0-1855-g21b3bd67-1jammy_amd64.deb ... 2024-03-23T14:54:09.848 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-routes. 2024-03-23T14:54:09.860 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking cephadm (19.0.0-1855-g21b3bd67-1jammy) ... 2024-03-23T14:54:09.875 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../078-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-03-23T14:54:09.915 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-03-23T14:54:10.210 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-03-23T14:54:10.236 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../079-ceph-mgr-dashboard_19.0.0-1855-g21b3bd67-1jammy_all.deb ... 2024-03-23T14:54:10.267 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking ceph-mgr-dashboard (19.0.0-1855-g21b3bd67-1jammy) ... 2024-03-23T14:54:10.346 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-asyncssh. 2024-03-23T14:54:10.373 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../072-python3-asyncssh_2.5.0-1_all.deb ... 2024-03-23T14:54:10.412 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-03-23T14:54:10.688 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-03-23T14:54:10.703 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../080-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-03-23T14:54:10.746 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-03-23T14:54:10.748 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python-babel-localedata. 2024-03-23T14:54:10.764 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../073-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-03-23T14:54:10.806 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-03-23T14:54:11.258 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-joblib. 2024-03-23T14:54:11.285 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../081-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-03-23T14:54:11.324 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-03-23T14:54:11.677 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-03-23T14:54:11.705 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../082-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-03-23T14:54:11.829 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-03-23T14:54:11.844 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-babel. 2024-03-23T14:54:11.861 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../074-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-03-23T14:54:11.903 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-03-23T14:54:12.132 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-sklearn. 2024-03-23T14:54:12.159 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../083-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-03-23T14:54:12.164 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-jinja2. 2024-03-23T14:54:12.191 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../075-python3-jinja2_3.0.3-1ubuntu0.1_all.deb ... 2024-03-23T14:54:12.198 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-03-23T14:54:12.222 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-03-23T14:54:12.542 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-03-23T14:54:12.569 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../076-ceph-mgr-cephadm_19.0.0-1855-g21b3bd67-1jammy_all.deb ... 2024-03-23T14:54:12.600 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking ceph-mgr-cephadm (19.0.0-1855-g21b3bd67-1jammy) ... 2024-03-23T14:54:12.849 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-03-23T14:54:12.877 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../084-ceph-mgr-diskprediction-local_19.0.0-1855-g21b3bd67-1jammy_all.deb ... 2024-03-23T14:54:12.886 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-repoze.lru. 2024-03-23T14:54:12.902 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../077-python3-repoze.lru_0.7-2_all.deb ... 2024-03-23T14:54:12.915 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking ceph-mgr-diskprediction-local (19.0.0-1855-g21b3bd67-1jammy) ... 2024-03-23T14:54:12.936 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-03-23T14:54:13.239 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-routes. 2024-03-23T14:54:13.267 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../078-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-03-23T14:54:13.306 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-03-23T14:54:13.642 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-03-23T14:54:13.669 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../079-ceph-mgr-dashboard_19.0.0-1855-g21b3bd67-1jammy_all.deb ... 2024-03-23T14:54:13.709 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking ceph-mgr-dashboard (19.0.0-1855-g21b3bd67-1jammy) ... 2024-03-23T14:54:13.902 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-cachetools. 2024-03-23T14:54:13.929 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../085-python3-cachetools_5.0.0-1_all.deb ... 2024-03-23T14:54:13.968 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-03-23T14:54:14.145 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-03-23T14:54:14.161 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../080-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-03-23T14:54:14.204 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-03-23T14:54:14.271 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-pyasn1. 2024-03-23T14:54:14.299 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../086-python3-pyasn1_0.4.8-1_all.deb ... 2024-03-23T14:54:14.357 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-03-23T14:54:14.699 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-joblib. 2024-03-23T14:54:14.726 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../081-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-03-23T14:54:14.765 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-03-23T14:54:14.908 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-03-23T14:54:14.936 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../087-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-03-23T14:54:14.974 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-03-23T14:54:15.135 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-03-23T14:54:15.163 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../082-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-03-23T14:54:15.287 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-03-23T14:54:15.302 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-rsa. 2024-03-23T14:54:15.329 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../088-python3-rsa_4.8-1_all.deb ... 2024-03-23T14:54:15.369 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-rsa (4.8-1) ... 2024-03-23T14:54:15.540 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-sklearn. 2024-03-23T14:54:15.556 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../083-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-03-23T14:54:15.590 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-03-23T14:54:15.656 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-google-auth. 2024-03-23T14:54:15.683 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../089-python3-google-auth_1.5.1-3_all.deb ... 2024-03-23T14:54:15.723 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-03-23T14:54:16.042 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-03-23T14:54:16.069 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../090-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-03-23T14:54:16.108 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-03-23T14:54:16.215 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-03-23T14:54:16.243 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../084-ceph-mgr-diskprediction-local_19.0.0-1855-g21b3bd67-1jammy_all.deb ... 2024-03-23T14:54:16.282 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking ceph-mgr-diskprediction-local (19.0.0-1855-g21b3bd67-1jammy) ... 2024-03-23T14:54:16.378 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-websocket. 2024-03-23T14:54:16.394 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../091-python3-websocket_1.2.3-1_all.deb ... 2024-03-23T14:54:16.428 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-03-23T14:54:16.698 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-kubernetes. 2024-03-23T14:54:16.725 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../092-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-03-23T14:54:16.764 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-03-23T14:54:17.243 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-cachetools. 2024-03-23T14:54:17.270 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../085-python3-cachetools_5.0.0-1_all.deb ... 2024-03-23T14:54:17.310 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-03-23T14:54:17.403 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-03-23T14:54:17.437 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../093-ceph-mgr-k8sevents_19.0.0-1855-g21b3bd67-1jammy_all.deb ... 2024-03-23T14:54:17.453 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking ceph-mgr-k8sevents (19.0.0-1855-g21b3bd67-1jammy) ... 2024-03-23T14:54:17.587 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-pyasn1. 2024-03-23T14:54:17.603 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../086-python3-pyasn1_0.4.8-1_all.deb ... 2024-03-23T14:54:17.637 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-03-23T14:54:17.748 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package libonig5:amd64. 2024-03-23T14:54:17.775 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../094-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-03-23T14:54:17.814 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-03-23T14:54:17.949 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-03-23T14:54:17.976 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../087-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-03-23T14:54:18.016 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-03-23T14:54:18.184 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package libjq1:amd64. 2024-03-23T14:54:18.211 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../095-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-03-23T14:54:18.250 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-03-23T14:54:18.319 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-rsa. 2024-03-23T14:54:18.346 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../088-python3-rsa_4.8-1_all.deb ... 2024-03-23T14:54:18.385 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-rsa (4.8-1) ... 2024-03-23T14:54:18.503 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package jq. 2024-03-23T14:54:18.519 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../096-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-03-23T14:54:18.552 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-03-23T14:54:18.722 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-google-auth. 2024-03-23T14:54:18.749 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../089-python3-google-auth_1.5.1-3_all.deb ... 2024-03-23T14:54:18.755 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package socat. 2024-03-23T14:54:18.771 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../097-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-03-23T14:54:18.788 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-03-23T14:54:18.805 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-03-23T14:54:19.100 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package xmlstarlet. 2024-03-23T14:54:19.108 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-03-23T14:54:19.128 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../098-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-03-23T14:54:19.136 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../090-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-03-23T14:54:19.166 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-03-23T14:54:19.175 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-03-23T14:54:19.445 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-websocket. 2024-03-23T14:54:19.472 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../091-python3-websocket_1.2.3-1_all.deb ... 2024-03-23T14:54:19.503 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-03-23T14:54:19.504 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package ceph-test. 2024-03-23T14:54:19.520 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../099-ceph-test_19.0.0-1855-g21b3bd67-1jammy_amd64.deb ... 2024-03-23T14:54:19.571 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking ceph-test (19.0.0-1855-g21b3bd67-1jammy) ... 2024-03-23T14:54:19.806 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-kubernetes. 2024-03-23T14:54:19.833 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../092-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-03-23T14:54:19.872 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-03-23T14:54:20.787 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-03-23T14:54:20.815 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../093-ceph-mgr-k8sevents_19.0.0-1855-g21b3bd67-1jammy_all.deb ... 2024-03-23T14:54:20.853 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking ceph-mgr-k8sevents (19.0.0-1855-g21b3bd67-1jammy) ... 2024-03-23T14:54:21.172 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package libonig5:amd64. 2024-03-23T14:54:21.200 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../094-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-03-23T14:54:21.239 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-03-23T14:54:21.517 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package libjq1:amd64. 2024-03-23T14:54:21.544 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../095-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-03-23T14:54:21.591 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-03-23T14:54:21.861 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package jq. 2024-03-23T14:54:21.877 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../096-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-03-23T14:54:21.916 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package ceph-volume. 2024-03-23T14:54:21.919 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-03-23T14:54:21.944 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../100-ceph-volume_19.0.0-1855-g21b3bd67-1jammy_all.deb ... 2024-03-23T14:54:21.983 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking ceph-volume (19.0.0-1855-g21b3bd67-1jammy) ... 2024-03-23T14:54:22.172 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package socat. 2024-03-23T14:54:22.188 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../097-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-03-23T14:54:22.230 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-03-23T14:54:22.353 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package cephfs-mirror. 2024-03-23T14:54:22.381 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../101-cephfs-mirror_19.0.0-1855-g21b3bd67-1jammy_amd64.deb ... 2024-03-23T14:54:22.420 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking cephfs-mirror (19.0.0-1855-g21b3bd67-1jammy) ... 2024-03-23T14:54:22.533 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package xmlstarlet. 2024-03-23T14:54:22.549 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../098-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-03-23T14:54:22.591 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-03-23T14:54:22.748 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-colorama. 2024-03-23T14:54:22.775 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../102-python3-colorama_0.4.4-1_all.deb ... 2024-03-23T14:54:22.814 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-colorama (0.4.4-1) ... 2024-03-23T14:54:22.954 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package ceph-test. 2024-03-23T14:54:22.982 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../099-ceph-test_19.0.0-1855-g21b3bd67-1jammy_amd64.deb ... 2024-03-23T14:54:23.020 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking ceph-test (19.0.0-1855-g21b3bd67-1jammy) ... 2024-03-23T14:54:23.067 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package cephfs-shell. 2024-03-23T14:54:23.094 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../103-cephfs-shell_19.0.0-1855-g21b3bd67-1jammy_all.deb ... 2024-03-23T14:54:23.133 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking cephfs-shell (19.0.0-1855-g21b3bd67-1jammy) ... 2024-03-23T14:54:23.352 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package cephfs-top. 2024-03-23T14:54:23.368 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../104-cephfs-top_19.0.0-1855-g21b3bd67-1jammy_all.deb ... 2024-03-23T14:54:23.402 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking cephfs-top (19.0.0-1855-g21b3bd67-1jammy) ... 2024-03-23T14:54:23.671 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package libcephfs-dev. 2024-03-23T14:54:23.687 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../105-libcephfs-dev_19.0.0-1855-g21b3bd67-1jammy_amd64.deb ... 2024-03-23T14:54:23.730 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking libcephfs-dev (19.0.0-1855-g21b3bd67-1jammy) ... 2024-03-23T14:54:24.032 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package lua-socket:amd64. 2024-03-23T14:54:24.060 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../106-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-03-23T14:54:24.099 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-03-23T14:54:24.428 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package lua-sec:amd64. 2024-03-23T14:54:24.456 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../107-lua-sec_1.0.2-1_amd64.deb ... 2024-03-23T14:54:24.494 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-03-23T14:54:24.799 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package nvme-cli. 2024-03-23T14:54:24.827 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../108-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2024-03-23T14:54:24.857 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2024-03-23T14:54:25.262 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package pkg-config. 2024-03-23T14:54:25.289 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../109-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-03-23T14:54:25.328 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-03-23T14:54:25.350 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package ceph-volume. 2024-03-23T14:54:25.378 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../100-ceph-volume_19.0.0-1855-g21b3bd67-1jammy_all.deb ... 2024-03-23T14:54:25.416 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking ceph-volume (19.0.0-1855-g21b3bd67-1jammy) ... 2024-03-23T14:54:25.583 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-03-23T14:54:25.599 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../110-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-03-23T14:54:25.641 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-03-23T14:54:25.820 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package cephfs-mirror. 2024-03-23T14:54:25.848 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../101-cephfs-mirror_19.0.0-1855-g21b3bd67-1jammy_amd64.deb ... 2024-03-23T14:54:25.887 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking cephfs-mirror (19.0.0-1855-g21b3bd67-1jammy) ... 2024-03-23T14:54:26.036 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-iniconfig. 2024-03-23T14:54:26.063 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../111-python3-iniconfig_1.1.1-2_all.deb ... 2024-03-23T14:54:26.102 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-03-23T14:54:26.139 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-colorama. 2024-03-23T14:54:26.167 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../102-python3-colorama_0.4.4-1_all.deb ... 2024-03-23T14:54:26.198 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-colorama (0.4.4-1) ... 2024-03-23T14:54:26.405 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-pastescript. 2024-03-23T14:54:26.416 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package cephfs-shell. 2024-03-23T14:54:26.433 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../103-cephfs-shell_19.0.0-1855-g21b3bd67-1jammy_all.deb ... 2024-03-23T14:54:26.433 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../112-python3-pastescript_2.0.2-4_all.deb ... 2024-03-23T14:54:26.471 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-03-23T14:54:26.475 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking cephfs-shell (19.0.0-1855-g21b3bd67-1jammy) ... 2024-03-23T14:54:26.719 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package cephfs-top. 2024-03-23T14:54:26.736 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../104-cephfs-top_19.0.0-1855-g21b3bd67-1jammy_all.deb ... 2024-03-23T14:54:26.777 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking cephfs-top (19.0.0-1855-g21b3bd67-1jammy) ... 2024-03-23T14:54:26.791 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-pluggy. 2024-03-23T14:54:26.819 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../113-python3-pluggy_0.13.0-7.1_all.deb ... 2024-03-23T14:54:26.858 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-03-23T14:54:27.055 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package libcephfs-dev. 2024-03-23T14:54:27.083 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../105-libcephfs-dev_19.0.0-1855-g21b3bd67-1jammy_amd64.deb ... 2024-03-23T14:54:27.119 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-psutil. 2024-03-23T14:54:27.121 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking libcephfs-dev (19.0.0-1855-g21b3bd67-1jammy) ... 2024-03-23T14:54:27.135 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../114-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-03-23T14:54:27.169 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-03-23T14:54:27.424 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package lua-socket:amd64. 2024-03-23T14:54:27.452 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../106-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-03-23T14:54:27.490 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-03-23T14:54:27.505 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-py. 2024-03-23T14:54:27.532 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../115-python3-py_1.10.0-1_all.deb ... 2024-03-23T14:54:27.580 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-py (1.10.0-1) ... 2024-03-23T14:54:27.787 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package lua-sec:amd64. 2024-03-23T14:54:27.815 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../107-lua-sec_1.0.2-1_amd64.deb ... 2024-03-23T14:54:27.844 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-03-23T14:54:27.892 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-pygments. 2024-03-23T14:54:27.919 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../116-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-03-23T14:54:27.958 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-03-23T14:54:28.215 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package nvme-cli. 2024-03-23T14:54:28.242 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../108-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2024-03-23T14:54:28.281 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2024-03-23T14:54:28.403 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-pyinotify. 2024-03-23T14:54:28.419 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../117-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-03-23T14:54:28.453 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-03-23T14:54:28.703 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package pkg-config. 2024-03-23T14:54:28.732 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../109-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-03-23T14:54:28.770 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-03-23T14:54:28.781 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-toml. 2024-03-23T14:54:28.808 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../118-python3-toml_0.10.2-1_all.deb ... 2024-03-23T14:54:28.847 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-03-23T14:54:29.031 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-03-23T14:54:29.059 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../110-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-03-23T14:54:29.097 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-03-23T14:54:29.150 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-pytest. 2024-03-23T14:54:29.178 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../119-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-03-23T14:54:29.216 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-03-23T14:54:29.509 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-iniconfig. 2024-03-23T14:54:29.537 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../111-python3-iniconfig_1.1.1-2_all.deb ... 2024-03-23T14:54:29.553 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package python3-simplejson. 2024-03-23T14:54:29.575 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-03-23T14:54:29.581 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../120-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-03-23T14:54:29.619 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-03-23T14:54:29.836 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-pastescript. 2024-03-23T14:54:29.861 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../112-python3-pastescript_2.0.2-4_all.deb ... 2024-03-23T14:54:29.889 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-03-23T14:54:29.894 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-03-23T14:54:29.905 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../121-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-03-23T14:54:29.947 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-03-23T14:54:30.181 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-pluggy. 2024-03-23T14:54:30.209 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../113-python3-pluggy_0.13.0-7.1_all.deb ... 2024-03-23T14:54:30.248 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-03-23T14:54:30.559 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-psutil. 2024-03-23T14:54:30.587 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../114-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-03-23T14:54:30.625 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-03-23T14:54:30.810 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package radosgw. 2024-03-23T14:54:30.837 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../122-radosgw_19.0.0-1855-g21b3bd67-1jammy_amd64.deb ... 2024-03-23T14:54:30.876 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking radosgw (19.0.0-1855-g21b3bd67-1jammy) ... 2024-03-23T14:54:31.153 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-py. 2024-03-23T14:54:31.181 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../115-python3-py_1.10.0-1_all.deb ... 2024-03-23T14:54:31.220 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-py (1.10.0-1) ... 2024-03-23T14:54:31.548 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-pygments. 2024-03-23T14:54:31.575 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../116-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-03-23T14:54:31.614 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-03-23T14:54:31.637 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package rbd-fuse. 2024-03-23T14:54:31.653 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../123-rbd-fuse_19.0.0-1855-g21b3bd67-1jammy_amd64.deb ... 2024-03-23T14:54:31.695 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking rbd-fuse (19.0.0-1855-g21b3bd67-1jammy) ... 2024-03-23T14:54:31.992 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-03-23T14:54:32.077 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-pyinotify. 2024-03-23T14:54:32.104 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../117-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-03-23T14:54:32.143 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-03-23T14:54:32.250 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-03-23T14:54:32.393 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2024-03-23T14:54:32.455 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-toml. 2024-03-23T14:54:32.483 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../118-python3-toml_0.10.2-1_all.deb ... 2024-03-23T14:54:32.521 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-03-23T14:54:32.657 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-03-23T14:54:32.782 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-pytest. 2024-03-23T14:54:32.810 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../119-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-03-23T14:54:32.840 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-03-23T14:54:32.988 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-03-23T14:54:33.210 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-simplejson. 2024-03-23T14:54:33.238 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../120-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-03-23T14:54:33.276 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-03-23T14:54:33.399 INFO:teuthology.orchestra.run.smithi164.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-03-23T14:54:33.470 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up cephadm (19.0.0-1855-g21b3bd67-1jammy) ... 2024-03-23T14:54:33.546 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-03-23T14:54:33.574 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../121-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-03-23T14:54:33.612 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-03-23T14:54:33.979 INFO:teuthology.orchestra.run.smithi164.stdout:Adding system user cephadm....done 2024-03-23T14:54:34.134 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-03-23T14:54:34.383 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package radosgw. 2024-03-23T14:54:34.411 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../122-radosgw_19.0.0-1855-g21b3bd67-1jammy_amd64.deb ... 2024-03-23T14:54:34.418 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-03-23T14:54:34.449 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking radosgw (19.0.0-1855-g21b3bd67-1jammy) ... 2024-03-23T14:54:34.678 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-03-23T14:54:34.804 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-03-23T14:54:35.055 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-03-23T14:54:35.269 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package rbd-fuse. 2024-03-23T14:54:35.285 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../123-rbd-fuse_19.0.0-1855-g21b3bd67-1jammy_amd64.deb ... 2024-03-23T14:54:35.306 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-03-23T14:54:35.327 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking rbd-fuse (19.0.0-1855-g21b3bd67-1jammy) ... 2024-03-23T14:54:35.416 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-py (1.10.0-1) ... 2024-03-23T14:54:35.624 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-03-23T14:54:35.700 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-03-23T14:54:35.890 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-03-23T14:54:36.016 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2024-03-23T14:54:36.052 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-03-23T14:54:36.274 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-03-23T14:54:36.320 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-colorama (0.4.4-1) ... 2024-03-23T14:54:36.563 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-03-23T14:54:36.616 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-03-23T14:54:36.831 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-03-23T14:54:37.008 INFO:teuthology.orchestra.run.smithi173.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-03-23T14:54:37.099 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-03-23T14:54:37.110 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up cephadm (19.0.0-1855-g21b3bd67-1jammy) ... 2024-03-23T14:54:37.350 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-03-23T14:54:37.609 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-03-23T14:54:37.679 INFO:teuthology.orchestra.run.smithi173.stdout:Adding system user cephadm....done 2024-03-23T14:54:37.850 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-03-23T14:54:37.869 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-ceph-argparse (19.0.0-1855-g21b3bd67-1jammy) ... 2024-03-23T14:54:38.278 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-03-23T14:54:38.388 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-03-23T14:54:38.436 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-03-23T14:54:38.497 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2024-03-23T14:54:38.687 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-03-23T14:54:38.740 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-03-23T14:54:38.805 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-03-23T14:54:38.857 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-03-23T14:54:39.047 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-03-23T14:54:39.092 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-03-23T14:54:39.298 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-03-23T14:54:39.425 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-py (1.10.0-1) ... 2024-03-23T14:54:39.477 INFO:teuthology.orchestra.run.smithi164.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-03-23T14:54:39.522 INFO:teuthology.orchestra.run.smithi164.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-03-23T14:54:39.586 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-03-23T14:54:39.712 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-03-23T14:54:39.726 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-03-23T14:54:40.055 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-03-23T14:54:40.086 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-03-23T14:54:40.190 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-03-23T14:54:40.321 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-colorama (0.4.4-1) ... 2024-03-23T14:54:40.441 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-03-23T14:54:40.564 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-03-23T14:54:40.700 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-03-23T14:54:40.831 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-03-23T14:54:40.993 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up zip (3.0-12build2) ... 2024-03-23T14:54:41.066 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-03-23T14:54:41.119 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-03-23T14:54:41.631 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-03-23T14:54:41.762 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-03-23T14:54:41.952 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-03-23T14:54:42.013 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-03-23T14:54:42.139 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-03-23T14:54:42.220 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-ceph-argparse (19.0.0-1855-g21b3bd67-1jammy) ... 2024-03-23T14:54:42.257 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-03-23T14:54:42.462 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-03-23T14:54:42.533 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-03-23T14:54:42.588 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-03-23T14:54:42.714 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2024-03-23T14:54:42.901 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-03-23T14:54:42.973 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-03-23T14:54:43.099 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-03-23T14:54:43.143 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-03-23T14:54:43.359 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-03-23T14:54:43.512 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-03-23T14:54:43.728 INFO:teuthology.orchestra.run.smithi173.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-03-23T14:54:43.757 INFO:teuthology.orchestra.run.smithi173.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-03-23T14:54:43.804 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-03-23T14:54:43.821 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2024-03-23T14:54:43.921 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-03-23T14:54:44.122 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-03-23T14:54:44.273 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-03-23T14:54:44.398 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-03-23T14:54:44.465 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-03-23T14:54:44.658 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-03-23T14:54:44.708 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-03-23T14:54:44.817 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-ceph-common (19.0.0-1855-g21b3bd67-1jammy) ... 2024-03-23T14:54:44.925 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-03-23T14:54:45.118 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-idna (3.3-1) ... 2024-03-23T14:54:45.226 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up zip (3.0-12build2) ... 2024-03-23T14:54:45.336 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-03-23T14:54:45.595 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-03-23T14:54:46.005 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-03-23T14:54:46.239 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-03-23T14:54:46.365 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-03-23T14:54:46.491 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-03-23T14:54:46.792 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-03-23T14:54:46.834 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-03-23T14:54:47.085 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-23T14:54:47.176 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-03-23T14:54:47.219 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-toml (0.10.2-1) ... 2024-03-23T14:54:47.436 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-03-23T14:54:47.478 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-03-23T14:54:47.579 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-03-23T14:54:47.680 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-03-23T14:54:47.804 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-03-23T14:54:47.982 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-03-23T14:54:48.097 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2024-03-23T14:54:48.233 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-03-23T14:54:48.389 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-03-23T14:54:48.467 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-23T14:54:48.593 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-03-23T14:54:48.724 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-03-23T14:54:48.886 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-03-23T14:54:48.983 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-03-23T14:54:49.109 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-ceph-common (19.0.0-1855-g21b3bd67-1jammy) ... 2024-03-23T14:54:49.137 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-03-23T14:54:49.380 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-03-23T14:54:49.419 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-idna (3.3-1) ... 2024-03-23T14:54:49.631 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-03-23T14:54:49.728 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-03-23T14:54:49.882 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-03-23T14:54:50.133 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-03-23T14:54:50.268 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-03-23T14:54:50.544 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-03-23T14:54:50.653 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-03-23T14:54:50.818 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-03-23T14:54:50.929 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2024-03-23T14:54:51.086 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-23T14:54:51.195 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-toml (0.10.2-1) ... 2024-03-23T14:54:51.222 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-03-23T14:54:51.446 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-03-23T14:54:51.481 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-03-23T14:54:51.573 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-03-23T14:54:51.582 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-03-23T14:54:51.683 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-03-23T14:54:51.698 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-03-23T14:54:51.755 INFO:teuthology.orchestra.run.smithi164.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-03-23T14:54:51.999 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-03-23T14:54:52.010 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-03-23T14:54:52.136 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-03-23T14:54:52.258 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-03-23T14:54:52.510 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-23T14:54:52.512 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-03-23T14:54:52.638 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-03-23T14:54:52.780 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up lua-any (27ubuntu1) ... 2024-03-23T14:54:52.915 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-portend (3.0.0-1) ... 2024-03-23T14:54:52.939 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-03-23T14:54:53.165 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-23T14:54:53.190 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-03-23T14:54:53.291 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-03-23T14:54:53.450 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-03-23T14:54:53.601 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-03-23T14:54:53.709 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-03-23T14:54:53.927 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-03-23T14:54:53.969 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-03-23T14:54:54.195 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-03-23T14:54:54.220 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-03-23T14:54:54.354 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-03-23T14:54:54.415 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-rsa (4.8-1) ... 2024-03-23T14:54:54.631 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-03-23T14:54:54.674 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-03-23T14:54:54.757 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-03-23T14:54:54.942 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-03-23T14:54:55.049 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2024-03-23T14:54:55.193 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-03-23T14:54:55.359 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-03-23T14:54:55.561 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-03-23T14:54:55.602 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-03-23T14:54:55.703 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-03-23T14:54:55.804 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-03-23T14:54:55.837 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-03-23T14:54:55.893 INFO:teuthology.orchestra.run.smithi173.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-03-23T14:54:56.123 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-03-23T14:54:56.182 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-03-23T14:54:56.249 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-03-23T14:54:56.283 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up librados2 (19.0.0-1855-g21b3bd67-1jammy) ... 2024-03-23T14:54:56.384 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up libsqlite3-mod-ceph (19.0.0-1855-g21b3bd67-1jammy) ... 2024-03-23T14:54:56.510 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-03-23T14:54:56.667 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-03-23T14:54:56.862 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up libcephfs2 (19.0.0-1855-g21b3bd67-1jammy) ... 2024-03-23T14:54:56.927 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up lua-any (27ubuntu1) ... 2024-03-23T14:54:56.988 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up libradosstriper1 (19.0.0-1855-g21b3bd67-1jammy) ... 2024-03-23T14:54:57.053 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-portend (3.0.0-1) ... 2024-03-23T14:54:57.123 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-03-23T14:54:57.312 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-23T14:54:57.399 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up librbd1 (19.0.0-1855-g21b3bd67-1jammy) ... 2024-03-23T14:54:57.438 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-03-23T14:54:57.500 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up ceph-mgr-modules-core (19.0.0-1855-g21b3bd67-1jammy) ... 2024-03-23T14:54:57.618 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up ceph-fuse (19.0.0-1855-g21b3bd67-1jammy) ... 2024-03-23T14:54:57.756 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-03-23T14:54:57.831 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-03-23T14:54:57.831 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-03-23T14:54:58.041 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-03-23T14:54:58.317 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-03-23T14:54:58.344 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up libcephfs-dev (19.0.0-1855-g21b3bd67-1jammy) ... 2024-03-23T14:54:58.470 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-rados (19.0.0-1855-g21b3bd67-1jammy) ... 2024-03-23T14:54:58.553 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-rsa (4.8-1) ... 2024-03-23T14:54:58.596 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up librgw2 (19.0.0-1855-g21b3bd67-1jammy) ... 2024-03-23T14:54:58.722 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up cephfs-top (19.0.0-1855-g21b3bd67-1jammy) ... 2024-03-23T14:54:58.813 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-03-23T14:54:58.856 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-03-23T14:54:59.090 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-03-23T14:54:59.350 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-03-23T14:54:59.727 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-03-23T14:55:00.003 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-03-23T14:55:00.068 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-rbd (19.0.0-1855-g21b3bd67-1jammy) ... 2024-03-23T14:55:00.169 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up rbd-fuse (19.0.0-1855-g21b3bd67-1jammy) ... 2024-03-23T14:55:00.270 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-rgw (19.0.0-1855-g21b3bd67-1jammy) ... 2024-03-23T14:55:00.322 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-03-23T14:55:00.371 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up python3-cephfs (19.0.0-1855-g21b3bd67-1jammy) ... 2024-03-23T14:55:00.449 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up librados2 (19.0.0-1855-g21b3bd67-1jammy) ... 2024-03-23T14:55:00.497 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up cephfs-shell (19.0.0-1855-g21b3bd67-1jammy) ... 2024-03-23T14:55:00.575 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up libsqlite3-mod-ceph (19.0.0-1855-g21b3bd67-1jammy) ... 2024-03-23T14:55:00.632 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up ceph-common (19.0.0-1855-g21b3bd67-1jammy) ... 2024-03-23T14:55:00.701 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-03-23T14:55:01.029 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up libcephfs2 (19.0.0-1855-g21b3bd67-1jammy) ... 2024-03-23T14:55:01.130 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up libradosstriper1 (19.0.0-1855-g21b3bd67-1jammy) ... 2024-03-23T14:55:01.149 INFO:teuthology.orchestra.run.smithi164.stdout:Adding group ceph....done 2024-03-23T14:55:01.256 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-03-23T14:55:01.533 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up librbd1 (19.0.0-1855-g21b3bd67-1jammy) ... 2024-03-23T14:55:01.542 INFO:teuthology.orchestra.run.smithi164.stdout:Adding system user ceph....done 2024-03-23T14:55:01.659 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up ceph-mgr-modules-core (19.0.0-1855-g21b3bd67-1jammy) ... 2024-03-23T14:55:01.711 INFO:teuthology.orchestra.run.smithi164.stdout:Setting system user ceph properties....done 2024-03-23T14:55:01.728 INFO:teuthology.orchestra.run.smithi164.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-03-23T14:55:01.827 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up ceph-fuse (19.0.0-1855-g21b3bd67-1jammy) ... 2024-03-23T14:55:01.860 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-03-23T14:55:02.030 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-03-23T14:55:02.030 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-03-23T14:55:02.186 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-03-23T14:55:02.526 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up libcephfs-dev (19.0.0-1855-g21b3bd67-1jammy) ... 2024-03-23T14:55:02.655 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-rados (19.0.0-1855-g21b3bd67-1jammy) ... 2024-03-23T14:55:02.781 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up librgw2 (19.0.0-1855-g21b3bd67-1jammy) ... 2024-03-23T14:55:02.875 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up cephfs-mirror (19.0.0-1855-g21b3bd67-1jammy) ... 2024-03-23T14:55:02.907 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up cephfs-top (19.0.0-1855-g21b3bd67-1jammy) ... 2024-03-23T14:55:03.050 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-03-23T14:55:03.099 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-03-23T14:55:03.099 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-03-23T14:55:03.627 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up ceph-test (19.0.0-1855-g21b3bd67-1jammy) ... 2024-03-23T14:55:03.727 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up radosgw (19.0.0-1855-g21b3bd67-1jammy) ... 2024-03-23T14:55:04.175 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-03-23T14:55:04.175 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-03-23T14:55:04.278 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-rbd (19.0.0-1855-g21b3bd67-1jammy) ... 2024-03-23T14:55:04.388 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up rbd-fuse (19.0.0-1855-g21b3bd67-1jammy) ... 2024-03-23T14:55:04.514 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-rgw (19.0.0-1855-g21b3bd67-1jammy) ... 2024-03-23T14:55:04.604 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up ceph-base (19.0.0-1855-g21b3bd67-1jammy) ... 2024-03-23T14:55:04.623 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-cephfs (19.0.0-1855-g21b3bd67-1jammy) ... 2024-03-23T14:55:04.749 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up cephfs-shell (19.0.0-1855-g21b3bd67-1jammy) ... 2024-03-23T14:55:04.876 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up ceph-common (19.0.0-1855-g21b3bd67-1jammy) ... 2024-03-23T14:55:04.952 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-03-23T14:55:05.414 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up ceph-mds (19.0.0-1855-g21b3bd67-1jammy) ... 2024-03-23T14:55:05.419 INFO:teuthology.orchestra.run.smithi173.stdout:Adding group ceph....done 2024-03-23T14:55:05.640 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-03-23T14:55:05.640 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-03-23T14:55:05.804 INFO:teuthology.orchestra.run.smithi173.stdout:Adding system user ceph....done 2024-03-23T14:55:05.973 INFO:teuthology.orchestra.run.smithi173.stdout:Setting system user ceph properties....done 2024-03-23T14:55:05.992 INFO:teuthology.orchestra.run.smithi173.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-03-23T14:55:06.128 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-03-23T14:55:06.408 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up ceph-mgr (19.0.0-1855-g21b3bd67-1jammy) ... 2024-03-23T14:55:06.444 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-03-23T14:55:06.622 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-03-23T14:55:06.622 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-03-23T14:55:07.011 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up cephfs-mirror (19.0.0-1855-g21b3bd67-1jammy) ... 2024-03-23T14:55:07.119 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up ceph-osd (19.0.0-1855-g21b3bd67-1jammy) ... 2024-03-23T14:55:07.215 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-03-23T14:55:07.215 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-03-23T14:55:07.375 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-03-23T14:55:07.375 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-03-23T14:55:07.704 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up ceph-test (19.0.0-1855-g21b3bd67-1jammy) ... 2024-03-23T14:55:07.822 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up radosgw (19.0.0-1855-g21b3bd67-1jammy) ... 2024-03-23T14:55:07.889 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up ceph-mgr-k8sevents (19.0.0-1855-g21b3bd67-1jammy) ... 2024-03-23T14:55:08.141 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up ceph-mgr-diskprediction-local (19.0.0-1855-g21b3bd67-1jammy) ... 2024-03-23T14:55:08.294 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-03-23T14:55:08.294 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-03-23T14:55:08.351 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up ceph-mon (19.0.0-1855-g21b3bd67-1jammy) ... 2024-03-23T14:55:08.615 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-03-23T14:55:08.615 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-03-23T14:55:08.808 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up ceph-base (19.0.0-1855-g21b3bd67-1jammy) ... 2024-03-23T14:55:09.092 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up ceph-mgr-cephadm (19.0.0-1855-g21b3bd67-1jammy) ... 2024-03-23T14:55:09.234 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up ceph (19.0.0-1855-g21b3bd67-1jammy) ... 2024-03-23T14:55:09.262 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-03-23T14:55:09.360 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up ceph-mgr-dashboard (19.0.0-1855-g21b3bd67-1jammy) ... 2024-03-23T14:55:09.511 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up ceph-volume (19.0.0-1855-g21b3bd67-1jammy) ... 2024-03-23T14:55:09.761 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up ceph-mds (19.0.0-1855-g21b3bd67-1jammy) ... 2024-03-23T14:55:09.839 INFO:teuthology.orchestra.run.smithi164.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-03-23T14:55:09.940 INFO:teuthology.orchestra.run.smithi164.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-03-23T14:55:09.974 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-03-23T14:55:09.975 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-03-23T14:55:10.101 INFO:teuthology.orchestra.run.smithi164.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-03-23T14:55:10.480 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up ceph-mgr (19.0.0-1855-g21b3bd67-1jammy) ... 2024-03-23T14:55:10.679 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-03-23T14:55:10.679 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-03-23T14:55:10.808 INFO:teuthology.orchestra.run.smithi164.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-03-23T14:55:11.207 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up ceph-osd (19.0.0-1855-g21b3bd67-1jammy) ... 2024-03-23T14:55:11.464 INFO:teuthology.orchestra.run.smithi164.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-03-23T14:55:11.467 DEBUG:teuthology.orchestra.run.smithi164:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2024-03-23T14:55:11.492 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-03-23T14:55:11.492 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-03-23T14:55:11.519 INFO:teuthology.orchestra.run.smithi164.stdout:Reading package lists... 2024-03-23T14:55:11.716 INFO:teuthology.orchestra.run.smithi164.stdout:Building dependency tree... 2024-03-23T14:55:11.717 INFO:teuthology.orchestra.run.smithi164.stdout:Reading state information... 2024-03-23T14:55:11.895 INFO:teuthology.orchestra.run.smithi164.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2024-03-23T14:55:11.895 INFO:teuthology.orchestra.run.smithi164.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2024-03-23T14:55:11.895 INFO:teuthology.orchestra.run.smithi164.stdout:util-linux is already the newest version (2.37.2-4ubuntu3). 2024-03-23T14:55:11.895 INFO:teuthology.orchestra.run.smithi164.stdout:uuid-dev is already the newest version (2.37.2-4ubuntu3). 2024-03-23T14:55:11.896 INFO:teuthology.orchestra.run.smithi164.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2024-03-23T14:55:11.896 INFO:teuthology.orchestra.run.smithi164.stdout:The following packages were automatically installed and are no longer required: 2024-03-23T14:55:11.896 INFO:teuthology.orchestra.run.smithi164.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-03-23T14:55:11.896 INFO:teuthology.orchestra.run.smithi164.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-03-23T14:55:11.896 INFO:teuthology.orchestra.run.smithi164.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-03-23T14:55:11.897 INFO:teuthology.orchestra.run.smithi164.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-03-23T14:55:11.897 INFO:teuthology.orchestra.run.smithi164.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-03-23T14:55:11.897 INFO:teuthology.orchestra.run.smithi164.stdout:Use 'sudo apt autoremove' to remove them. 2024-03-23T14:55:11.897 INFO:teuthology.orchestra.run.smithi164.stdout:The following additional packages will be installed: 2024-03-23T14:55:11.897 INFO:teuthology.orchestra.run.smithi164.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2024-03-23T14:55:11.898 INFO:teuthology.orchestra.run.smithi164.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2024-03-23T14:55:11.898 INFO:teuthology.orchestra.run.smithi164.stdout: libqmi-proxy libssl3 libteamdctl0 modemmanager network-manager-pptp 2024-03-23T14:55:11.898 INFO:teuthology.orchestra.run.smithi164.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2024-03-23T14:55:11.898 INFO:teuthology.orchestra.run.smithi164.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2024-03-23T14:55:11.899 INFO:teuthology.orchestra.run.smithi164.stdout:Suggested packages: 2024-03-23T14:55:11.899 INFO:teuthology.orchestra.run.smithi164.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2024-03-23T14:55:11.899 INFO:teuthology.orchestra.run.smithi164.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial wpagui 2024-03-23T14:55:11.899 INFO:teuthology.orchestra.run.smithi164.stdout: libengine-pkcs11-openssl 2024-03-23T14:55:11.937 INFO:teuthology.orchestra.run.smithi164.stdout:The following NEW packages will be installed: 2024-03-23T14:55:11.938 INFO:teuthology.orchestra.run.smithi164.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2024-03-23T14:55:11.938 INFO:teuthology.orchestra.run.smithi164.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2024-03-23T14:55:11.939 INFO:teuthology.orchestra.run.smithi164.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2024-03-23T14:55:11.939 INFO:teuthology.orchestra.run.smithi164.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2024-03-23T14:55:11.939 INFO:teuthology.orchestra.run.smithi164.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2024-03-23T14:55:11.939 INFO:teuthology.orchestra.run.smithi164.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2024-03-23T14:55:11.939 INFO:teuthology.orchestra.run.smithi164.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2024-03-23T14:55:11.940 INFO:teuthology.orchestra.run.smithi164.stdout:The following packages will be upgraded: 2024-03-23T14:55:11.941 INFO:teuthology.orchestra.run.smithi164.stdout: libssl3 2024-03-23T14:55:12.121 INFO:teuthology.orchestra.run.smithi164.stdout:1 upgraded, 39 newly installed, 0 to remove and 284 not upgraded. 2024-03-23T14:55:12.122 INFO:teuthology.orchestra.run.smithi164.stdout:Need to get 54.6 MB of archives. 2024-03-23T14:55:12.122 INFO:teuthology.orchestra.run.smithi164.stdout:After this operation, 202 MB of additional disk space will be used. 2024-03-23T14:55:12.122 INFO:teuthology.orchestra.run.smithi164.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2024-03-23T14:55:12.171 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up ceph-mgr-k8sevents (19.0.0-1855-g21b3bd67-1jammy) ... 2024-03-23T14:55:12.597 INFO:teuthology.orchestra.run.smithi164.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.15 [1,905 kB] 2024-03-23T14:55:12.735 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up ceph-mgr-diskprediction-local (19.0.0-1855-g21b3bd67-1jammy) ... 2024-03-23T14:55:12.855 INFO:teuthology.orchestra.run.smithi164.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2024-03-23T14:55:12.886 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up ceph-mon (19.0.0-1855-g21b3bd67-1jammy) ... 2024-03-23T14:55:12.889 INFO:teuthology.orchestra.run.smithi164.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2024-03-23T14:55:12.891 INFO:teuthology.orchestra.run.smithi164.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2024-03-23T14:55:12.896 INFO:teuthology.orchestra.run.smithi164.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2024-03-23T14:55:12.899 INFO:teuthology.orchestra.run.smithi164.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2024-03-23T14:55:12.900 INFO:teuthology.orchestra.run.smithi164.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libbluetooth3 amd64 5.64-0ubuntu1.1 [87.0 kB] 2024-03-23T14:55:12.903 INFO:teuthology.orchestra.run.smithi164.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2024-03-23T14:55:12.905 INFO:teuthology.orchestra.run.smithi164.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2024-03-23T14:55:13.071 INFO:teuthology.orchestra.run.smithi164.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2024-03-23T14:55:13.071 INFO:teuthology.orchestra.run.smithi164.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2024-03-23T14:55:13.086 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-03-23T14:55:13.086 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-03-23T14:55:13.328 INFO:teuthology.orchestra.run.smithi164.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2024-03-23T14:55:13.571 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up ceph-mgr-cephadm (19.0.0-1855-g21b3bd67-1jammy) ... 2024-03-23T14:55:13.706 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up ceph (19.0.0-1855-g21b3bd67-1jammy) ... 2024-03-23T14:55:13.823 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up ceph-mgr-dashboard (19.0.0-1855-g21b3bd67-1jammy) ... 2024-03-23T14:55:13.974 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up ceph-volume (19.0.0-1855-g21b3bd67-1jammy) ... 2024-03-23T14:55:14.319 INFO:teuthology.orchestra.run.smithi173.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-03-23T14:55:14.420 INFO:teuthology.orchestra.run.smithi173.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-03-23T14:55:14.496 INFO:teuthology.orchestra.run.smithi164.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2024-03-23T14:55:14.501 INFO:teuthology.orchestra.run.smithi164.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2024-03-23T14:55:14.501 INFO:teuthology.orchestra.run.smithi164.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.3 [263 kB] 2024-03-23T14:55:14.508 INFO:teuthology.orchestra.run.smithi164.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/main amd64 libndp0 amd64 1.8-0ubuntu3 [12.3 kB] 2024-03-23T14:55:14.508 INFO:teuthology.orchestra.run.smithi164.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2024-03-23T14:55:14.519 INFO:teuthology.orchestra.run.smithi164.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.11-0ubuntu0.22.04.1 [144 kB] 2024-03-23T14:55:14.523 INFO:teuthology.orchestra.run.smithi164.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2024-03-23T14:55:14.541 INFO:teuthology.orchestra.run.smithi164.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2024-03-23T14:55:14.572 INFO:teuthology.orchestra.run.smithi173.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-03-23T14:55:14.705 INFO:teuthology.orchestra.run.smithi164.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.15 [2,376 kB] 2024-03-23T14:55:15.307 INFO:teuthology.orchestra.run.smithi173.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-03-23T14:55:15.425 INFO:teuthology.orchestra.run.smithi164.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2024-03-23T14:55:15.426 INFO:teuthology.orchestra.run.smithi164.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.3 [1,094 kB] 2024-03-23T14:55:15.452 INFO:teuthology.orchestra.run.smithi164.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2024-03-23T14:55:15.530 INFO:teuthology.orchestra.run.smithi164.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2024-03-23T14:55:15.538 INFO:teuthology.orchestra.run.smithi164.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2024-03-23T14:55:15.540 INFO:teuthology.orchestra.run.smithi164.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2024-03-23T14:55:15.541 INFO:teuthology.orchestra.run.smithi164.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2024-03-23T14:55:15.542 INFO:teuthology.orchestra.run.smithi164.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.11-0ubuntu0.22.04.1 [1,222 kB] 2024-03-23T14:55:15.567 INFO:teuthology.orchestra.run.smithi164.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2024-03-23T14:55:15.571 INFO:teuthology.orchestra.run.smithi164.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.11-0ubuntu0.22.04.1 [16.2 MB] 2024-03-23T14:55:15.942 INFO:teuthology.orchestra.run.smithi173.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-03-23T14:55:15.945 DEBUG:teuthology.orchestra.run.smithi173:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2024-03-23T14:55:15.997 INFO:teuthology.orchestra.run.smithi173.stdout:Reading package lists... 2024-03-23T14:55:16.193 INFO:teuthology.orchestra.run.smithi173.stdout:Building dependency tree... 2024-03-23T14:55:16.193 INFO:teuthology.orchestra.run.smithi173.stdout:Reading state information... 2024-03-23T14:55:16.275 INFO:teuthology.orchestra.run.smithi164.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2024-03-23T14:55:16.361 INFO:teuthology.orchestra.run.smithi173.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2024-03-23T14:55:16.361 INFO:teuthology.orchestra.run.smithi173.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2024-03-23T14:55:16.361 INFO:teuthology.orchestra.run.smithi173.stdout:util-linux is already the newest version (2.37.2-4ubuntu3). 2024-03-23T14:55:16.361 INFO:teuthology.orchestra.run.smithi173.stdout:uuid-dev is already the newest version (2.37.2-4ubuntu3). 2024-03-23T14:55:16.361 INFO:teuthology.orchestra.run.smithi173.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2024-03-23T14:55:16.362 INFO:teuthology.orchestra.run.smithi173.stdout:The following packages were automatically installed and are no longer required: 2024-03-23T14:55:16.362 INFO:teuthology.orchestra.run.smithi173.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-03-23T14:55:16.362 INFO:teuthology.orchestra.run.smithi173.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-03-23T14:55:16.362 INFO:teuthology.orchestra.run.smithi173.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-03-23T14:55:16.362 INFO:teuthology.orchestra.run.smithi173.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-03-23T14:55:16.363 INFO:teuthology.orchestra.run.smithi173.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-03-23T14:55:16.363 INFO:teuthology.orchestra.run.smithi173.stdout:Use 'sudo apt autoremove' to remove them. 2024-03-23T14:55:16.363 INFO:teuthology.orchestra.run.smithi173.stdout:The following additional packages will be installed: 2024-03-23T14:55:16.363 INFO:teuthology.orchestra.run.smithi173.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2024-03-23T14:55:16.363 INFO:teuthology.orchestra.run.smithi173.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2024-03-23T14:55:16.363 INFO:teuthology.orchestra.run.smithi173.stdout: libqmi-proxy libssl3 libteamdctl0 modemmanager network-manager-pptp 2024-03-23T14:55:16.364 INFO:teuthology.orchestra.run.smithi173.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2024-03-23T14:55:16.364 INFO:teuthology.orchestra.run.smithi173.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2024-03-23T14:55:16.364 INFO:teuthology.orchestra.run.smithi164.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2024-03-23T14:55:16.364 INFO:teuthology.orchestra.run.smithi173.stdout:Suggested packages: 2024-03-23T14:55:16.364 INFO:teuthology.orchestra.run.smithi173.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2024-03-23T14:55:16.364 INFO:teuthology.orchestra.run.smithi173.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial wpagui 2024-03-23T14:55:16.365 INFO:teuthology.orchestra.run.smithi173.stdout: libengine-pkcs11-openssl 2024-03-23T14:55:16.365 INFO:teuthology.orchestra.run.smithi164.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2024-03-23T14:55:16.365 INFO:teuthology.orchestra.run.smithi164.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2024-03-23T14:55:16.401 INFO:teuthology.orchestra.run.smithi173.stdout:The following NEW packages will be installed: 2024-03-23T14:55:16.401 INFO:teuthology.orchestra.run.smithi173.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2024-03-23T14:55:16.402 INFO:teuthology.orchestra.run.smithi173.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2024-03-23T14:55:16.402 INFO:teuthology.orchestra.run.smithi173.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2024-03-23T14:55:16.402 INFO:teuthology.orchestra.run.smithi173.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2024-03-23T14:55:16.402 INFO:teuthology.orchestra.run.smithi173.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2024-03-23T14:55:16.403 INFO:teuthology.orchestra.run.smithi173.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2024-03-23T14:55:16.403 INFO:teuthology.orchestra.run.smithi173.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2024-03-23T14:55:16.404 INFO:teuthology.orchestra.run.smithi173.stdout:The following packages will be upgraded: 2024-03-23T14:55:16.404 INFO:teuthology.orchestra.run.smithi173.stdout: libssl3 2024-03-23T14:55:16.454 INFO:teuthology.orchestra.run.smithi164.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2024-03-23T14:55:16.460 INFO:teuthology.orchestra.run.smithi164.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2 [1,481 kB] 2024-03-23T14:55:16.462 INFO:teuthology.orchestra.run.smithi173.stdout:1 upgraded, 39 newly installed, 0 to remove and 284 not upgraded. 2024-03-23T14:55:16.462 INFO:teuthology.orchestra.run.smithi173.stdout:Need to get 54.6 MB of archives. 2024-03-23T14:55:16.462 INFO:teuthology.orchestra.run.smithi173.stdout:After this operation, 202 MB of additional disk space will be used. 2024-03-23T14:55:16.462 INFO:teuthology.orchestra.run.smithi173.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2024-03-23T14:55:16.578 INFO:teuthology.orchestra.run.smithi173.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.15 [1,905 kB] 2024-03-23T14:55:16.646 INFO:teuthology.orchestra.run.smithi173.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2024-03-23T14:55:16.660 INFO:teuthology.orchestra.run.smithi173.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2024-03-23T14:55:16.662 INFO:teuthology.orchestra.run.smithi173.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2024-03-23T14:55:16.664 INFO:teuthology.orchestra.run.smithi173.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2024-03-23T14:55:16.666 INFO:teuthology.orchestra.run.smithi173.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2024-03-23T14:55:16.667 INFO:teuthology.orchestra.run.smithi173.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libbluetooth3 amd64 5.64-0ubuntu1.1 [87.0 kB] 2024-03-23T14:55:16.668 INFO:teuthology.orchestra.run.smithi173.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2024-03-23T14:55:16.669 INFO:teuthology.orchestra.run.smithi173.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2024-03-23T14:55:16.670 INFO:teuthology.orchestra.run.smithi173.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2024-03-23T14:55:16.671 INFO:teuthology.orchestra.run.smithi173.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2024-03-23T14:55:16.690 INFO:teuthology.orchestra.run.smithi173.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2024-03-23T14:55:16.917 INFO:teuthology.orchestra.run.smithi164.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2024-03-23T14:55:16.919 INFO:teuthology.orchestra.run.smithi164.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2024-03-23T14:55:17.126 INFO:teuthology.orchestra.run.smithi173.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2024-03-23T14:55:17.129 INFO:teuthology.orchestra.run.smithi173.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2024-03-23T14:55:17.130 INFO:teuthology.orchestra.run.smithi173.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.3 [263 kB] 2024-03-23T14:55:17.134 INFO:teuthology.orchestra.run.smithi173.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/main amd64 libndp0 amd64 1.8-0ubuntu3 [12.3 kB] 2024-03-23T14:55:17.134 INFO:teuthology.orchestra.run.smithi173.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2024-03-23T14:55:17.142 INFO:teuthology.orchestra.run.smithi173.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.11-0ubuntu0.22.04.1 [144 kB] 2024-03-23T14:55:17.144 INFO:teuthology.orchestra.run.smithi173.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2024-03-23T14:55:17.156 INFO:teuthology.orchestra.run.smithi173.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2024-03-23T14:55:17.156 INFO:teuthology.orchestra.run.smithi173.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.15 [2,376 kB] 2024-03-23T14:55:17.189 INFO:teuthology.orchestra.run.smithi173.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2024-03-23T14:55:17.189 INFO:teuthology.orchestra.run.smithi173.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.3 [1,094 kB] 2024-03-23T14:55:17.200 INFO:teuthology.orchestra.run.smithi173.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2024-03-23T14:55:17.220 INFO:teuthology.orchestra.run.smithi173.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2024-03-23T14:55:17.225 INFO:teuthology.orchestra.run.smithi173.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2024-03-23T14:55:17.226 INFO:teuthology.orchestra.run.smithi173.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2024-03-23T14:55:17.226 INFO:teuthology.orchestra.run.smithi173.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2024-03-23T14:55:17.227 INFO:teuthology.orchestra.run.smithi173.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.11-0ubuntu0.22.04.1 [1,222 kB] 2024-03-23T14:55:17.241 INFO:teuthology.orchestra.run.smithi173.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2024-03-23T14:55:17.243 INFO:teuthology.orchestra.run.smithi173.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.11-0ubuntu0.22.04.1 [16.2 MB] 2024-03-23T14:55:17.329 INFO:teuthology.orchestra.run.smithi164.stdout:Preconfiguring packages ... 2024-03-23T14:55:17.497 INFO:teuthology.orchestra.run.smithi164.stdout:Fetched 54.6 MB in 5s (11.0 MB/s) 2024-03-23T14:55:17.498 INFO:teuthology.orchestra.run.smithi173.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2024-03-23T14:55:17.498 INFO:teuthology.orchestra.run.smithi173.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2024-03-23T14:55:17.499 INFO:teuthology.orchestra.run.smithi173.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2024-03-23T14:55:17.499 INFO:teuthology.orchestra.run.smithi173.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2024-03-23T14:55:17.499 INFO:teuthology.orchestra.run.smithi173.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2024-03-23T14:55:17.499 INFO:teuthology.orchestra.run.smithi173.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2 [1,481 kB] 2024-03-23T14:55:17.510 INFO:teuthology.orchestra.run.smithi173.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2024-03-23T14:55:17.510 INFO:teuthology.orchestra.run.smithi173.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2024-03-23T14:55:17.635 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package flex. 2024-03-23T14:55:17.735 INFO:teuthology.orchestra.run.smithi164.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173657 files and directories currently installed.) 2024-03-23T14:55:17.741 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2024-03-23T14:55:17.783 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking flex (2.6.4-8build2) ... 2024-03-23T14:55:17.914 INFO:teuthology.orchestra.run.smithi173.stdout:Preconfiguring packages ... 2024-03-23T14:55:18.073 INFO:teuthology.orchestra.run.smithi173.stdout:Fetched 54.6 MB in 1s (49.7 MB/s) 2024-03-23T14:55:18.214 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package flex. 2024-03-23T14:55:18.309 INFO:teuthology.orchestra.run.smithi173.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173657 files and directories currently installed.) 2024-03-23T14:55:18.315 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2024-03-23T14:55:18.358 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.15_amd64.deb ... 2024-03-23T14:55:18.371 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking flex (2.6.4-8build2) ... 2024-03-23T14:55:18.530 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.15) over (3.0.2-0ubuntu1.7) ... 2024-03-23T14:55:18.905 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.15_amd64.deb ... 2024-03-23T14:55:18.960 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.15) ... 2024-03-23T14:55:19.061 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.15) over (3.0.2-0ubuntu1.7) ... 2024-03-23T14:55:19.475 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package bison. 2024-03-23T14:55:19.489 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.15) ... 2024-03-23T14:55:19.576 INFO:teuthology.orchestra.run.smithi164.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173730 files and directories currently installed.) 2024-03-23T14:55:19.582 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2024-03-23T14:55:19.626 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2024-03-23T14:55:19.971 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package bison. 2024-03-23T14:55:19.977 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-03-23T14:55:20.006 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2024-03-23T14:55:20.035 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2024-03-23T14:55:20.070 INFO:teuthology.orchestra.run.smithi173.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173730 files and directories currently installed.) 2024-03-23T14:55:20.076 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2024-03-23T14:55:20.124 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2024-03-23T14:55:20.380 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package dump. 2024-03-23T14:55:20.408 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2024-03-23T14:55:20.446 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking dump (0.4b47-3) ... 2024-03-23T14:55:20.516 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-03-23T14:55:20.544 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2024-03-23T14:55:20.583 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2024-03-23T14:55:20.741 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package indent. 2024-03-23T14:55:20.758 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2024-03-23T14:55:20.800 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2024-03-23T14:55:20.936 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package dump. 2024-03-23T14:55:20.964 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2024-03-23T14:55:21.002 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking dump (0.4b47-3) ... 2024-03-23T14:55:21.035 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-03-23T14:55:21.051 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2024-03-23T14:55:21.085 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2024-03-23T14:55:21.281 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package indent. 2024-03-23T14:55:21.297 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2024-03-23T14:55:21.331 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2024-03-23T14:55:21.380 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-03-23T14:55:21.408 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1.1_amd64.deb ... 2024-03-23T14:55:21.447 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-03-23T14:55:21.592 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-03-23T14:55:21.608 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2024-03-23T14:55:21.651 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2024-03-23T14:55:21.708 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-03-23T14:55:21.735 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2024-03-23T14:55:21.783 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2024-03-23T14:55:21.980 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-03-23T14:55:22.007 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1.1_amd64.deb ... 2024-03-23T14:55:22.046 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-03-23T14:55:22.119 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package libfl2:amd64. 2024-03-23T14:55:22.147 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2024-03-23T14:55:22.186 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2024-03-23T14:55:22.299 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-03-23T14:55:22.315 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2024-03-23T14:55:22.349 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2024-03-23T14:55:22.413 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package libfl-dev:amd64. 2024-03-23T14:55:22.437 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2024-03-23T14:55:22.480 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2024-03-23T14:55:22.694 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package libfl2:amd64. 2024-03-23T14:55:22.715 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package libjson-perl. 2024-03-23T14:55:22.722 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2024-03-23T14:55:22.731 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2024-03-23T14:55:22.761 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2024-03-23T14:55:22.773 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking libjson-perl (4.04000-1) ... 2024-03-23T14:55:23.102 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package libllvm14:amd64. 2024-03-23T14:55:23.129 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2024-03-23T14:55:23.157 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package libfl-dev:amd64. 2024-03-23T14:55:23.168 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-03-23T14:55:23.185 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2024-03-23T14:55:23.224 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2024-03-23T14:55:23.443 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package libjson-perl. 2024-03-23T14:55:23.471 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2024-03-23T14:55:23.510 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking libjson-perl (4.04000-1) ... 2024-03-23T14:55:23.830 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package libllvm14:amd64. 2024-03-23T14:55:23.857 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2024-03-23T14:55:23.898 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-03-23T14:55:24.846 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-03-23T14:55:24.874 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-03-23T14:55:24.912 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-03-23T14:55:25.181 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package libmbim-proxy. 2024-03-23T14:55:25.197 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-03-23T14:55:25.240 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-03-23T14:55:25.541 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-03-23T14:55:25.569 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-03-23T14:55:25.576 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-03-23T14:55:25.599 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-03-23T14:55:25.603 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-03-23T14:55:25.642 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-03-23T14:55:25.835 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package libmbim-proxy. 2024-03-23T14:55:25.851 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-03-23T14:55:25.894 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-03-23T14:55:25.953 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package libndp0:amd64. 2024-03-23T14:55:25.981 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3_amd64.deb ... 2024-03-23T14:55:26.011 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3) ... 2024-03-23T14:55:26.221 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-03-23T14:55:26.249 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-03-23T14:55:26.288 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-03-23T14:55:26.339 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package libnm0:amd64. 2024-03-23T14:55:26.367 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2024-03-23T14:55:26.405 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-03-23T14:55:26.650 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package libndp0:amd64. 2024-03-23T14:55:26.677 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3_amd64.deb ... 2024-03-23T14:55:26.716 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3) ... 2024-03-23T14:55:26.750 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package libpq5:amd64. 2024-03-23T14:55:26.778 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../16-libpq5_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-03-23T14:55:26.817 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-03-23T14:55:26.985 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package libnm0:amd64. 2024-03-23T14:55:27.013 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2024-03-23T14:55:27.044 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-03-23T14:55:27.136 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-03-23T14:55:27.164 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-03-23T14:55:27.202 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-03-23T14:55:27.414 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package libpq5:amd64. 2024-03-23T14:55:27.441 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../16-libpq5_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-03-23T14:55:27.480 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-03-23T14:55:27.480 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package libqmi-proxy. 2024-03-23T14:55:27.508 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-03-23T14:55:27.546 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-03-23T14:55:27.773 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-03-23T14:55:27.790 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.15_amd64.deb ... 2024-03-23T14:55:27.808 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-03-23T14:55:27.832 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-03-23T14:55:27.836 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-03-23T14:55:27.875 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-03-23T14:55:28.237 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package libqmi-proxy. 2024-03-23T14:55:28.265 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-03-23T14:55:28.303 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-03-23T14:55:28.344 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-03-23T14:55:28.372 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2024-03-23T14:55:28.410 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2024-03-23T14:55:28.564 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-03-23T14:55:28.592 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.15_amd64.deb ... 2024-03-23T14:55:28.631 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-03-23T14:55:28.755 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package modemmanager. 2024-03-23T14:55:28.782 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-03-23T14:55:28.821 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-03-23T14:55:29.151 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-03-23T14:55:29.179 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2024-03-23T14:55:29.218 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2024-03-23T14:55:29.460 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package network-manager. 2024-03-23T14:55:29.488 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2024-03-23T14:55:29.535 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2024-03-23T14:55:29.571 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package modemmanager. 2024-03-23T14:55:29.598 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-03-23T14:55:29.637 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-03-23T14:55:30.176 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package ppp. 2024-03-23T14:55:30.203 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2024-03-23T14:55:30.252 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package network-manager. 2024-03-23T14:55:30.280 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2024-03-23T14:55:30.327 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2024-03-23T14:55:30.472 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2024-03-23T14:55:30.780 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package pptp-linux. 2024-03-23T14:55:30.808 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2024-03-23T14:55:30.846 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2024-03-23T14:55:30.952 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package ppp. 2024-03-23T14:55:30.968 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2024-03-23T14:55:31.099 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package network-manager-pptp. 2024-03-23T14:55:31.114 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2024-03-23T14:55:31.157 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2024-03-23T14:55:31.254 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2024-03-23T14:55:31.460 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package postgresql-client-common. 2024-03-23T14:55:31.488 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2024-03-23T14:55:31.526 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking postgresql-client-common (238) ... 2024-03-23T14:55:31.574 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package pptp-linux. 2024-03-23T14:55:31.601 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2024-03-23T14:55:31.640 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2024-03-23T14:55:31.805 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package postgresql-client-14. 2024-03-23T14:55:31.833 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../27-postgresql-client-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-03-23T14:55:31.872 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-03-23T14:55:31.884 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package network-manager-pptp. 2024-03-23T14:55:31.902 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2024-03-23T14:55:31.942 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2024-03-23T14:55:32.212 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package postgresql-client-common. 2024-03-23T14:55:32.240 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2024-03-23T14:55:32.279 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking postgresql-client-common (238) ... 2024-03-23T14:55:32.418 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package postgresql-common. 2024-03-23T14:55:32.447 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2024-03-23T14:55:32.502 INFO:teuthology.orchestra.run.smithi164.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2024-03-23T14:55:32.533 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking postgresql-common (238) ... 2024-03-23T14:55:32.608 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package postgresql-client-14. 2024-03-23T14:55:32.636 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../27-postgresql-client-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-03-23T14:55:32.675 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-03-23T14:55:33.000 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package postgresql-14. 2024-03-23T14:55:33.028 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../29-postgresql-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-03-23T14:55:33.078 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-03-23T14:55:33.188 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package postgresql-common. 2024-03-23T14:55:33.216 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2024-03-23T14:55:33.300 INFO:teuthology.orchestra.run.smithi173.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2024-03-23T14:55:33.330 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking postgresql-common (238) ... 2024-03-23T14:55:33.714 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package postgresql-14. 2024-03-23T14:55:33.743 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../29-postgresql-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-03-23T14:55:33.785 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-03-23T14:55:34.636 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package postgresql. 2024-03-23T14:55:34.653 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2024-03-23T14:55:34.694 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking postgresql (14+238) ... 2024-03-23T14:55:34.938 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package postgresql-client. 2024-03-23T14:55:34.954 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2024-03-23T14:55:34.996 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking postgresql-client (14+238) ... 2024-03-23T14:55:35.177 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package postgresql. 2024-03-23T14:55:35.194 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2024-03-23T14:55:35.207 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package postgresql-contrib. 2024-03-23T14:55:35.235 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2024-03-23T14:55:35.235 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking postgresql (14+238) ... 2024-03-23T14:55:35.265 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking postgresql-contrib (14+238) ... 2024-03-23T14:55:35.479 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package postgresql-client. 2024-03-23T14:55:35.495 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2024-03-23T14:55:35.537 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking postgresql-client (14+238) ... 2024-03-23T14:55:35.551 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package tcl8.6. 2024-03-23T14:55:35.579 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2024-03-23T14:55:35.617 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2024-03-23T14:55:35.731 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package postgresql-contrib. 2024-03-23T14:55:35.748 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2024-03-23T14:55:35.781 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking postgresql-contrib (14+238) ... 2024-03-23T14:55:35.903 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package tcl. 2024-03-23T14:55:35.932 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2024-03-23T14:55:35.974 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking tcl (8.6.11+1build2) ... 2024-03-23T14:55:36.109 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package tcl8.6. 2024-03-23T14:55:36.137 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2024-03-23T14:55:36.176 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2024-03-23T14:55:36.323 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package wpasupplicant. 2024-03-23T14:55:36.351 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2_amd64.deb ... 2024-03-23T14:55:36.389 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2) ... 2024-03-23T14:55:36.462 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package tcl. 2024-03-23T14:55:36.490 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2024-03-23T14:55:36.533 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking tcl (8.6.11+1build2) ... 2024-03-23T14:55:36.718 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-03-23T14:55:36.735 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2024-03-23T14:55:36.778 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2024-03-23T14:55:36.848 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package wpasupplicant. 2024-03-23T14:55:36.876 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2_amd64.deb ... 2024-03-23T14:55:36.908 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2) ... 2024-03-23T14:55:37.171 INFO:teuthology.orchestra.run.smithi164.stdout:Selecting previously unselected package usb-modeswitch. 2024-03-23T14:55:37.199 INFO:teuthology.orchestra.run.smithi164.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2024-03-23T14:55:37.246 INFO:teuthology.orchestra.run.smithi164.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2024-03-23T14:55:37.271 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-03-23T14:55:37.287 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2024-03-23T14:55:37.329 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2024-03-23T14:55:37.542 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up postgresql-client-common (238) ... 2024-03-23T14:55:37.657 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package usb-modeswitch. 2024-03-23T14:55:37.686 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2024-03-23T14:55:37.732 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2024-03-23T14:55:37.752 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up flex (2.6.4-8build2) ... 2024-03-23T14:55:37.878 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2024-03-23T14:55:37.996 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-03-23T14:55:38.012 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up postgresql-client-common (238) ... 2024-03-23T14:55:38.105 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2024-03-23T14:55:38.188 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up flex (2.6.4-8build2) ... 2024-03-23T14:55:38.206 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2024-03-23T14:55:38.314 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2024-03-23T14:55:38.331 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-03-23T14:55:38.609 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-03-23T14:55:38.735 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2024-03-23T14:55:38.868 INFO:teuthology.orchestra.run.smithi164.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2024-03-23T14:55:38.869 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2024-03-23T14:55:38.954 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2) ... 2024-03-23T14:55:38.986 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-03-23T14:55:39.335 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-03-23T14:55:39.335 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-03-23T14:55:39.460 INFO:teuthology.orchestra.run.smithi173.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2024-03-23T14:55:39.550 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2) ... 2024-03-23T14:55:39.865 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2024-03-23T14:55:39.934 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-03-23T14:55:39.934 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-03-23T14:55:39.966 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-03-23T14:55:40.067 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2024-03-23T14:55:40.185 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2024-03-23T14:55:40.310 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2024-03-23T14:55:40.436 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-03-23T14:55:40.445 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2024-03-23T14:55:40.554 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-03-23T14:55:40.562 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2024-03-23T14:55:40.663 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2024-03-23T14:55:40.789 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2024-03-23T14:55:40.915 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2024-03-23T14:55:41.041 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-03-23T14:55:41.168 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2024-03-23T14:55:41.226 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-03-23T14:55:41.360 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up dump (0.4b47-3) ... 2024-03-23T14:55:41.454 INFO:teuthology.orchestra.run.smithi164.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-03-23T14:55:41.520 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-03-23T14:55:41.629 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-03-23T14:55:41.738 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up tcl (8.6.11+1build2) ... 2024-03-23T14:55:41.840 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up libjson-perl (4.04000-1) ... 2024-03-23T14:55:41.914 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-03-23T14:55:41.965 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2024-03-23T14:55:42.040 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up dump (0.4b47-3) ... 2024-03-23T14:55:42.055 INFO:teuthology.orchestra.run.smithi164.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2024-03-23T14:55:42.125 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2024-03-23T14:55:42.134 INFO:teuthology.orchestra.run.smithi173.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-03-23T14:55:42.217 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-03-23T14:55:42.351 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-03-23T14:55:42.477 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up tcl (8.6.11+1build2) ... 2024-03-23T14:55:42.586 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up libjson-perl (4.04000-1) ... 2024-03-23T14:55:42.643 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2024-03-23T14:55:42.687 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2024-03-23T14:55:42.761 INFO:teuthology.orchestra.run.smithi173.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2024-03-23T14:55:42.820 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3) ... 2024-03-23T14:55:42.822 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2024-03-23T14:55:42.937 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-03-23T14:55:42.948 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2024-03-23T14:55:43.038 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2024-03-23T14:55:43.124 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3) ... 2024-03-23T14:55:43.189 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up postgresql-client (14+238) ... 2024-03-23T14:55:43.250 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-03-23T14:55:43.315 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2024-03-23T14:55:43.385 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2024-03-23T14:55:43.578 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up postgresql-client (14+238) ... 2024-03-23T14:55:43.687 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2024-03-23T14:55:43.802 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-23T14:55:43.802 INFO:teuthology.orchestra.run.smithi164.stdout:The following network interfaces were found in /etc/network/interfaces 2024-03-23T14:55:43.802 INFO:teuthology.orchestra.run.smithi164.stdout:which means they are currently configured by ifupdown: 2024-03-23T14:55:43.802 INFO:teuthology.orchestra.run.smithi164.stdout:- enp3s0f1 2024-03-23T14:55:43.803 INFO:teuthology.orchestra.run.smithi164.stdout:If you want to manage those interfaces with NetworkManager instead 2024-03-23T14:55:43.803 INFO:teuthology.orchestra.run.smithi164.stdout:remove their configuration from /etc/network/interfaces. 2024-03-23T14:55:43.803 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-23T14:55:43.922 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-03-23T14:55:44.252 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-03-23T14:55:44.562 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-03-23T14:55:44.562 INFO:teuthology.orchestra.run.smithi173.stdout:The following network interfaces were found in /etc/network/interfaces 2024-03-23T14:55:44.566 INFO:teuthology.orchestra.run.smithi173.stdout:which means they are currently configured by ifupdown: 2024-03-23T14:55:44.566 INFO:teuthology.orchestra.run.smithi173.stdout:- enp3s0f1 2024-03-23T14:55:44.566 INFO:teuthology.orchestra.run.smithi173.stdout:If you want to manage those interfaces with NetworkManager instead 2024-03-23T14:55:44.566 INFO:teuthology.orchestra.run.smithi173.stdout:remove their configuration from /etc/network/interfaces. 2024-03-23T14:55:44.566 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-03-23T14:55:44.573 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-03-23T14:55:44.677 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-03-23T14:55:45.009 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-03-23T14:55:45.084 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-03-23T14:55:45.211 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2024-03-23T14:55:45.337 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2024-03-23T14:55:45.344 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-03-23T14:55:45.438 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up postgresql-common (238) ... 2024-03-23T14:55:45.847 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-03-23T14:55:45.999 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2024-03-23T14:55:46.125 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2024-03-23T14:55:46.259 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up postgresql-common (238) ... 2024-03-23T14:55:46.297 INFO:teuthology.orchestra.run.smithi164.stdout:Adding user postgres to group ssl-cert 2024-03-23T14:55:46.537 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-23T14:55:46.537 INFO:teuthology.orchestra.run.smithi164.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2024-03-23T14:55:46.724 INFO:teuthology.orchestra.run.smithi164.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2024-03-23T14:55:46.725 INFO:teuthology.orchestra.run.smithi164.stdout:Removing obsolete dictionary files: 2024-03-23T14:55:47.146 INFO:teuthology.orchestra.run.smithi173.stdout:Adding user postgres to group ssl-cert 2024-03-23T14:55:47.267 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2024-03-23T14:55:47.440 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-03-23T14:55:47.440 INFO:teuthology.orchestra.run.smithi173.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2024-03-23T14:55:47.643 INFO:teuthology.orchestra.run.smithi173.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2024-03-23T14:55:47.643 INFO:teuthology.orchestra.run.smithi173.stdout:Removing obsolete dictionary files: 2024-03-23T14:55:47.655 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-03-23T14:55:47.981 INFO:teuthology.orchestra.run.smithi164.stdout:Creating new PostgreSQL cluster 14/main ... 2024-03-23T14:55:48.004 INFO:teuthology.orchestra.run.smithi164.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2024-03-23T14:55:48.053 INFO:teuthology.orchestra.run.smithi164.stdout:The files belonging to this database system will be owned by user "postgres". 2024-03-23T14:55:48.053 INFO:teuthology.orchestra.run.smithi164.stdout:This user must also own the server process. 2024-03-23T14:55:48.053 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-23T14:55:48.054 INFO:teuthology.orchestra.run.smithi164.stdout:The database cluster will be initialized with locale "en_US". 2024-03-23T14:55:48.054 INFO:teuthology.orchestra.run.smithi164.stdout:The default database encoding has accordingly been set to "LATIN1". 2024-03-23T14:55:48.054 INFO:teuthology.orchestra.run.smithi164.stdout:The default text search configuration will be set to "english". 2024-03-23T14:55:48.054 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-23T14:55:48.054 INFO:teuthology.orchestra.run.smithi164.stdout:Data page checksums are disabled. 2024-03-23T14:55:48.054 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-23T14:55:48.054 INFO:teuthology.orchestra.run.smithi164.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2024-03-23T14:55:48.054 INFO:teuthology.orchestra.run.smithi164.stdout:creating subdirectories ... ok 2024-03-23T14:55:48.054 INFO:teuthology.orchestra.run.smithi164.stdout:selecting dynamic shared memory implementation ... posix 2024-03-23T14:55:48.069 INFO:teuthology.orchestra.run.smithi164.stdout:selecting default max_connections ... 100 2024-03-23T14:55:48.090 INFO:teuthology.orchestra.run.smithi164.stdout:selecting default shared_buffers ... 128MB 2024-03-23T14:55:48.100 INFO:teuthology.orchestra.run.smithi164.stdout:selecting default time zone ... Etc/UTC 2024-03-23T14:55:48.101 INFO:teuthology.orchestra.run.smithi164.stdout:creating configuration files ... ok 2024-03-23T14:55:48.183 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2024-03-23T14:55:48.216 INFO:teuthology.orchestra.run.smithi164.stdout:running bootstrap script ... ok 2024-03-23T14:55:48.586 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-03-23T14:55:48.867 INFO:teuthology.orchestra.run.smithi164.stdout:performing post-bootstrap initialization ... ok 2024-03-23T14:55:48.905 INFO:teuthology.orchestra.run.smithi173.stdout:Creating new PostgreSQL cluster 14/main ... 2024-03-23T14:55:48.930 INFO:teuthology.orchestra.run.smithi173.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2024-03-23T14:55:48.973 INFO:teuthology.orchestra.run.smithi173.stdout:The files belonging to this database system will be owned by user "postgres". 2024-03-23T14:55:48.973 INFO:teuthology.orchestra.run.smithi173.stdout:This user must also own the server process. 2024-03-23T14:55:48.973 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-03-23T14:55:48.973 INFO:teuthology.orchestra.run.smithi173.stdout:The database cluster will be initialized with locale "en_US". 2024-03-23T14:55:48.973 INFO:teuthology.orchestra.run.smithi173.stdout:The default database encoding has accordingly been set to "LATIN1". 2024-03-23T14:55:48.973 INFO:teuthology.orchestra.run.smithi173.stdout:The default text search configuration will be set to "english". 2024-03-23T14:55:48.973 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-03-23T14:55:48.973 INFO:teuthology.orchestra.run.smithi173.stdout:Data page checksums are disabled. 2024-03-23T14:55:48.973 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-03-23T14:55:48.973 INFO:teuthology.orchestra.run.smithi173.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2024-03-23T14:55:48.974 INFO:teuthology.orchestra.run.smithi173.stdout:creating subdirectories ... ok 2024-03-23T14:55:48.974 INFO:teuthology.orchestra.run.smithi173.stdout:selecting dynamic shared memory implementation ... posix 2024-03-23T14:55:48.989 INFO:teuthology.orchestra.run.smithi173.stdout:selecting default max_connections ... 100 2024-03-23T14:55:49.006 INFO:teuthology.orchestra.run.smithi173.stdout:selecting default shared_buffers ... 128MB 2024-03-23T14:55:49.016 INFO:teuthology.orchestra.run.smithi173.stdout:selecting default time zone ... Etc/UTC 2024-03-23T14:55:49.017 INFO:teuthology.orchestra.run.smithi173.stdout:creating configuration files ... ok 2024-03-23T14:55:49.128 INFO:teuthology.orchestra.run.smithi173.stdout:running bootstrap script ... ok 2024-03-23T14:55:49.718 INFO:teuthology.orchestra.run.smithi164.stdout:syncing data to disk ... ok 2024-03-23T14:55:49.769 INFO:teuthology.orchestra.run.smithi173.stdout:performing post-bootstrap initialization ... ok 2024-03-23T14:55:50.122 INFO:teuthology.orchestra.run.smithi164.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2024-03-23T14:55:50.515 INFO:teuthology.orchestra.run.smithi173.stdout:syncing data to disk ... ok 2024-03-23T14:55:50.916 INFO:teuthology.orchestra.run.smithi173.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2024-03-23T14:55:52.913 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-03-23T14:55:53.038 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up postgresql-contrib (14+238) ... 2024-03-23T14:55:53.164 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up postgresql (14+238) ... 2024-03-23T14:55:53.290 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-03-23T14:55:53.416 INFO:teuthology.orchestra.run.smithi164.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-03-23T14:55:53.676 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-03-23T14:55:53.681 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-03-23T14:55:53.681 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-03-23T14:55:53.794 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up postgresql-contrib (14+238) ... 2024-03-23T14:55:53.894 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up postgresql (14+238) ... 2024-03-23T14:55:54.004 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-03-23T14:55:54.130 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-03-23T14:55:54.261 INFO:teuthology.orchestra.run.smithi164.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-03-23T14:55:54.381 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-03-23T14:55:54.382 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-03-23T14:55:54.571 INFO:teuthology.orchestra.run.smithi164.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-03-23T14:55:54.714 INFO:teuthology.orchestra.run.smithi164.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-03-23T14:55:54.915 INFO:teuthology.orchestra.run.smithi173.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-03-23T14:55:55.226 INFO:teuthology.orchestra.run.smithi173.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-03-23T14:55:55.378 INFO:teuthology.orchestra.run.smithi173.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-03-23T14:55:56.034 INFO:teuthology.orchestra.run.smithi164.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2024-03-23T14:55:56.133 INFO:teuthology.orchestra.run.smithi164.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-03-23T14:55:56.626 INFO:teuthology.orchestra.run.smithi164.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-03-23T14:55:56.631 DEBUG:teuthology.parallel:result is None 2024-03-23T14:55:56.764 INFO:teuthology.orchestra.run.smithi173.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2024-03-23T14:55:56.864 INFO:teuthology.orchestra.run.smithi173.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-03-23T14:55:57.323 INFO:teuthology.orchestra.run.smithi173.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-03-23T14:55:57.328 DEBUG:teuthology.parallel:result is None 2024-03-23T14:55:57.328 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=21b3bd672163092b4e542f772797fcba02d3d4de 2024-03-23T14:55:57.475 DEBUG:teuthology.orchestra.run.smithi164:> dpkg-query -W -f '${Version}' ceph 2024-03-23T14:55:57.495 INFO:teuthology.orchestra.run.smithi164.stdout:19.0.0-1855-g21b3bd67-1jammy 2024-03-23T14:55:57.495 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1855-g21b3bd67-1jammy 2024-03-23T14:55:57.495 INFO:teuthology.task.install:The correct ceph version 19.0.0-1855-g21b3bd67-1jammy is installed. 2024-03-23T14:55:57.497 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=21b3bd672163092b4e542f772797fcba02d3d4de 2024-03-23T14:55:57.714 DEBUG:teuthology.orchestra.run.smithi173:> dpkg-query -W -f '${Version}' ceph 2024-03-23T14:55:57.733 INFO:teuthology.orchestra.run.smithi173.stdout:19.0.0-1855-g21b3bd67-1jammy 2024-03-23T14:55:57.733 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1855-g21b3bd67-1jammy 2024-03-23T14:55:57.733 INFO:teuthology.task.install:The correct ceph version 19.0.0-1855-g21b3bd67-1jammy is installed. 2024-03-23T14:55:57.735 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-03-23T14:55:57.735 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-23T14:55:57.735 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-23T14:55:57.750 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-03-23T14:55:57.750 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-23T14:55:57.789 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-03-23T14:55:57.789 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-23T14:55:57.789 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/usr/bin/daemon-helper 2024-03-23T14:55:57.805 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-23T14:55:57.860 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-03-23T14:55:57.860 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd of=/usr/bin/daemon-helper 2024-03-23T14:55:57.873 DEBUG:teuthology.orchestra.run.smithi173:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-23T14:55:57.927 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-03-23T14:55:57.927 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-23T14:55:57.927 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-23T14:55:57.939 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-23T14:55:57.995 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-03-23T14:55:57.995 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-23T14:55:58.008 DEBUG:teuthology.orchestra.run.smithi173:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-23T14:55:58.062 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-03-23T14:55:58.063 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-23T14:55:58.063 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/usr/bin/stdin-killer 2024-03-23T14:55:58.075 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-23T14:55:58.127 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-03-23T14:55:58.127 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd of=/usr/bin/stdin-killer 2024-03-23T14:55:58.140 DEBUG:teuthology.orchestra.run.smithi173:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-23T14:55:58.195 INFO:teuthology.run_tasks:Running task ceph... 2024-03-23T14:55:58.281 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-03-23T14:55:58.281 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod 777 /var/log/ceph 2024-03-23T14:55:58.283 DEBUG:teuthology.orchestra.run.smithi173:> sudo chmod 777 /var/log/ceph 2024-03-23T14:55:58.296 INFO:tasks.ceph:Disabling ceph logrotate... 2024-03-23T14:55:58.297 DEBUG:teuthology.orchestra.run.smithi164:> sudo rm -f -- /etc/logrotate.d/ceph 2024-03-23T14:55:58.337 DEBUG:teuthology.orchestra.run.smithi173:> sudo rm -f -- /etc/logrotate.d/ceph 2024-03-23T14:55:58.351 INFO:tasks.ceph:Creating extra log directories... 2024-03-23T14:55:58.351 DEBUG:teuthology.orchestra.run.smithi164:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-03-23T14:55:58.393 DEBUG:teuthology.orchestra.run.smithi173:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-03-23T14:55:58.412 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-03-23T14:55:58.412 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-03-23T14:55:58.412 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-03-23T14:55:58.412 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-23T14:55:58.412 DEBUG:teuthology.orchestra.run.smithi164:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-03-23T14:55:58.453 DEBUG:teuthology.orchestra.run.smithi164:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-03-23T14:55:58.527 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-03-23T14:55:58.527 DEBUG:teuthology.orchestra.run.smithi173:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-03-23T14:55:58.533 DEBUG:teuthology.orchestra.run.smithi173:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-03-23T14:55:58.615 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-03-23T14:55:58.615 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-03-23T14:55:58.615 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/pdonnell-2024-03-23_09:56:53-fs-wip-batrick-testing-20240323.003144-squid-distro-default-smithi/7618407', 'branch': 'wip-batrick-testing-20240323.003144-squid', 'description': 'fs/shell/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1-mds-1-client-coloc conf/{client mds mgr mon osd} distro/ubuntu_latest mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/cephfs-shell}', 'email': 'pdonnell@redhat.com', 'first_in_suite': False, 'job_id': '7618407', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'mgrmodules': {'sequential': [{'print': 'Enabling mgr modules'}]}, 'name': 'pdonnell-2024-03-23_09:56:53-fs-wip-batrick-testing-20240323.003144-squid-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'ubuntu', 'os_version': '22.04', 'overrides': {'admin_socket': {'branch': 'wip-batrick-testing-20240323.003144-squid'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'sha1': '21b3bd672163092b4e542f772797fcba02d3d4de'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '21b3bd672163092b4e542f772797fcba02d3d4de'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-batrick-testing-20240323.003144-squid', 'sha1': 'd39ab7bb1f1162ec7b83b4f13a6bb876d4baf21b'}}, 'owner': 'scheduled_pdonnell@teuthology', 'priority': 85, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'osd.4', 'osd.5', 'osd.6', 'osd.7']], 'seed': 4284, 'sha1': '21b3bd672163092b4e542f772797fcba02d3d4de', 'sleep_before_teardown': 0, 'subset': '405/512', 'suite': 'fs', 'suite_branch': 'wip-batrick-testing-20240323.003144-squid', 'suite_path': '/home/teuthworker/src/github.com_batrick_ceph_d39ab7bb1f1162ec7b83b4f13a6bb876d4baf21b/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/batrick/ceph.git', 'suite_sha1': 'd39ab7bb1f1162ec7b83b4f13a6bb876d4baf21b', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi164.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBG8pFKkZbSowF13bo8/baC7ODVq9vuxuXd51QOBTdiI3/0A2eM6xZaLeP6WOiDQ2yG3mqamUVAmozkvqaQGo+hw=', 'smithi173.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDYERFStNwl9aEPr/ypxY64Yl00iN9HbehUIiBGulOVP8YqeAQM6Ms8/NjAJgIZLbg0ZS+tGZV+5TDftGxwMUgA='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '21b3bd672163092b4e542f772797fcba02d3d4de'}}, {'ceph': {'create_rbd_pool': False, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'sha1': '21b3bd672163092b4e542f772797fcba02d3d4de', 'cluster': 'ceph'}}, {'sequential': ['mgrmodules']}, {'ceph-fuse': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_cephfs_shell']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': 'e691533f9cbb33d85b2187bba20d7102f098636d', 'timestamp': '2024-03-23_09:56:53', 'tube': 'smithi', 'user': 'pdonnell', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-03-23T14:55:58.616 DEBUG:teuthology.orchestra.run.smithi164:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-03-23T14:55:58.618 DEBUG:teuthology.orchestra.run.smithi173:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-03-23T14:55:58.656 DEBUG:teuthology.orchestra.run.smithi164:> sudo install -d -m0777 -- /var/run/ceph 2024-03-23T14:55:58.666 DEBUG:teuthology.orchestra.run.smithi173:> sudo install -d -m0777 -- /var/run/ceph 2024-03-23T14:55:58.712 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-23T14:55:58.712 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/scratch_devs of=/dev/stdout 2024-03-23T14:55:58.724 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-23T14:55:58.725 DEBUG:teuthology.orchestra.run.smithi164:> stat /dev/vg_nvme/lv_1 2024-03-23T14:55:58.772 INFO:teuthology.orchestra.run.smithi164.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-23T14:55:58.773 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-23T14:55:58.773 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 5h/5d Inode: 871 Links: 1 2024-03-23T14:55:58.773 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-23T14:55:58.773 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-03-23 14:49:51.489992161 +0000 2024-03-23T14:55:58.773 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-03-23 14:49:51.217994589 +0000 2024-03-23T14:55:58.773 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-03-23 14:49:51.217994589 +0000 2024-03-23T14:55:58.773 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: - 2024-03-23T14:55:58.773 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-23T14:55:58.827 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records in 2024-03-23T14:55:58.827 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records out 2024-03-23T14:55:58.827 INFO:teuthology.orchestra.run.smithi164.stderr:512 bytes copied, 0.000466013 s, 1.1 MB/s 2024-03-23T14:55:58.828 DEBUG:teuthology.orchestra.run.smithi164:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-23T14:55:58.878 DEBUG:teuthology.orchestra.run.smithi164:> stat /dev/vg_nvme/lv_2 2024-03-23T14:55:58.924 INFO:teuthology.orchestra.run.smithi164.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-23T14:55:58.924 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-23T14:55:58.925 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 5h/5d Inode: 887 Links: 1 2024-03-23T14:55:58.925 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-23T14:55:58.925 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-03-23 14:49:51.737989946 +0000 2024-03-23T14:55:58.925 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-03-23 14:49:51.733989982 +0000 2024-03-23T14:55:58.925 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-03-23 14:49:51.733989982 +0000 2024-03-23T14:55:58.925 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: - 2024-03-23T14:55:58.925 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-23T14:55:58.978 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records in 2024-03-23T14:55:58.978 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records out 2024-03-23T14:55:58.979 INFO:teuthology.orchestra.run.smithi164.stderr:512 bytes copied, 0.000310269 s, 1.7 MB/s 2024-03-23T14:55:58.980 DEBUG:teuthology.orchestra.run.smithi164:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-23T14:55:59.034 DEBUG:teuthology.orchestra.run.smithi164:> stat /dev/vg_nvme/lv_3 2024-03-23T14:55:59.080 INFO:teuthology.orchestra.run.smithi164.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-23T14:55:59.080 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-23T14:55:59.080 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 5h/5d Inode: 905 Links: 1 2024-03-23T14:55:59.080 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-23T14:55:59.080 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-03-23 14:49:52.245985408 +0000 2024-03-23T14:55:59.080 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-03-23 14:49:52.241985444 +0000 2024-03-23T14:55:59.081 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-03-23 14:49:52.241985444 +0000 2024-03-23T14:55:59.081 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: - 2024-03-23T14:55:59.081 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-23T14:55:59.134 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records in 2024-03-23T14:55:59.134 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records out 2024-03-23T14:55:59.134 INFO:teuthology.orchestra.run.smithi164.stderr:512 bytes copied, 0.000325519 s, 1.6 MB/s 2024-03-23T14:55:59.135 DEBUG:teuthology.orchestra.run.smithi164:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-23T14:55:59.182 DEBUG:teuthology.orchestra.run.smithi164:> stat /dev/vg_nvme/lv_4 2024-03-23T14:55:59.228 INFO:teuthology.orchestra.run.smithi164.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-23T14:55:59.228 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-23T14:55:59.229 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 5h/5d Inode: 925 Links: 1 2024-03-23T14:55:59.229 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-23T14:55:59.229 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-03-23 14:49:52.721981157 +0000 2024-03-23T14:55:59.229 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-03-23 14:49:52.717981191 +0000 2024-03-23T14:55:59.229 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-03-23 14:49:52.717981191 +0000 2024-03-23T14:55:59.229 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: - 2024-03-23T14:55:59.229 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-23T14:55:59.282 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records in 2024-03-23T14:55:59.283 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records out 2024-03-23T14:55:59.283 INFO:teuthology.orchestra.run.smithi164.stderr:512 bytes copied, 0.000321594 s, 1.6 MB/s 2024-03-23T14:55:59.284 DEBUG:teuthology.orchestra.run.smithi164:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-23T14:55:59.335 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-23T14:55:59.335 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-03-23T14:55:59.335 DEBUG:teuthology.orchestra.run.smithi173:> dd if=/scratch_devs of=/dev/stdout 2024-03-23T14:55:59.340 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-23T14:55:59.341 DEBUG:teuthology.orchestra.run.smithi173:> stat /dev/vg_nvme/lv_1 2024-03-23T14:55:59.388 INFO:teuthology.orchestra.run.smithi173.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-23T14:55:59.389 INFO:teuthology.orchestra.run.smithi173.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-23T14:55:59.389 INFO:teuthology.orchestra.run.smithi173.stdout:Device: 5h/5d Inode: 870 Links: 1 2024-03-23T14:55:59.389 INFO:teuthology.orchestra.run.smithi173.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-23T14:55:59.389 INFO:teuthology.orchestra.run.smithi173.stdout:Access: 2024-03-23 14:50:01.876323844 +0000 2024-03-23T14:55:59.389 INFO:teuthology.orchestra.run.smithi173.stdout:Modify: 2024-03-23 14:50:01.612326193 +0000 2024-03-23T14:55:59.389 INFO:teuthology.orchestra.run.smithi173.stdout:Change: 2024-03-23 14:50:01.612326193 +0000 2024-03-23T14:55:59.389 INFO:teuthology.orchestra.run.smithi173.stdout: Birth: - 2024-03-23T14:55:59.389 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-23T14:55:59.443 INFO:teuthology.orchestra.run.smithi173.stderr:1+0 records in 2024-03-23T14:55:59.443 INFO:teuthology.orchestra.run.smithi173.stderr:1+0 records out 2024-03-23T14:55:59.443 INFO:teuthology.orchestra.run.smithi173.stderr:512 bytes copied, 0.000446045 s, 1.1 MB/s 2024-03-23T14:55:59.444 DEBUG:teuthology.orchestra.run.smithi173:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-23T14:55:59.493 DEBUG:teuthology.orchestra.run.smithi173:> stat /dev/vg_nvme/lv_2 2024-03-23T14:55:59.540 INFO:teuthology.orchestra.run.smithi173.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-23T14:55:59.540 INFO:teuthology.orchestra.run.smithi173.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-23T14:55:59.540 INFO:teuthology.orchestra.run.smithi173.stdout:Device: 5h/5d Inode: 885 Links: 1 2024-03-23T14:55:59.540 INFO:teuthology.orchestra.run.smithi173.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-23T14:55:59.540 INFO:teuthology.orchestra.run.smithi173.stdout:Access: 2024-03-23 14:50:02.396319218 +0000 2024-03-23T14:55:59.540 INFO:teuthology.orchestra.run.smithi173.stdout:Modify: 2024-03-23 14:50:02.120321673 +0000 2024-03-23T14:55:59.541 INFO:teuthology.orchestra.run.smithi173.stdout:Change: 2024-03-23 14:50:02.120321673 +0000 2024-03-23T14:55:59.541 INFO:teuthology.orchestra.run.smithi173.stdout: Birth: - 2024-03-23T14:55:59.541 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-23T14:55:59.594 INFO:teuthology.orchestra.run.smithi173.stderr:1+0 records in 2024-03-23T14:55:59.594 INFO:teuthology.orchestra.run.smithi173.stderr:1+0 records out 2024-03-23T14:55:59.595 INFO:teuthology.orchestra.run.smithi173.stderr:512 bytes copied, 0.000349257 s, 1.5 MB/s 2024-03-23T14:55:59.596 DEBUG:teuthology.orchestra.run.smithi173:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-23T14:55:59.645 DEBUG:teuthology.orchestra.run.smithi173:> stat /dev/vg_nvme/lv_3 2024-03-23T14:55:59.692 INFO:teuthology.orchestra.run.smithi173.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-23T14:55:59.692 INFO:teuthology.orchestra.run.smithi173.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-23T14:55:59.693 INFO:teuthology.orchestra.run.smithi173.stdout:Device: 5h/5d Inode: 907 Links: 1 2024-03-23T14:55:59.693 INFO:teuthology.orchestra.run.smithi173.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-23T14:55:59.693 INFO:teuthology.orchestra.run.smithi173.stdout:Access: 2024-03-23 14:50:02.892314806 +0000 2024-03-23T14:55:59.693 INFO:teuthology.orchestra.run.smithi173.stdout:Modify: 2024-03-23 14:50:02.624317190 +0000 2024-03-23T14:55:59.693 INFO:teuthology.orchestra.run.smithi173.stdout:Change: 2024-03-23 14:50:02.624317190 +0000 2024-03-23T14:55:59.693 INFO:teuthology.orchestra.run.smithi173.stdout: Birth: - 2024-03-23T14:55:59.693 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-23T14:55:59.751 INFO:teuthology.orchestra.run.smithi173.stderr:1+0 records in 2024-03-23T14:55:59.751 INFO:teuthology.orchestra.run.smithi173.stderr:1+0 records out 2024-03-23T14:55:59.751 INFO:teuthology.orchestra.run.smithi173.stderr:512 bytes copied, 0.000428236 s, 1.2 MB/s 2024-03-23T14:55:59.752 DEBUG:teuthology.orchestra.run.smithi173:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-23T14:55:59.801 DEBUG:teuthology.orchestra.run.smithi173:> stat /dev/vg_nvme/lv_4 2024-03-23T14:55:59.852 INFO:teuthology.orchestra.run.smithi173.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-23T14:55:59.852 INFO:teuthology.orchestra.run.smithi173.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-23T14:55:59.852 INFO:teuthology.orchestra.run.smithi173.stdout:Device: 5h/5d Inode: 924 Links: 1 2024-03-23T14:55:59.852 INFO:teuthology.orchestra.run.smithi173.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-23T14:55:59.852 INFO:teuthology.orchestra.run.smithi173.stdout:Access: 2024-03-23 14:50:03.364310606 +0000 2024-03-23T14:55:59.852 INFO:teuthology.orchestra.run.smithi173.stdout:Modify: 2024-03-23 14:50:03.132312671 +0000 2024-03-23T14:55:59.852 INFO:teuthology.orchestra.run.smithi173.stdout:Change: 2024-03-23 14:50:03.132312671 +0000 2024-03-23T14:55:59.853 INFO:teuthology.orchestra.run.smithi173.stdout: Birth: - 2024-03-23T14:55:59.853 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-23T14:55:59.906 INFO:teuthology.orchestra.run.smithi173.stderr:1+0 records in 2024-03-23T14:55:59.907 INFO:teuthology.orchestra.run.smithi173.stderr:1+0 records out 2024-03-23T14:55:59.907 INFO:teuthology.orchestra.run.smithi173.stderr:512 bytes copied, 0.000410656 s, 1.2 MB/s 2024-03-23T14:55:59.908 DEBUG:teuthology.orchestra.run.smithi173:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-23T14:55:59.961 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-23T14:55:59.961 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi164.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi173.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-03-23T14:55:59.961 INFO:tasks.ceph:Generating config... 2024-03-23T14:55:59.962 INFO:tasks.ceph:[client] client mount timeout = 600 2024-03-23T14:55:59.962 INFO:tasks.ceph:[client] debug client = 20 2024-03-23T14:55:59.963 INFO:tasks.ceph:[client] debug ms = 1 2024-03-23T14:55:59.963 INFO:tasks.ceph:[client] pid file = 2024-03-23T14:55:59.963 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-03-23T14:55:59.963 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-03-23T14:55:59.963 INFO:tasks.ceph:[mds] debug mds = 20 2024-03-23T14:55:59.963 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-03-23T14:55:59.963 INFO:tasks.ceph:[mds] debug ms = 1 2024-03-23T14:55:59.963 INFO:tasks.ceph:[mds] mds debug frag = True 2024-03-23T14:55:59.963 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-03-23T14:55:59.963 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-03-23T14:55:59.963 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-03-23T14:55:59.963 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-03-23T14:55:59.963 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-03-23T14:55:59.963 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-03-23T14:55:59.964 INFO:tasks.ceph:[mgr] debug client = 20 2024-03-23T14:55:59.964 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-03-23T14:55:59.964 INFO:tasks.ceph:[mgr] debug ms = 1 2024-03-23T14:55:59.964 INFO:tasks.ceph:[mon] debug mon = 20 2024-03-23T14:55:59.964 INFO:tasks.ceph:[mon] debug ms = 1 2024-03-23T14:55:59.964 INFO:tasks.ceph:[mon] debug paxos = 20 2024-03-23T14:55:59.964 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-03-23T14:55:59.964 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-03-23T14:55:59.964 INFO:tasks.ceph:[osd] bdev async discard = True 2024-03-23T14:55:59.964 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-03-23T14:55:59.964 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-03-23T14:55:59.964 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-03-23T14:55:59.964 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-03-23T14:55:59.964 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-03-23T14:55:59.964 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-03-23T14:55:59.964 INFO:tasks.ceph:[osd] debug ms = 1 2024-03-23T14:55:59.964 INFO:tasks.ceph:[osd] debug osd = 20 2024-03-23T14:55:59.965 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-03-23T14:55:59.965 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-03-23T14:55:59.965 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-03-23T14:55:59.965 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-03-23T14:55:59.965 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-03-23T14:55:59.965 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-03-23T14:55:59.965 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-03-23T14:55:59.965 INFO:tasks.ceph:Setting up mon.a... 2024-03-23T14:55:59.965 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-03-23T14:55:59.998 INFO:teuthology.orchestra.run.smithi164.stdout:creating /etc/ceph/ceph.keyring 2024-03-23T14:56:00.015 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-03-23T14:56:00.095 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-03-23T14:56:00.160 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.164'), ('mon.b', '172.21.15.173'), ('mon.c', '[v2:172.21.15.173:3301,v1:172.21.15.173:6790]')] 2024-03-23T14:56:00.160 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.164,172.21.15.173,[v2:172.21.15.173:3301,v1:172.21.15.173:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.b': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-03-23T14:56:00.162 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-23T14:56:00.162 DEBUG:teuthology.orchestra.run.smithi164:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-03-23T14:56:00.209 DEBUG:teuthology.orchestra.run.smithi164:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.164 --add b 172.21.15.173 --addv c '[v2:172.21.15.173:3301,v1:172.21.15.173:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-03-23T14:56:00.295 INFO:teuthology.orchestra.run.smithi164.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-03-23T14:56:00.295 INFO:teuthology.orchestra.run.smithi164.stdout:monmaptool: generated fsid 45c7911b-b69d-4e68-907b-fbfeb780aa3b 2024-03-23T14:56:00.295 INFO:teuthology.orchestra.run.smithi164.stdout:setting min_mon_release = quincy 2024-03-23T14:56:00.295 INFO:teuthology.orchestra.run.smithi164.stdout:epoch 0 2024-03-23T14:56:00.295 INFO:teuthology.orchestra.run.smithi164.stdout:fsid 45c7911b-b69d-4e68-907b-fbfeb780aa3b 2024-03-23T14:56:00.295 INFO:teuthology.orchestra.run.smithi164.stdout:last_changed 2024-03-23T14:56:00.295122+0000 2024-03-23T14:56:00.295 INFO:teuthology.orchestra.run.smithi164.stdout:created 2024-03-23T14:56:00.295122+0000 2024-03-23T14:56:00.295 INFO:teuthology.orchestra.run.smithi164.stdout:min_mon_release 17 (quincy) 2024-03-23T14:56:00.295 INFO:teuthology.orchestra.run.smithi164.stdout:election_strategy: 1 2024-03-23T14:56:00.295 INFO:teuthology.orchestra.run.smithi164.stdout:0: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.a 2024-03-23T14:56:00.295 INFO:teuthology.orchestra.run.smithi164.stdout:1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.b 2024-03-23T14:56:00.295 INFO:teuthology.orchestra.run.smithi164.stdout:2: [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] mon.c 2024-03-23T14:56:00.296 INFO:teuthology.orchestra.run.smithi164.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-03-23T14:56:00.297 DEBUG:teuthology.orchestra.run.smithi164:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-03-23T14:56:00.345 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 45c7911b-b69d-4e68-907b-fbfeb780aa3b... 2024-03-23T14:56:00.346 DEBUG:teuthology.orchestra.run.smithi164:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-03-23T14:56:00.390 DEBUG:teuthology.orchestra.run.smithi173:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-03-23T14:56:00.418 INFO:teuthology.orchestra.run.smithi164.stdout:[global] 2024-03-23T14:56:00.418 INFO:teuthology.orchestra.run.smithi164.stdout: chdir = "" 2024-03-23T14:56:00.419 INFO:teuthology.orchestra.run.smithi164.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-03-23T14:56:00.419 INFO:teuthology.orchestra.run.smithi164.stdout: auth supported = cephx 2024-03-23T14:56:00.419 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-23T14:56:00.419 INFO:teuthology.orchestra.run.smithi164.stdout: filestore xattr use omap = true 2024-03-23T14:56:00.419 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-23T14:56:00.419 INFO:teuthology.orchestra.run.smithi164.stdout: mon clock drift allowed = 1.000 2024-03-23T14:56:00.419 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-23T14:56:00.419 INFO:teuthology.orchestra.run.smithi164.stdout: osd crush chooseleaf type = 0 2024-03-23T14:56:00.419 INFO:teuthology.orchestra.run.smithi164.stdout: auth debug = true 2024-03-23T14:56:00.419 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-23T14:56:00.419 INFO:teuthology.orchestra.run.smithi164.stdout: ms die on old message = true 2024-03-23T14:56:00.419 INFO:teuthology.orchestra.run.smithi164.stdout: ms die on bug = true 2024-03-23T14:56:00.419 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-23T14:56:00.419 INFO:teuthology.orchestra.run.smithi164.stdout: mon max pg per osd = 10000 # >= luminous 2024-03-23T14:56:00.419 INFO:teuthology.orchestra.run.smithi164.stdout: mon pg warn max object skew = 0 2024-03-23T14:56:00.420 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-23T14:56:00.420 INFO:teuthology.orchestra.run.smithi164.stdout: # disable pg_autoscaler by default for new pools 2024-03-23T14:56:00.420 INFO:teuthology.orchestra.run.smithi164.stdout: osd_pool_default_pg_autoscale_mode = off 2024-03-23T14:56:00.420 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-23T14:56:00.420 INFO:teuthology.orchestra.run.smithi164.stdout: osd pool default size = 2 2024-03-23T14:56:00.420 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-23T14:56:00.420 INFO:teuthology.orchestra.run.smithi164.stdout: mon osd allow primary affinity = true 2024-03-23T14:56:00.420 INFO:teuthology.orchestra.run.smithi164.stdout: mon osd allow pg remap = true 2024-03-23T14:56:00.420 INFO:teuthology.orchestra.run.smithi164.stdout: mon warn on legacy crush tunables = false 2024-03-23T14:56:00.420 INFO:teuthology.orchestra.run.smithi164.stdout: mon warn on crush straw calc version zero = false 2024-03-23T14:56:00.420 INFO:teuthology.orchestra.run.smithi164.stdout: mon warn on no sortbitwise = false 2024-03-23T14:56:00.420 INFO:teuthology.orchestra.run.smithi164.stdout: mon warn on osd down out interval zero = false 2024-03-23T14:56:00.420 INFO:teuthology.orchestra.run.smithi164.stdout: mon warn on too few osds = false 2024-03-23T14:56:00.420 INFO:teuthology.orchestra.run.smithi164.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-03-23T14:56:00.421 INFO:teuthology.orchestra.run.smithi164.stdout: mon_warn_on_pool_no_redundancy = false 2024-03-23T14:56:00.421 INFO:teuthology.orchestra.run.smithi164.stdout: mon_allow_pool_size_one = true 2024-03-23T14:56:00.421 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-23T14:56:00.421 INFO:teuthology.orchestra.run.smithi164.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-03-23T14:56:00.421 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-23T14:56:00.421 INFO:teuthology.orchestra.run.smithi164.stdout: osd default data pool replay window = 5 2024-03-23T14:56:00.421 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-23T14:56:00.421 INFO:teuthology.orchestra.run.smithi164.stdout: mon allow pool delete = true 2024-03-23T14:56:00.421 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-23T14:56:00.421 INFO:teuthology.orchestra.run.smithi164.stdout: mon cluster log file level = debug 2024-03-23T14:56:00.421 INFO:teuthology.orchestra.run.smithi164.stdout: debug asserts on shutdown = true 2024-03-23T14:56:00.421 INFO:teuthology.orchestra.run.smithi164.stdout: mon health detail to clog = false 2024-03-23T14:56:00.421 INFO:teuthology.orchestra.run.smithi164.stdout: mon host = "172.21.15.164,172.21.15.173,[v2:172.21.15.173:3301,v1:172.21.15.173:6790]" 2024-03-23T14:56:00.421 INFO:teuthology.orchestra.run.smithi164.stdout: fsid = 45c7911b-b69d-4e68-907b-fbfeb780aa3b 2024-03-23T14:56:00.422 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-23T14:56:00.422 INFO:teuthology.orchestra.run.smithi164.stdout:[osd] 2024-03-23T14:56:00.422 INFO:teuthology.orchestra.run.smithi164.stdout: osd journal size = 100 2024-03-23T14:56:00.422 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-23T14:56:00.422 INFO:teuthology.orchestra.run.smithi164.stdout: osd scrub load threshold = 5.0 2024-03-23T14:56:00.422 INFO:teuthology.orchestra.run.smithi164.stdout: osd scrub max interval = 600 2024-03-23T14:56:00.422 INFO:teuthology.orchestra.run.smithi164.stdout: osd mclock profile = high_recovery_ops 2024-03-23T14:56:00.422 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-23T14:56:00.422 INFO:teuthology.orchestra.run.smithi164.stdout: osd recover clone overlap = true 2024-03-23T14:56:00.422 INFO:teuthology.orchestra.run.smithi164.stdout: osd recovery max chunk = 1048576 2024-03-23T14:56:00.422 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-23T14:56:00.422 INFO:teuthology.orchestra.run.smithi164.stdout: osd debug shutdown = true 2024-03-23T14:56:00.422 INFO:teuthology.orchestra.run.smithi164.stdout: osd debug op order = true 2024-03-23T14:56:00.422 INFO:teuthology.orchestra.run.smithi164.stdout: osd debug verify stray on activate = true 2024-03-23T14:56:00.422 INFO:teuthology.orchestra.run.smithi164.stdout: osd debug trim objects = true 2024-03-23T14:56:00.423 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-23T14:56:00.423 INFO:teuthology.orchestra.run.smithi164.stdout: osd open classes on start = true 2024-03-23T14:56:00.423 INFO:teuthology.orchestra.run.smithi164.stdout: osd debug pg log writeout = true 2024-03-23T14:56:00.423 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-23T14:56:00.423 INFO:teuthology.orchestra.run.smithi164.stdout: osd deep scrub update digest min age = 30 2024-03-23T14:56:00.423 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-23T14:56:00.423 INFO:teuthology.orchestra.run.smithi164.stdout: osd map max advance = 10 2024-03-23T14:56:00.423 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-23T14:56:00.423 INFO:teuthology.orchestra.run.smithi164.stdout: journal zero on create = true 2024-03-23T14:56:00.423 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-23T14:56:00.423 INFO:teuthology.orchestra.run.smithi164.stdout: filestore ondisk finisher threads = 3 2024-03-23T14:56:00.423 INFO:teuthology.orchestra.run.smithi164.stdout: filestore apply finisher threads = 3 2024-03-23T14:56:00.423 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-23T14:56:00.423 INFO:teuthology.orchestra.run.smithi164.stdout: bdev debug aio = true 2024-03-23T14:56:00.424 INFO:teuthology.orchestra.run.smithi164.stdout: osd debug misdirected ops = true 2024-03-23T14:56:00.424 INFO:teuthology.orchestra.run.smithi164.stdout: bdev async discard = True 2024-03-23T14:56:00.424 INFO:teuthology.orchestra.run.smithi164.stdout: bdev enable discard = True 2024-03-23T14:56:00.424 INFO:teuthology.orchestra.run.smithi164.stdout: bluestore allocator = bitmap 2024-03-23T14:56:00.424 INFO:teuthology.orchestra.run.smithi164.stdout: bluestore block size = 96636764160 2024-03-23T14:56:00.424 INFO:teuthology.orchestra.run.smithi164.stdout: bluestore fsck on mount = True 2024-03-23T14:56:00.424 INFO:teuthology.orchestra.run.smithi164.stdout: debug bluefs = 1/20 2024-03-23T14:56:00.424 INFO:teuthology.orchestra.run.smithi164.stdout: debug bluestore = 1/20 2024-03-23T14:56:00.424 INFO:teuthology.orchestra.run.smithi164.stdout: debug ms = 1 2024-03-23T14:56:00.424 INFO:teuthology.orchestra.run.smithi164.stdout: debug osd = 20 2024-03-23T14:56:00.424 INFO:teuthology.orchestra.run.smithi164.stdout: debug rocksdb = 4/10 2024-03-23T14:56:00.424 INFO:teuthology.orchestra.run.smithi164.stdout: mon osd backfillfull_ratio = 0.85 2024-03-23T14:56:00.424 INFO:teuthology.orchestra.run.smithi164.stdout: mon osd full ratio = 0.9 2024-03-23T14:56:00.424 INFO:teuthology.orchestra.run.smithi164.stdout: mon osd nearfull ratio = 0.8 2024-03-23T14:56:00.424 INFO:teuthology.orchestra.run.smithi164.stdout: osd failsafe full ratio = 0.95 2024-03-23T14:56:00.425 INFO:teuthology.orchestra.run.smithi164.stdout: osd objectstore = bluestore 2024-03-23T14:56:00.425 INFO:teuthology.orchestra.run.smithi164.stdout: osd op complaint time = 180 2024-03-23T14:56:00.425 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-23T14:56:00.425 INFO:teuthology.orchestra.run.smithi164.stdout:[mgr] 2024-03-23T14:56:00.425 INFO:teuthology.orchestra.run.smithi164.stdout: debug ms = 1 2024-03-23T14:56:00.425 INFO:teuthology.orchestra.run.smithi164.stdout: debug mgr = 20 2024-03-23T14:56:00.425 INFO:teuthology.orchestra.run.smithi164.stdout: debug mon = 20 2024-03-23T14:56:00.425 INFO:teuthology.orchestra.run.smithi164.stdout: debug auth = 20 2024-03-23T14:56:00.425 INFO:teuthology.orchestra.run.smithi164.stdout: mon reweight min pgs per osd = 4 2024-03-23T14:56:00.425 INFO:teuthology.orchestra.run.smithi164.stdout: mon reweight min bytes per osd = 10 2024-03-23T14:56:00.425 INFO:teuthology.orchestra.run.smithi164.stdout: mgr/telemetry/nag = false 2024-03-23T14:56:00.425 INFO:teuthology.orchestra.run.smithi164.stdout: debug client = 20 2024-03-23T14:56:00.425 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-23T14:56:00.425 INFO:teuthology.orchestra.run.smithi164.stdout:[mon] 2024-03-23T14:56:00.426 INFO:teuthology.orchestra.run.smithi164.stdout: debug ms = 1 2024-03-23T14:56:00.426 INFO:teuthology.orchestra.run.smithi164.stdout: debug mon = 20 2024-03-23T14:56:00.426 INFO:teuthology.orchestra.run.smithi164.stdout: debug paxos = 20 2024-03-23T14:56:00.426 INFO:teuthology.orchestra.run.smithi164.stdout: debug auth = 20 2024-03-23T14:56:00.426 INFO:teuthology.orchestra.run.smithi164.stdout: mon data avail warn = 5 2024-03-23T14:56:00.426 INFO:teuthology.orchestra.run.smithi164.stdout: mon mgr mkfs grace = 240 2024-03-23T14:56:00.426 INFO:teuthology.orchestra.run.smithi164.stdout: mon reweight min pgs per osd = 4 2024-03-23T14:56:00.426 INFO:teuthology.orchestra.run.smithi164.stdout: mon osd reporter subtree level = osd 2024-03-23T14:56:00.426 INFO:teuthology.orchestra.run.smithi164.stdout: mon osd prime pg temp = true 2024-03-23T14:56:00.426 INFO:teuthology.orchestra.run.smithi164.stdout: mon reweight min bytes per osd = 10 2024-03-23T14:56:00.426 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-23T14:56:00.426 INFO:teuthology.orchestra.run.smithi164.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-03-23T14:56:00.426 INFO:teuthology.orchestra.run.smithi164.stdout: auth mon ticket ttl = 660 # 11m 2024-03-23T14:56:00.426 INFO:teuthology.orchestra.run.smithi164.stdout: auth service ticket ttl = 240 # 4m 2024-03-23T14:56:00.427 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-23T14:56:00.427 INFO:teuthology.orchestra.run.smithi164.stdout: # don't complain about insecure global_id in the test suite 2024-03-23T14:56:00.427 INFO:teuthology.orchestra.run.smithi164.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-03-23T14:56:00.427 INFO:teuthology.orchestra.run.smithi164.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-03-23T14:56:00.427 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-23T14:56:00.427 INFO:teuthology.orchestra.run.smithi164.stdout: # 1m isn't quite enough 2024-03-23T14:56:00.427 INFO:teuthology.orchestra.run.smithi164.stdout: mon_down_mkfs_grace = 2m 2024-03-23T14:56:00.427 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-23T14:56:00.427 INFO:teuthology.orchestra.run.smithi164.stdout: mon_warn_on_filestore_osds = false 2024-03-23T14:56:00.427 INFO:teuthology.orchestra.run.smithi164.stdout: mon down mkfs grace = 300 2024-03-23T14:56:00.427 INFO:teuthology.orchestra.run.smithi164.stdout: mon op complaint time = 120 2024-03-23T14:56:00.427 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-23T14:56:00.427 INFO:teuthology.orchestra.run.smithi164.stdout:[client] 2024-03-23T14:56:00.427 INFO:teuthology.orchestra.run.smithi164.stdout: rgw cache enabled = true 2024-03-23T14:56:00.428 INFO:teuthology.orchestra.run.smithi164.stdout: rgw enable ops log = true 2024-03-23T14:56:00.428 INFO:teuthology.orchestra.run.smithi164.stdout: rgw enable usage log = true 2024-03-23T14:56:00.428 INFO:teuthology.orchestra.run.smithi164.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-03-23T14:56:00.428 INFO:teuthology.orchestra.run.smithi164.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-03-23T14:56:00.428 INFO:teuthology.orchestra.run.smithi164.stdout: client mount timeout = 600 2024-03-23T14:56:00.428 INFO:teuthology.orchestra.run.smithi164.stdout: debug client = 20 2024-03-23T14:56:00.428 INFO:teuthology.orchestra.run.smithi164.stdout: debug ms = 1 2024-03-23T14:56:00.428 INFO:teuthology.orchestra.run.smithi164.stdout: pid file = "" 2024-03-23T14:56:00.428 INFO:teuthology.orchestra.run.smithi164.stdout: rados mon op timeout = 900 2024-03-23T14:56:00.428 INFO:teuthology.orchestra.run.smithi164.stdout: rados osd op timeout = 900 2024-03-23T14:56:00.428 INFO:teuthology.orchestra.run.smithi164.stdout:[mon.a] 2024-03-23T14:56:00.428 INFO:teuthology.orchestra.run.smithi164.stdout:[mon.b] 2024-03-23T14:56:00.428 INFO:teuthology.orchestra.run.smithi164.stdout:[mon.c] 2024-03-23T14:56:00.428 INFO:teuthology.orchestra.run.smithi164.stdout:[mds.a] 2024-03-23T14:56:00.428 INFO:teuthology.orchestra.run.smithi164.stdout:[mds.b] 2024-03-23T14:56:00.429 INFO:teuthology.orchestra.run.smithi164.stdout:[mds] 2024-03-23T14:56:00.429 INFO:teuthology.orchestra.run.smithi164.stdout: debug mds = 20 2024-03-23T14:56:00.429 INFO:teuthology.orchestra.run.smithi164.stdout: debug mds balancer = 20 2024-03-23T14:56:00.429 INFO:teuthology.orchestra.run.smithi164.stdout: debug ms = 1 2024-03-23T14:56:00.429 INFO:teuthology.orchestra.run.smithi164.stdout: mds debug frag = True 2024-03-23T14:56:00.429 INFO:teuthology.orchestra.run.smithi164.stdout: mds debug scatterstat = True 2024-03-23T14:56:00.429 INFO:teuthology.orchestra.run.smithi164.stdout: mds op complaint time = 180 2024-03-23T14:56:00.429 INFO:teuthology.orchestra.run.smithi164.stdout: mds verify scatter = True 2024-03-23T14:56:00.429 INFO:teuthology.orchestra.run.smithi164.stdout: osd op complaint time = 180 2024-03-23T14:56:00.429 INFO:teuthology.orchestra.run.smithi164.stdout: rados mon op timeout = 900 2024-03-23T14:56:00.429 INFO:teuthology.orchestra.run.smithi164.stdout: rados osd op timeout = 900 2024-03-23T14:56:00.430 INFO:teuthology.orchestra.run.smithi173.stdout:[global] 2024-03-23T14:56:00.430 INFO:teuthology.orchestra.run.smithi173.stdout: chdir = "" 2024-03-23T14:56:00.430 INFO:teuthology.orchestra.run.smithi173.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-03-23T14:56:00.430 INFO:teuthology.orchestra.run.smithi173.stdout: auth supported = cephx 2024-03-23T14:56:00.430 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-03-23T14:56:00.431 INFO:teuthology.orchestra.run.smithi173.stdout: filestore xattr use omap = true 2024-03-23T14:56:00.431 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-03-23T14:56:00.431 INFO:teuthology.orchestra.run.smithi173.stdout: mon clock drift allowed = 1.000 2024-03-23T14:56:00.431 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-03-23T14:56:00.431 INFO:teuthology.orchestra.run.smithi173.stdout: osd crush chooseleaf type = 0 2024-03-23T14:56:00.431 INFO:teuthology.orchestra.run.smithi173.stdout: auth debug = true 2024-03-23T14:56:00.431 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-03-23T14:56:00.431 INFO:teuthology.orchestra.run.smithi173.stdout: ms die on old message = true 2024-03-23T14:56:00.431 INFO:teuthology.orchestra.run.smithi173.stdout: ms die on bug = true 2024-03-23T14:56:00.431 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-03-23T14:56:00.432 INFO:teuthology.orchestra.run.smithi173.stdout: mon max pg per osd = 10000 # >= luminous 2024-03-23T14:56:00.432 INFO:teuthology.orchestra.run.smithi173.stdout: mon pg warn max object skew = 0 2024-03-23T14:56:00.432 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-03-23T14:56:00.432 INFO:teuthology.orchestra.run.smithi173.stdout: # disable pg_autoscaler by default for new pools 2024-03-23T14:56:00.432 INFO:teuthology.orchestra.run.smithi173.stdout: osd_pool_default_pg_autoscale_mode = off 2024-03-23T14:56:00.432 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-03-23T14:56:00.432 INFO:teuthology.orchestra.run.smithi173.stdout: osd pool default size = 2 2024-03-23T14:56:00.432 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-03-23T14:56:00.432 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd allow primary affinity = true 2024-03-23T14:56:00.432 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd allow pg remap = true 2024-03-23T14:56:00.432 INFO:teuthology.orchestra.run.smithi173.stdout: mon warn on legacy crush tunables = false 2024-03-23T14:56:00.433 INFO:teuthology.orchestra.run.smithi173.stdout: mon warn on crush straw calc version zero = false 2024-03-23T14:56:00.433 INFO:teuthology.orchestra.run.smithi173.stdout: mon warn on no sortbitwise = false 2024-03-23T14:56:00.433 INFO:teuthology.orchestra.run.smithi173.stdout: mon warn on osd down out interval zero = false 2024-03-23T14:56:00.433 INFO:teuthology.orchestra.run.smithi173.stdout: mon warn on too few osds = false 2024-03-23T14:56:00.433 INFO:teuthology.orchestra.run.smithi173.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-03-23T14:56:00.433 INFO:teuthology.orchestra.run.smithi173.stdout: mon_warn_on_pool_no_redundancy = false 2024-03-23T14:56:00.433 INFO:teuthology.orchestra.run.smithi173.stdout: mon_allow_pool_size_one = true 2024-03-23T14:56:00.433 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-03-23T14:56:00.433 INFO:teuthology.orchestra.run.smithi173.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-03-23T14:56:00.433 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-03-23T14:56:00.433 INFO:teuthology.orchestra.run.smithi173.stdout: osd default data pool replay window = 5 2024-03-23T14:56:00.433 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-03-23T14:56:00.434 INFO:teuthology.orchestra.run.smithi173.stdout: mon allow pool delete = true 2024-03-23T14:56:00.434 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-03-23T14:56:00.434 INFO:teuthology.orchestra.run.smithi173.stdout: mon cluster log file level = debug 2024-03-23T14:56:00.434 INFO:teuthology.orchestra.run.smithi173.stdout: debug asserts on shutdown = true 2024-03-23T14:56:00.434 INFO:teuthology.orchestra.run.smithi173.stdout: mon health detail to clog = false 2024-03-23T14:56:00.434 INFO:teuthology.orchestra.run.smithi173.stdout: mon host = "172.21.15.164,172.21.15.173,[v2:172.21.15.173:3301,v1:172.21.15.173:6790]" 2024-03-23T14:56:00.434 INFO:teuthology.orchestra.run.smithi173.stdout: fsid = 45c7911b-b69d-4e68-907b-fbfeb780aa3b 2024-03-23T14:56:00.434 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-03-23T14:56:00.434 INFO:teuthology.orchestra.run.smithi173.stdout:[osd] 2024-03-23T14:56:00.434 INFO:teuthology.orchestra.run.smithi173.stdout: osd journal size = 100 2024-03-23T14:56:00.434 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-03-23T14:56:00.434 INFO:teuthology.orchestra.run.smithi173.stdout: osd scrub load threshold = 5.0 2024-03-23T14:56:00.434 INFO:teuthology.orchestra.run.smithi173.stdout: osd scrub max interval = 600 2024-03-23T14:56:00.434 INFO:teuthology.orchestra.run.smithi173.stdout: osd mclock profile = high_recovery_ops 2024-03-23T14:56:00.435 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-03-23T14:56:00.435 INFO:teuthology.orchestra.run.smithi173.stdout: osd recover clone overlap = true 2024-03-23T14:56:00.435 INFO:teuthology.orchestra.run.smithi173.stdout: osd recovery max chunk = 1048576 2024-03-23T14:56:00.435 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-03-23T14:56:00.435 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug shutdown = true 2024-03-23T14:56:00.435 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug op order = true 2024-03-23T14:56:00.435 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug verify stray on activate = true 2024-03-23T14:56:00.435 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug trim objects = true 2024-03-23T14:56:00.435 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-03-23T14:56:00.435 INFO:teuthology.orchestra.run.smithi173.stdout: osd open classes on start = true 2024-03-23T14:56:00.435 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug pg log writeout = true 2024-03-23T14:56:00.435 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-03-23T14:56:00.435 INFO:teuthology.orchestra.run.smithi173.stdout: osd deep scrub update digest min age = 30 2024-03-23T14:56:00.435 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-03-23T14:56:00.436 INFO:teuthology.orchestra.run.smithi173.stdout: osd map max advance = 10 2024-03-23T14:56:00.436 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-03-23T14:56:00.436 INFO:teuthology.orchestra.run.smithi173.stdout: journal zero on create = true 2024-03-23T14:56:00.436 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-03-23T14:56:00.436 INFO:teuthology.orchestra.run.smithi173.stdout: filestore ondisk finisher threads = 3 2024-03-23T14:56:00.436 INFO:teuthology.orchestra.run.smithi173.stdout: filestore apply finisher threads = 3 2024-03-23T14:56:00.436 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-03-23T14:56:00.436 INFO:teuthology.orchestra.run.smithi173.stdout: bdev debug aio = true 2024-03-23T14:56:00.436 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug misdirected ops = true 2024-03-23T14:56:00.436 INFO:teuthology.orchestra.run.smithi173.stdout: bdev async discard = True 2024-03-23T14:56:00.436 INFO:teuthology.orchestra.run.smithi173.stdout: bdev enable discard = True 2024-03-23T14:56:00.436 INFO:teuthology.orchestra.run.smithi173.stdout: bluestore allocator = bitmap 2024-03-23T14:56:00.436 INFO:teuthology.orchestra.run.smithi173.stdout: bluestore block size = 96636764160 2024-03-23T14:56:00.436 INFO:teuthology.orchestra.run.smithi173.stdout: bluestore fsck on mount = True 2024-03-23T14:56:00.436 INFO:teuthology.orchestra.run.smithi173.stdout: debug bluefs = 1/20 2024-03-23T14:56:00.437 INFO:teuthology.orchestra.run.smithi173.stdout: debug bluestore = 1/20 2024-03-23T14:56:00.437 INFO:teuthology.orchestra.run.smithi173.stdout: debug ms = 1 2024-03-23T14:56:00.437 INFO:teuthology.orchestra.run.smithi173.stdout: debug osd = 20 2024-03-23T14:56:00.437 INFO:teuthology.orchestra.run.smithi173.stdout: debug rocksdb = 4/10 2024-03-23T14:56:00.437 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd backfillfull_ratio = 0.85 2024-03-23T14:56:00.437 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd full ratio = 0.9 2024-03-23T14:56:00.437 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd nearfull ratio = 0.8 2024-03-23T14:56:00.437 INFO:teuthology.orchestra.run.smithi173.stdout: osd failsafe full ratio = 0.95 2024-03-23T14:56:00.437 INFO:teuthology.orchestra.run.smithi173.stdout: osd objectstore = bluestore 2024-03-23T14:56:00.437 INFO:teuthology.orchestra.run.smithi173.stdout: osd op complaint time = 180 2024-03-23T14:56:00.437 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-03-23T14:56:00.437 INFO:teuthology.orchestra.run.smithi173.stdout:[mgr] 2024-03-23T14:56:00.437 INFO:teuthology.orchestra.run.smithi173.stdout: debug ms = 1 2024-03-23T14:56:00.437 INFO:teuthology.orchestra.run.smithi173.stdout: debug mgr = 20 2024-03-23T14:56:00.437 INFO:teuthology.orchestra.run.smithi173.stdout: debug mon = 20 2024-03-23T14:56:00.438 INFO:teuthology.orchestra.run.smithi173.stdout: debug auth = 20 2024-03-23T14:56:00.438 INFO:teuthology.orchestra.run.smithi173.stdout: mon reweight min pgs per osd = 4 2024-03-23T14:56:00.438 INFO:teuthology.orchestra.run.smithi173.stdout: mon reweight min bytes per osd = 10 2024-03-23T14:56:00.438 INFO:teuthology.orchestra.run.smithi173.stdout: mgr/telemetry/nag = false 2024-03-23T14:56:00.438 INFO:teuthology.orchestra.run.smithi173.stdout: debug client = 20 2024-03-23T14:56:00.438 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-03-23T14:56:00.438 INFO:teuthology.orchestra.run.smithi173.stdout:[mon] 2024-03-23T14:56:00.438 INFO:teuthology.orchestra.run.smithi173.stdout: debug ms = 1 2024-03-23T14:56:00.438 INFO:teuthology.orchestra.run.smithi173.stdout: debug mon = 20 2024-03-23T14:56:00.438 INFO:teuthology.orchestra.run.smithi173.stdout: debug paxos = 20 2024-03-23T14:56:00.438 INFO:teuthology.orchestra.run.smithi173.stdout: debug auth = 20 2024-03-23T14:56:00.438 INFO:teuthology.orchestra.run.smithi173.stdout: mon data avail warn = 5 2024-03-23T14:56:00.438 INFO:teuthology.orchestra.run.smithi173.stdout: mon mgr mkfs grace = 240 2024-03-23T14:56:00.438 INFO:teuthology.orchestra.run.smithi173.stdout: mon reweight min pgs per osd = 4 2024-03-23T14:56:00.438 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd reporter subtree level = osd 2024-03-23T14:56:00.439 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd prime pg temp = true 2024-03-23T14:56:00.439 INFO:teuthology.orchestra.run.smithi173.stdout: mon reweight min bytes per osd = 10 2024-03-23T14:56:00.439 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-03-23T14:56:00.439 INFO:teuthology.orchestra.run.smithi173.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-03-23T14:56:00.439 INFO:teuthology.orchestra.run.smithi173.stdout: auth mon ticket ttl = 660 # 11m 2024-03-23T14:56:00.439 INFO:teuthology.orchestra.run.smithi173.stdout: auth service ticket ttl = 240 # 4m 2024-03-23T14:56:00.439 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-03-23T14:56:00.439 INFO:teuthology.orchestra.run.smithi173.stdout: # don't complain about insecure global_id in the test suite 2024-03-23T14:56:00.439 INFO:teuthology.orchestra.run.smithi173.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-03-23T14:56:00.439 INFO:teuthology.orchestra.run.smithi173.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-03-23T14:56:00.439 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-03-23T14:56:00.439 INFO:teuthology.orchestra.run.smithi173.stdout: # 1m isn't quite enough 2024-03-23T14:56:00.439 INFO:teuthology.orchestra.run.smithi173.stdout: mon_down_mkfs_grace = 2m 2024-03-23T14:56:00.439 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-03-23T14:56:00.439 INFO:teuthology.orchestra.run.smithi173.stdout: mon_warn_on_filestore_osds = false 2024-03-23T14:56:00.440 INFO:teuthology.orchestra.run.smithi173.stdout: mon down mkfs grace = 300 2024-03-23T14:56:00.440 INFO:teuthology.orchestra.run.smithi173.stdout: mon op complaint time = 120 2024-03-23T14:56:00.440 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-03-23T14:56:00.440 INFO:teuthology.orchestra.run.smithi173.stdout:[client] 2024-03-23T14:56:00.440 INFO:teuthology.orchestra.run.smithi173.stdout: rgw cache enabled = true 2024-03-23T14:56:00.440 INFO:teuthology.orchestra.run.smithi173.stdout: rgw enable ops log = true 2024-03-23T14:56:00.440 INFO:teuthology.orchestra.run.smithi173.stdout: rgw enable usage log = true 2024-03-23T14:56:00.440 INFO:teuthology.orchestra.run.smithi173.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-03-23T14:56:00.440 INFO:teuthology.orchestra.run.smithi173.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-03-23T14:56:00.440 INFO:teuthology.orchestra.run.smithi173.stdout: client mount timeout = 600 2024-03-23T14:56:00.440 INFO:teuthology.orchestra.run.smithi173.stdout: debug client = 20 2024-03-23T14:56:00.440 INFO:teuthology.orchestra.run.smithi173.stdout: debug ms = 1 2024-03-23T14:56:00.440 INFO:teuthology.orchestra.run.smithi173.stdout: pid file = "" 2024-03-23T14:56:00.440 INFO:teuthology.orchestra.run.smithi173.stdout: rados mon op timeout = 900 2024-03-23T14:56:00.440 INFO:teuthology.orchestra.run.smithi173.stdout: rados osd op timeout = 900 2024-03-23T14:56:00.441 INFO:teuthology.orchestra.run.smithi173.stdout:[mon.a] 2024-03-23T14:56:00.441 INFO:teuthology.orchestra.run.smithi173.stdout:[mon.b] 2024-03-23T14:56:00.441 INFO:teuthology.orchestra.run.smithi173.stdout:[mon.c] 2024-03-23T14:56:00.441 INFO:teuthology.orchestra.run.smithi173.stdout:[mds.a] 2024-03-23T14:56:00.441 INFO:teuthology.orchestra.run.smithi173.stdout:[mds.b] 2024-03-23T14:56:00.441 INFO:teuthology.orchestra.run.smithi173.stdout:[mds] 2024-03-23T14:56:00.441 INFO:teuthology.orchestra.run.smithi173.stdout: debug mds = 20 2024-03-23T14:56:00.441 INFO:teuthology.orchestra.run.smithi173.stdout: debug mds balancer = 20 2024-03-23T14:56:00.441 INFO:teuthology.orchestra.run.smithi173.stdout: debug ms = 1 2024-03-23T14:56:00.441 INFO:teuthology.orchestra.run.smithi173.stdout: mds debug frag = True 2024-03-23T14:56:00.441 INFO:teuthology.orchestra.run.smithi173.stdout: mds debug scatterstat = True 2024-03-23T14:56:00.441 INFO:teuthology.orchestra.run.smithi173.stdout: mds op complaint time = 180 2024-03-23T14:56:00.441 INFO:teuthology.orchestra.run.smithi173.stdout: mds verify scatter = True 2024-03-23T14:56:00.442 INFO:teuthology.orchestra.run.smithi173.stdout: osd op complaint time = 180 2024-03-23T14:56:00.442 INFO:teuthology.orchestra.run.smithi173.stdout: rados mon op timeout = 900 2024-03-23T14:56:00.442 INFO:teuthology.orchestra.run.smithi173.stdout: rados osd op timeout = 900 2024-03-23T14:56:00.444 INFO:tasks.ceph:Creating admin key on mon.a... 2024-03-23T14:56:00.444 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-03-23T14:56:00.532 INFO:tasks.ceph:Copying monmap to all nodes... 2024-03-23T14:56:00.532 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-23T14:56:00.533 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-03-23T14:56:00.538 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-23T14:56:00.538 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-03-23T14:56:00.585 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi164.front.sepia.ceph.com 2024-03-23T14:56:00.585 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-23T14:56:00.585 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/etc/ceph/ceph.keyring 2024-03-23T14:56:00.585 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-03-23T14:56:00.647 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-23T14:56:00.648 DEBUG:teuthology.orchestra.run.smithi164:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-03-23T14:56:00.699 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi173.front.sepia.ceph.com 2024-03-23T14:56:00.699 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-03-23T14:56:00.699 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd of=/etc/ceph/ceph.keyring 2024-03-23T14:56:00.699 DEBUG:teuthology.orchestra.run.smithi173:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-03-23T14:56:00.724 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-03-23T14:56:00.724 DEBUG:teuthology.orchestra.run.smithi173:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-03-23T14:56:00.773 INFO:tasks.ceph:Setting up mon nodes... 2024-03-23T14:56:00.773 INFO:tasks.ceph:Setting up mgr nodes... 2024-03-23T14:56:00.773 DEBUG:teuthology.orchestra.run.smithi164:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-03-23T14:56:00.816 INFO:teuthology.orchestra.run.smithi164.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-03-23T14:56:00.819 DEBUG:teuthology.orchestra.run.smithi173:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-03-23T14:56:00.864 INFO:teuthology.orchestra.run.smithi173.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-03-23T14:56:00.878 INFO:tasks.ceph:Setting up mds nodes... 2024-03-23T14:56:00.878 DEBUG:teuthology.orchestra.run.smithi164:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-03-23T14:56:00.918 INFO:teuthology.orchestra.run.smithi164.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-03-23T14:56:00.922 DEBUG:teuthology.orchestra.run.smithi164:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-03-23T14:56:00.978 DEBUG:teuthology.orchestra.run.smithi173:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-03-23T14:56:01.021 INFO:teuthology.orchestra.run.smithi173.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-03-23T14:56:01.024 DEBUG:teuthology.orchestra.run.smithi173:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-03-23T14:56:01.081 INFO:tasks.ceph_client:Setting up client nodes... 2024-03-23T14:56:01.081 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-03-23T14:56:01.117 INFO:teuthology.orchestra.run.smithi164.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-03-23T14:56:01.128 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-03-23T14:56:01.128 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi164.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi173.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-03-23T14:56:01.129 DEBUG:teuthology.orchestra.run.smithi164:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-03-23T14:56:01.185 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-23T14:56:01.185 INFO:tasks.ceph:role: osd.0 2024-03-23T14:56:01.185 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi164.front.sepia.ceph.com 2024-03-23T14:56:01.185 DEBUG:teuthology.orchestra.run.smithi164:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-03-23T14:56:01.250 INFO:teuthology.orchestra.run.smithi164.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-03-23T14:56:01.251 INFO:teuthology.orchestra.run.smithi164.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-23T14:56:01.251 INFO:teuthology.orchestra.run.smithi164.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-23T14:56:01.251 INFO:teuthology.orchestra.run.smithi164.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-03-23T14:56:01.251 INFO:teuthology.orchestra.run.smithi164.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-23T14:56:01.251 INFO:teuthology.orchestra.run.smithi164.stdout: = sunit=0 swidth=0 blks 2024-03-23T14:56:01.251 INFO:teuthology.orchestra.run.smithi164.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-23T14:56:01.251 INFO:teuthology.orchestra.run.smithi164.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-03-23T14:56:01.251 INFO:teuthology.orchestra.run.smithi164.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-23T14:56:01.251 INFO:teuthology.orchestra.run.smithi164.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-23T14:56:01.724 INFO:teuthology.orchestra.run.smithi164.stdout:Discarding blocks...Done. 2024-03-23T14:56:01.726 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi164.front.sepia.ceph.com -o noatime 2024-03-23T14:56:01.726 DEBUG:teuthology.orchestra.run.smithi164:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-03-23T14:56:01.868 DEBUG:teuthology.orchestra.run.smithi164:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-03-23T14:56:01.879 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-23T14:56:01.880 INFO:teuthology.orchestra.run.smithi164.stderr:sudo: /sbin/restorecon: command not found 2024-03-23T14:56:01.880 DEBUG:teuthology.orchestra.run.smithi164:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-03-23T14:56:01.936 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-23T14:56:01.936 INFO:tasks.ceph:role: osd.1 2024-03-23T14:56:01.936 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi164.front.sepia.ceph.com 2024-03-23T14:56:01.937 DEBUG:teuthology.orchestra.run.smithi164:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-03-23T14:56:02.000 INFO:teuthology.orchestra.run.smithi164.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-03-23T14:56:02.001 INFO:teuthology.orchestra.run.smithi164.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-23T14:56:02.001 INFO:teuthology.orchestra.run.smithi164.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-23T14:56:02.001 INFO:teuthology.orchestra.run.smithi164.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-03-23T14:56:02.001 INFO:teuthology.orchestra.run.smithi164.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-23T14:56:02.001 INFO:teuthology.orchestra.run.smithi164.stdout: = sunit=0 swidth=0 blks 2024-03-23T14:56:02.001 INFO:teuthology.orchestra.run.smithi164.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-23T14:56:02.001 INFO:teuthology.orchestra.run.smithi164.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-03-23T14:56:02.001 INFO:teuthology.orchestra.run.smithi164.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-23T14:56:02.001 INFO:teuthology.orchestra.run.smithi164.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-23T14:56:02.437 INFO:teuthology.orchestra.run.smithi164.stdout:Discarding blocks...Done. 2024-03-23T14:56:02.439 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi164.front.sepia.ceph.com -o noatime 2024-03-23T14:56:02.439 DEBUG:teuthology.orchestra.run.smithi164:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-03-23T14:56:02.501 DEBUG:teuthology.orchestra.run.smithi164:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-03-23T14:56:02.553 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-23T14:56:02.553 INFO:teuthology.orchestra.run.smithi164.stderr:sudo: /sbin/restorecon: command not found 2024-03-23T14:56:02.554 DEBUG:teuthology.orchestra.run.smithi164:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-03-23T14:56:02.606 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-23T14:56:02.607 INFO:tasks.ceph:role: osd.2 2024-03-23T14:56:02.607 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi164.front.sepia.ceph.com 2024-03-23T14:56:02.607 DEBUG:teuthology.orchestra.run.smithi164:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-03-23T14:56:02.669 INFO:teuthology.orchestra.run.smithi164.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-03-23T14:56:02.669 INFO:teuthology.orchestra.run.smithi164.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-23T14:56:02.669 INFO:teuthology.orchestra.run.smithi164.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-23T14:56:02.669 INFO:teuthology.orchestra.run.smithi164.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-03-23T14:56:02.669 INFO:teuthology.orchestra.run.smithi164.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-23T14:56:02.669 INFO:teuthology.orchestra.run.smithi164.stdout: = sunit=0 swidth=0 blks 2024-03-23T14:56:02.669 INFO:teuthology.orchestra.run.smithi164.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-23T14:56:02.669 INFO:teuthology.orchestra.run.smithi164.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-03-23T14:56:02.669 INFO:teuthology.orchestra.run.smithi164.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-23T14:56:02.669 INFO:teuthology.orchestra.run.smithi164.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-23T14:56:03.122 INFO:teuthology.orchestra.run.smithi164.stdout:Discarding blocks...Done. 2024-03-23T14:56:03.124 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi164.front.sepia.ceph.com -o noatime 2024-03-23T14:56:03.124 DEBUG:teuthology.orchestra.run.smithi164:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-03-23T14:56:03.222 DEBUG:teuthology.orchestra.run.smithi164:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-03-23T14:56:03.231 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-23T14:56:03.231 INFO:teuthology.orchestra.run.smithi164.stderr:sudo: /sbin/restorecon: command not found 2024-03-23T14:56:03.232 DEBUG:teuthology.orchestra.run.smithi164:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-03-23T14:56:03.285 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-23T14:56:03.285 INFO:tasks.ceph:role: osd.3 2024-03-23T14:56:03.285 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi164.front.sepia.ceph.com 2024-03-23T14:56:03.285 DEBUG:teuthology.orchestra.run.smithi164:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-03-23T14:56:03.347 INFO:teuthology.orchestra.run.smithi164.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-03-23T14:56:03.347 INFO:teuthology.orchestra.run.smithi164.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-23T14:56:03.348 INFO:teuthology.orchestra.run.smithi164.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-23T14:56:03.348 INFO:teuthology.orchestra.run.smithi164.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-03-23T14:56:03.348 INFO:teuthology.orchestra.run.smithi164.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-23T14:56:03.348 INFO:teuthology.orchestra.run.smithi164.stdout: = sunit=0 swidth=0 blks 2024-03-23T14:56:03.348 INFO:teuthology.orchestra.run.smithi164.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-23T14:56:03.348 INFO:teuthology.orchestra.run.smithi164.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-03-23T14:56:03.348 INFO:teuthology.orchestra.run.smithi164.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-23T14:56:03.348 INFO:teuthology.orchestra.run.smithi164.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-23T14:56:03.811 INFO:teuthology.orchestra.run.smithi164.stdout:Discarding blocks...Done. 2024-03-23T14:56:03.813 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi164.front.sepia.ceph.com -o noatime 2024-03-23T14:56:03.813 DEBUG:teuthology.orchestra.run.smithi164:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-03-23T14:56:03.982 DEBUG:teuthology.orchestra.run.smithi164:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-03-23T14:56:03.993 INFO:teuthology.orchestra.run.smithi164.stderr:sudo: /sbin/restorecon: command not found 2024-03-23T14:56:03.994 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-23T14:56:03.994 DEBUG:teuthology.orchestra.run.smithi164:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-23T14:56:04.108 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:04.102+0000 7f12099a67c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-03-23T14:56:04.111 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:04.106+0000 7f12099a67c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-03-23T14:56:04.111 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:04.106+0000 7f12099a67c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-03-23T14:56:04.111 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:04.106+0000 7f12099a67c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-03-23T14:56:04.111 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:04.106+0000 7f12099a67c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-03-23T14:56:04.119 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:04.114+0000 7f12099a67c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-03-23T14:56:07.100 DEBUG:teuthology.orchestra.run.smithi164:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-03-23T14:56:07.112 DEBUG:teuthology.orchestra.run.smithi164:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-23T14:56:07.193 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:07.190+0000 7f8da60027c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-03-23T14:56:07.194 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:07.190+0000 7f8da60027c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-03-23T14:56:07.194 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:07.190+0000 7f8da60027c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-03-23T14:56:07.194 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:07.190+0000 7f8da60027c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-03-23T14:56:07.194 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:07.190+0000 7f8da60027c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-03-23T14:56:07.194 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:07.190+0000 7f8da60027c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-03-23T14:56:10.170 DEBUG:teuthology.orchestra.run.smithi164:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-03-23T14:56:10.225 DEBUG:teuthology.orchestra.run.smithi164:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-23T14:56:10.305 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:10.302+0000 7f79df6d57c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-03-23T14:56:10.305 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:10.302+0000 7f79df6d57c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-03-23T14:56:10.305 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:10.302+0000 7f79df6d57c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-03-23T14:56:10.305 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:10.302+0000 7f79df6d57c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-03-23T14:56:10.305 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:10.302+0000 7f79df6d57c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-03-23T14:56:10.306 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:10.302+0000 7f79df6d57c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-03-23T14:56:13.287 DEBUG:teuthology.orchestra.run.smithi164:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-03-23T14:56:13.341 DEBUG:teuthology.orchestra.run.smithi164:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-23T14:56:13.422 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:13.418+0000 7fad5b1437c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-03-23T14:56:13.423 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:13.418+0000 7fad5b1437c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-03-23T14:56:13.423 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:13.418+0000 7fad5b1437c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-03-23T14:56:13.423 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:13.418+0000 7fad5b1437c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-03-23T14:56:13.423 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:13.418+0000 7fad5b1437c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-03-23T14:56:13.424 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:13.418+0000 7fad5b1437c0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-03-23T14:56:16.407 DEBUG:teuthology.orchestra.run.smithi164:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-03-23T14:56:16.461 DEBUG:teuthology.orchestra.run.smithi173:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-03-23T14:56:16.475 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-23T14:56:16.475 INFO:tasks.ceph:role: osd.4 2024-03-23T14:56:16.475 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi173.front.sepia.ceph.com 2024-03-23T14:56:16.476 DEBUG:teuthology.orchestra.run.smithi173:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-03-23T14:56:16.540 INFO:teuthology.orchestra.run.smithi173.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-03-23T14:56:16.541 INFO:teuthology.orchestra.run.smithi173.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-23T14:56:16.541 INFO:teuthology.orchestra.run.smithi173.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-23T14:56:16.541 INFO:teuthology.orchestra.run.smithi173.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-03-23T14:56:16.541 INFO:teuthology.orchestra.run.smithi173.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-23T14:56:16.541 INFO:teuthology.orchestra.run.smithi173.stdout: = sunit=0 swidth=0 blks 2024-03-23T14:56:16.541 INFO:teuthology.orchestra.run.smithi173.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-23T14:56:16.541 INFO:teuthology.orchestra.run.smithi173.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-03-23T14:56:16.541 INFO:teuthology.orchestra.run.smithi173.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-23T14:56:16.541 INFO:teuthology.orchestra.run.smithi173.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-23T14:56:17.022 INFO:teuthology.orchestra.run.smithi173.stdout:Discarding blocks...Done. 2024-03-23T14:56:17.023 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi173.front.sepia.ceph.com -o noatime 2024-03-23T14:56:17.023 DEBUG:teuthology.orchestra.run.smithi173:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-03-23T14:56:17.173 DEBUG:teuthology.orchestra.run.smithi173:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-03-23T14:56:17.183 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: /sbin/restorecon: command not found 2024-03-23T14:56:17.184 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-23T14:56:17.184 DEBUG:teuthology.orchestra.run.smithi173:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-03-23T14:56:17.239 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-23T14:56:17.240 INFO:tasks.ceph:role: osd.5 2024-03-23T14:56:17.240 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi173.front.sepia.ceph.com 2024-03-23T14:56:17.240 DEBUG:teuthology.orchestra.run.smithi173:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-03-23T14:56:17.304 INFO:teuthology.orchestra.run.smithi173.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-03-23T14:56:17.304 INFO:teuthology.orchestra.run.smithi173.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-23T14:56:17.304 INFO:teuthology.orchestra.run.smithi173.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-23T14:56:17.304 INFO:teuthology.orchestra.run.smithi173.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-03-23T14:56:17.305 INFO:teuthology.orchestra.run.smithi173.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-23T14:56:17.305 INFO:teuthology.orchestra.run.smithi173.stdout: = sunit=0 swidth=0 blks 2024-03-23T14:56:17.305 INFO:teuthology.orchestra.run.smithi173.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-23T14:56:17.305 INFO:teuthology.orchestra.run.smithi173.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-03-23T14:56:17.305 INFO:teuthology.orchestra.run.smithi173.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-23T14:56:17.305 INFO:teuthology.orchestra.run.smithi173.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-23T14:56:17.756 INFO:teuthology.orchestra.run.smithi173.stdout:Discarding blocks...Done. 2024-03-23T14:56:17.758 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi173.front.sepia.ceph.com -o noatime 2024-03-23T14:56:17.758 DEBUG:teuthology.orchestra.run.smithi173:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-03-23T14:56:17.844 DEBUG:teuthology.orchestra.run.smithi173:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-03-23T14:56:17.898 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-23T14:56:17.898 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: /sbin/restorecon: command not found 2024-03-23T14:56:17.899 DEBUG:teuthology.orchestra.run.smithi173:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-03-23T14:56:17.951 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-23T14:56:17.952 INFO:tasks.ceph:role: osd.6 2024-03-23T14:56:17.952 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi173.front.sepia.ceph.com 2024-03-23T14:56:17.952 DEBUG:teuthology.orchestra.run.smithi173:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-03-23T14:56:18.016 INFO:teuthology.orchestra.run.smithi173.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-03-23T14:56:18.017 INFO:teuthology.orchestra.run.smithi173.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-23T14:56:18.017 INFO:teuthology.orchestra.run.smithi173.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-23T14:56:18.017 INFO:teuthology.orchestra.run.smithi173.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-03-23T14:56:18.017 INFO:teuthology.orchestra.run.smithi173.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-23T14:56:18.017 INFO:teuthology.orchestra.run.smithi173.stdout: = sunit=0 swidth=0 blks 2024-03-23T14:56:18.017 INFO:teuthology.orchestra.run.smithi173.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-23T14:56:18.017 INFO:teuthology.orchestra.run.smithi173.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-03-23T14:56:18.017 INFO:teuthology.orchestra.run.smithi173.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-23T14:56:18.017 INFO:teuthology.orchestra.run.smithi173.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-23T14:56:18.475 INFO:teuthology.orchestra.run.smithi173.stdout:Discarding blocks...Done. 2024-03-23T14:56:18.477 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi173.front.sepia.ceph.com -o noatime 2024-03-23T14:56:18.477 DEBUG:teuthology.orchestra.run.smithi173:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-03-23T14:56:18.604 DEBUG:teuthology.orchestra.run.smithi173:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-03-23T14:56:18.615 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-23T14:56:18.615 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: /sbin/restorecon: command not found 2024-03-23T14:56:18.615 DEBUG:teuthology.orchestra.run.smithi173:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-03-23T14:56:18.671 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-23T14:56:18.672 INFO:tasks.ceph:role: osd.7 2024-03-23T14:56:18.672 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi173.front.sepia.ceph.com 2024-03-23T14:56:18.672 DEBUG:teuthology.orchestra.run.smithi173:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-03-23T14:56:18.737 INFO:teuthology.orchestra.run.smithi173.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-03-23T14:56:18.737 INFO:teuthology.orchestra.run.smithi173.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-23T14:56:18.737 INFO:teuthology.orchestra.run.smithi173.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-23T14:56:18.737 INFO:teuthology.orchestra.run.smithi173.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-03-23T14:56:18.737 INFO:teuthology.orchestra.run.smithi173.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-23T14:56:18.737 INFO:teuthology.orchestra.run.smithi173.stdout: = sunit=0 swidth=0 blks 2024-03-23T14:56:18.737 INFO:teuthology.orchestra.run.smithi173.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-23T14:56:18.737 INFO:teuthology.orchestra.run.smithi173.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-03-23T14:56:18.737 INFO:teuthology.orchestra.run.smithi173.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-23T14:56:18.737 INFO:teuthology.orchestra.run.smithi173.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-23T14:56:19.227 INFO:teuthology.orchestra.run.smithi173.stdout:Discarding blocks...Done. 2024-03-23T14:56:19.229 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi173.front.sepia.ceph.com -o noatime 2024-03-23T14:56:19.229 DEBUG:teuthology.orchestra.run.smithi173:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-03-23T14:56:19.373 DEBUG:teuthology.orchestra.run.smithi173:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-03-23T14:56:19.382 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-23T14:56:19.383 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: /sbin/restorecon: command not found 2024-03-23T14:56:19.383 DEBUG:teuthology.orchestra.run.smithi173:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-23T14:56:19.493 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:19.490+0000 7f68f25317c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-03-23T14:56:19.496 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:19.494+0000 7f68f25317c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-03-23T14:56:19.497 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:19.494+0000 7f68f25317c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-03-23T14:56:19.497 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:19.494+0000 7f68f25317c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-03-23T14:56:19.497 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:19.494+0000 7f68f25317c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-03-23T14:56:19.497 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:19.494+0000 7f68f25317c0 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-03-23T14:56:22.484 DEBUG:teuthology.orchestra.run.smithi173:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-03-23T14:56:22.498 DEBUG:teuthology.orchestra.run.smithi173:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-23T14:56:22.582 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:22.578+0000 7ff1ffed27c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-03-23T14:56:22.583 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:22.578+0000 7ff1ffed27c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-03-23T14:56:22.583 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:22.578+0000 7ff1ffed27c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-03-23T14:56:22.583 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:22.578+0000 7ff1ffed27c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-03-23T14:56:22.583 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:22.578+0000 7ff1ffed27c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-03-23T14:56:22.584 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:22.578+0000 7ff1ffed27c0 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-03-23T14:56:25.560 DEBUG:teuthology.orchestra.run.smithi173:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-03-23T14:56:25.621 DEBUG:teuthology.orchestra.run.smithi173:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-23T14:56:25.702 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:25.698+0000 7f67f8a327c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-03-23T14:56:25.703 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:25.698+0000 7f67f8a327c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-03-23T14:56:25.703 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:25.698+0000 7f67f8a327c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-03-23T14:56:25.703 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:25.698+0000 7f67f8a327c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-03-23T14:56:25.703 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:25.698+0000 7f67f8a327c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-03-23T14:56:25.703 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:25.698+0000 7f67f8a327c0 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-03-23T14:56:28.617 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-23T14:56:28.620 DEBUG:teuthology.orchestra.run.smithi173:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-23T14:56:28.714 DEBUG:teuthology.orchestra.run.smithi173:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-03-23T14:56:28.772 DEBUG:teuthology.orchestra.run.smithi173:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-23T14:56:28.856 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:28.854+0000 7f03667c27c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-03-23T14:56:28.857 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:28.854+0000 7f03667c27c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-03-23T14:56:28.857 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:28.854+0000 7f03667c27c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-03-23T14:56:28.857 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:28.854+0000 7f03667c27c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-03-23T14:56:28.857 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:28.854+0000 7f03667c27c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-03-23T14:56:28.857 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:28.854+0000 7f03667c27c0 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-03-23T14:56:31.855 DEBUG:teuthology.orchestra.run.smithi173:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-03-23T14:56:31.909 INFO:tasks.ceph:Reading keys from all nodes... 2024-03-23T14:56:31.909 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-23T14:56:31.910 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-03-23T14:56:31.924 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-23T14:56:31.924 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-03-23T14:56:31.980 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-23T14:56:31.980 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-03-23T14:56:32.035 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-23T14:56:32.035 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-03-23T14:56:32.087 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-23T14:56:32.087 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-03-23T14:56:32.139 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-23T14:56:32.139 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-03-23T14:56:32.195 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-03-23T14:56:32.195 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-03-23T14:56:32.208 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-03-23T14:56:32.208 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-03-23T14:56:32.263 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-03-23T14:56:32.263 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-03-23T14:56:32.316 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-03-23T14:56:32.316 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-03-23T14:56:32.372 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-03-23T14:56:32.372 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-03-23T14:56:32.428 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-03-23T14:56:32.428 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-03-23T14:56:32.483 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-23T14:56:32.483 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-03-23T14:56:32.489 INFO:tasks.ceph:Adding keys to all mons... 2024-03-23T14:56:32.489 DEBUG:teuthology.orchestra.run.smithi164:> sudo tee -a /etc/ceph/ceph.keyring 2024-03-23T14:56:32.533 DEBUG:teuthology.orchestra.run.smithi173:> sudo tee -a /etc/ceph/ceph.keyring 2024-03-23T14:56:32.543 INFO:teuthology.orchestra.run.smithi164.stdout:[mgr.y] 2024-03-23T14:56:32.543 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQCA7f5lZsO0MBAAuIBRk8br/Uai/75hGuHnpA== 2024-03-23T14:56:32.543 INFO:teuthology.orchestra.run.smithi164.stdout:[mds.a] 2024-03-23T14:56:32.543 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQCA7f5lorXGNhAAjdjj6HC1um5/oohvuTLBAw== 2024-03-23T14:56:32.543 INFO:teuthology.orchestra.run.smithi164.stdout:[osd.0] 2024-03-23T14:56:32.543 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQCE7f5l0od2BhAAxjRnHvlZPAOTJcg/vVeEHg== 2024-03-23T14:56:32.543 INFO:teuthology.orchestra.run.smithi164.stdout:[osd.1] 2024-03-23T14:56:32.543 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQCH7f5lFsGOCxAAMHsOolSe7Mk/oxuT/crNuQ== 2024-03-23T14:56:32.544 INFO:teuthology.orchestra.run.smithi164.stdout:[osd.2] 2024-03-23T14:56:32.544 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQCK7f5lNig1EhAAr1K4ulKZ5udSrhqmWOjsOg== 2024-03-23T14:56:32.544 INFO:teuthology.orchestra.run.smithi164.stdout:[osd.3] 2024-03-23T14:56:32.544 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQCN7f5lM7A2GRAAkLg1AGmOL1GTDqrrBAsC/Q== 2024-03-23T14:56:32.544 INFO:teuthology.orchestra.run.smithi164.stdout:[mgr.x] 2024-03-23T14:56:32.544 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQCA7f5l7BaWMxAAeIPg+NXPYnVBG+rK9VUZAg== 2024-03-23T14:56:32.544 INFO:teuthology.orchestra.run.smithi164.stdout:[mds.b] 2024-03-23T14:56:32.544 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQCB7f5lZfVOARAAVbo4snMrB3dMWB8BpS9UHA== 2024-03-23T14:56:32.544 INFO:teuthology.orchestra.run.smithi164.stdout:[osd.4] 2024-03-23T14:56:32.544 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQCT7f5lP3FwHRAAspF4SEUl4WLR8bQJGKnJfQ== 2024-03-23T14:56:32.544 INFO:teuthology.orchestra.run.smithi164.stdout:[osd.5] 2024-03-23T14:56:32.544 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQCW7f5l24HAIhAAt6VXAYDqJlBP317/cnwJbQ== 2024-03-23T14:56:32.545 INFO:teuthology.orchestra.run.smithi164.stdout:[osd.6] 2024-03-23T14:56:32.545 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQCZ7f5lpQfmKRAAnME++cPS67LD/BgEXA931w== 2024-03-23T14:56:32.545 INFO:teuthology.orchestra.run.smithi164.stdout:[osd.7] 2024-03-23T14:56:32.545 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQCc7f5lIlQTMxAAAZNbA78j5XuIcbfNLgQsuA== 2024-03-23T14:56:32.545 INFO:teuthology.orchestra.run.smithi164.stdout:[client.0] 2024-03-23T14:56:32.545 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQCB7f5ldBX+BhAAqsE6K6jE0UUaUO+MTCYmOg== 2024-03-23T14:56:32.546 INFO:teuthology.orchestra.run.smithi173.stdout:[mgr.y] 2024-03-23T14:56:32.546 INFO:teuthology.orchestra.run.smithi173.stdout: key = AQCA7f5lZsO0MBAAuIBRk8br/Uai/75hGuHnpA== 2024-03-23T14:56:32.546 INFO:teuthology.orchestra.run.smithi173.stdout:[mds.a] 2024-03-23T14:56:32.547 INFO:teuthology.orchestra.run.smithi173.stdout: key = AQCA7f5lorXGNhAAjdjj6HC1um5/oohvuTLBAw== 2024-03-23T14:56:32.547 INFO:teuthology.orchestra.run.smithi173.stdout:[osd.0] 2024-03-23T14:56:32.547 INFO:teuthology.orchestra.run.smithi173.stdout: key = AQCE7f5l0od2BhAAxjRnHvlZPAOTJcg/vVeEHg== 2024-03-23T14:56:32.547 INFO:teuthology.orchestra.run.smithi173.stdout:[osd.1] 2024-03-23T14:56:32.547 INFO:teuthology.orchestra.run.smithi173.stdout: key = AQCH7f5lFsGOCxAAMHsOolSe7Mk/oxuT/crNuQ== 2024-03-23T14:56:32.547 INFO:teuthology.orchestra.run.smithi173.stdout:[osd.2] 2024-03-23T14:56:32.547 INFO:teuthology.orchestra.run.smithi173.stdout: key = AQCK7f5lNig1EhAAr1K4ulKZ5udSrhqmWOjsOg== 2024-03-23T14:56:32.547 INFO:teuthology.orchestra.run.smithi173.stdout:[osd.3] 2024-03-23T14:56:32.547 INFO:teuthology.orchestra.run.smithi173.stdout: key = AQCN7f5lM7A2GRAAkLg1AGmOL1GTDqrrBAsC/Q== 2024-03-23T14:56:32.547 INFO:teuthology.orchestra.run.smithi173.stdout:[mgr.x] 2024-03-23T14:56:32.547 INFO:teuthology.orchestra.run.smithi173.stdout: key = AQCA7f5l7BaWMxAAeIPg+NXPYnVBG+rK9VUZAg== 2024-03-23T14:56:32.547 INFO:teuthology.orchestra.run.smithi173.stdout:[mds.b] 2024-03-23T14:56:32.547 INFO:teuthology.orchestra.run.smithi173.stdout: key = AQCB7f5lZfVOARAAVbo4snMrB3dMWB8BpS9UHA== 2024-03-23T14:56:32.547 INFO:teuthology.orchestra.run.smithi173.stdout:[osd.4] 2024-03-23T14:56:32.548 INFO:teuthology.orchestra.run.smithi173.stdout: key = AQCT7f5lP3FwHRAAspF4SEUl4WLR8bQJGKnJfQ== 2024-03-23T14:56:32.548 INFO:teuthology.orchestra.run.smithi173.stdout:[osd.5] 2024-03-23T14:56:32.548 INFO:teuthology.orchestra.run.smithi173.stdout: key = AQCW7f5l24HAIhAAt6VXAYDqJlBP317/cnwJbQ== 2024-03-23T14:56:32.548 INFO:teuthology.orchestra.run.smithi173.stdout:[osd.6] 2024-03-23T14:56:32.548 INFO:teuthology.orchestra.run.smithi173.stdout: key = AQCZ7f5lpQfmKRAAnME++cPS67LD/BgEXA931w== 2024-03-23T14:56:32.548 INFO:teuthology.orchestra.run.smithi173.stdout:[osd.7] 2024-03-23T14:56:32.548 INFO:teuthology.orchestra.run.smithi173.stdout: key = AQCc7f5lIlQTMxAAAZNbA78j5XuIcbfNLgQsuA== 2024-03-23T14:56:32.548 INFO:teuthology.orchestra.run.smithi173.stdout:[client.0] 2024-03-23T14:56:32.548 INFO:teuthology.orchestra.run.smithi173.stdout: key = AQCB7f5ldBX+BhAAqsE6K6jE0UUaUO+MTCYmOg== 2024-03-23T14:56:32.548 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-23T14:56:32.589 DEBUG:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-23T14:56:32.634 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-23T14:56:32.636 DEBUG:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-23T14:56:32.720 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-23T14:56:32.723 DEBUG:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-23T14:56:32.761 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-23T14:56:32.806 DEBUG:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-23T14:56:32.846 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-23T14:56:32.886 DEBUG:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-23T14:56:32.930 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-23T14:56:32.966 DEBUG:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-23T14:56:33.011 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-23T14:56:33.045 DEBUG:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-23T14:56:33.095 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-23T14:56:33.125 DEBUG:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-23T14:56:33.179 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-23T14:56:33.209 DEBUG:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-23T14:56:33.257 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-23T14:56:33.293 DEBUG:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-23T14:56:33.343 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-23T14:56:33.373 DEBUG:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-23T14:56:33.433 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-23T14:56:33.453 DEBUG:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-23T14:56:33.496 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-23T14:56:33.541 DEBUG:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-23T14:56:33.588 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-03-23T14:56:33.588 DEBUG:teuthology.orchestra.run.smithi164:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-03-23T14:56:33.600 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-03-23T14:56:33.690 DEBUG:teuthology.orchestra.run.smithi164:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-03-23T14:56:33.700 DEBUG:teuthology.orchestra.run.smithi173:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-03-23T14:56:33.714 DEBUG:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-03-23T14:56:33.807 DEBUG:teuthology.orchestra.run.smithi173:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-03-23T14:56:33.817 DEBUG:teuthology.orchestra.run.smithi173:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-03-23T14:56:33.871 DEBUG:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-03-23T14:56:33.967 DEBUG:teuthology.orchestra.run.smithi173:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-03-23T14:56:33.977 DEBUG:teuthology.orchestra.run.smithi164:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-03-23T14:56:33.979 DEBUG:teuthology.orchestra.run.smithi173:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-03-23T14:56:34.023 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-03-23T14:56:34.024 INFO:tasks.ceph.mon.a:Restarting daemon 2024-03-23T14:56:34.024 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-03-23T14:56:34.026 INFO:tasks.ceph.mon.a:Started 2024-03-23T14:56:34.026 INFO:tasks.ceph.mon.b:Restarting daemon 2024-03-23T14:56:34.027 DEBUG:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-03-23T14:56:34.069 INFO:tasks.ceph.mon.b:Started 2024-03-23T14:56:34.069 INFO:tasks.ceph.mon.c:Restarting daemon 2024-03-23T14:56:34.069 DEBUG:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-03-23T14:56:34.071 INFO:tasks.ceph.mon.c:Started 2024-03-23T14:56:34.071 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-03-23T14:56:34.072 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-03-23T14:56:34.072 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-03-23T14:56:34.074 INFO:tasks.ceph.mgr.y:Started 2024-03-23T14:56:34.074 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-03-23T14:56:34.074 DEBUG:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-03-23T14:56:34.076 INFO:tasks.ceph.mgr.x:Started 2024-03-23T14:56:34.076 INFO:tasks.ceph:Setting crush tunables to default 2024-03-23T14:56:34.076 DEBUG:teuthology.orchestra.run.smithi164:> sudo ceph --cluster ceph osd crush tunables default 2024-03-23T14:56:34.193 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:34.190+0000 7f973a020640 1 Processor -- start 2024-03-23T14:56:34.193 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:34.190+0000 7f973a020640 1 -- start start 2024-03-23T14:56:34.195 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:34.190+0000 7f973a020640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f97340b56f0 0x7f973414a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:34.195 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:34.190+0000 7f973a020640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f9734141640 0x7f9734151c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:34.195 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:34.190+0000 7f973a020640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97340b4300 con 0x7f97340b4a70 2024-03-23T14:56:34.195 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:34.190+0000 7f973a020640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 -- 0x7f97340b5fa0 con 0x7f9734141640 2024-03-23T14:56:34.195 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:34.190+0000 7f973a020640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f97340b45e0 con 0x7f97340b56f0 2024-03-23T14:56:34.195 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:34.190+0000 7f9732ffd640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f97340b4a70 0x7f9734140f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:60848/0 (socket says 172.21.15.164:60848) 2024-03-23T14:56:34.195 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:34.190+0000 7f9732ffd640 1 -- 172.21.15.164:0/2709259426 learned_addr learned my addr 172.21.15.164:0/2709259426 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-03-23T14:56:34.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:34.190+0000 7f97337fe640 1 --2- 172.21.15.164:0/2709259426 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f97340b56f0 0x7f973414a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:34.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:34.190+0000 7f9733fff640 1 --2- 172.21.15.164:0/2709259426 >> v2:172.21.15.173:3300/0 conn(0x7f9734141640 0x7f9734151c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:34.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:34.190+0000 7f97337fe640 1 -- 172.21.15.164:0/2709259426 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f97340b56f0 msgr2=0x7f973414a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-23T14:56:34.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:34.190+0000 7f97337fe640 1 -- 172.21.15.164:0/2709259426 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f97340b56f0 msgr2=0x7f973414a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-23T14:56:34.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:34.190+0000 7f97337fe640 1 --2- 172.21.15.164:0/2709259426 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f97340b56f0 0x7f973414a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-23T14:56:34.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:34.190+0000 7f97337fe640 1 --2- 172.21.15.164:0/2709259426 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f97340b56f0 0x7f973414a8a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-23T14:56:34.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:34.190+0000 7f9733fff640 1 -- 172.21.15.164:0/2709259426 >> v2:172.21.15.173:3300/0 conn(0x7f9734141640 msgr2=0x7f9734151c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-23T14:56:34.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:34.190+0000 7f9733fff640 1 -- 172.21.15.164:0/2709259426 >> v2:172.21.15.173:3300/0 conn(0x7f9734141640 msgr2=0x7f9734151c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-23T14:56:34.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:34.190+0000 7f9733fff640 1 --2- 172.21.15.164:0/2709259426 >> v2:172.21.15.173:3300/0 conn(0x7f9734141640 0x7f9734151c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-23T14:56:34.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:34.190+0000 7f9733fff640 1 --2- 172.21.15.164:0/2709259426 >> v2:172.21.15.173:3300/0 conn(0x7f9734141640 0x7f9734151c30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-23T14:56:34.391 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:34.386+0000 7f97327fc640 1 -- 172.21.15.164:0/2709259426 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 757734372 0 0) 0x7f97340b4300 con 0x7f97340b4a70 2024-03-23T14:56:34.391 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:34.386+0000 7f97327fc640 1 -- 172.21.15.164:0/2709259426 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f971c003530 con 0x7f97340b4a70 2024-03-23T14:56:34.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:34.390+0000 7f97327fc640 1 -- 172.21.15.164:0/2709259426 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 34796297 0 0) 0x7f971c003530 con 0x7f97340b4a70 2024-03-23T14:56:34.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:34.390+0000 7f97327fc640 1 -- 172.21.15.164:0/2709259426 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f97340b56f0 msgr2=0x7f973414a8a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-23T14:56:34.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:34.390+0000 7f97327fc640 1 --2- 172.21.15.164:0/2709259426 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f97340b56f0 0x7f973414a8a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:34.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:34.390+0000 7f97327fc640 1 -- 172.21.15.164:0/2709259426 >> v2:172.21.15.173:3300/0 conn(0x7f9734141640 msgr2=0x7f9734151c30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-23T14:56:34.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:34.390+0000 7f97327fc640 1 --2- 172.21.15.164:0/2709259426 >> v2:172.21.15.173:3300/0 conn(0x7f9734141640 0x7f9734151c30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:34.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:34.390+0000 7f97327fc640 1 -- 172.21.15.164:0/2709259426 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f973414ade0 con 0x7f97340b4a70 2024-03-23T14:56:34.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:34.390+0000 7f97327fc640 1 -- 172.21.15.164:0/2709259426 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2526390548 0 0) 0x7f9728001680 con 0x7f97340b4a70 2024-03-23T14:56:34.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:34.390+0000 7f97327fc640 1 -- 172.21.15.164:0/2709259426 >> v1:172.21.15.164:6789/0 conn(0x7f97340b4a70 legacy=0x7f9734140f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:56:34.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:34.390+0000 7f97327fc640 1 --2- 172.21.15.164:0/2709259426 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f971c003980 0x7f971c003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:34.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:34.390+0000 7f97327fc640 1 --2- 172.21.15.164:0/2709259426 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f97340b56f0 0x7f971c0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:34.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:34.390+0000 7f97327fc640 1 --2- 172.21.15.164:0/2709259426 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f971c004d10 0x7f971c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:34.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:34.390+0000 7f97327fc640 1 -- 172.21.15.164:0/2709259426 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f97340b45e0 con 0x7f971c004d10 2024-03-23T14:56:34.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:34.390+0000 7f97327fc640 1 -- 172.21.15.164:0/2709259426 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f971c00ba40 con 0x7f971c003980 2024-03-23T14:56:34.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:34.390+0000 7f97327fc640 1 -- 172.21.15.164:0/2709259426 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f971c00bb80 con 0x7f97340b56f0 2024-03-23T14:56:34.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:34.390+0000 7f9733fff640 1 --2- 172.21.15.164:0/2709259426 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f971c004d10 0x7f971c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:34.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:34.390+0000 7f9732ffd640 1 --2- 172.21.15.164:0/2709259426 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f97340b56f0 0x7f971c0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:34.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:34.390+0000 7f97337fe640 1 --2- 172.21.15.164:0/2709259426 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f971c003980 0x7f971c003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:34.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:34.390+0000 7f9733fff640 1 -- 172.21.15.164:0/2709259426 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f97340b56f0 msgr2=0x7f971c0047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:56:34.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:34.390+0000 7f9733fff640 1 --2- 172.21.15.164:0/2709259426 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f97340b56f0 0x7f971c0047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:34.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:34.390+0000 7f9733fff640 1 -- 172.21.15.164:0/2709259426 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f971c003980 msgr2=0x7f971c003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:56:34.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:34.390+0000 7f9733fff640 1 --2- 172.21.15.164:0/2709259426 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f971c003980 0x7f971c003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:34.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:34.390+0000 7f9733fff640 1 -- 172.21.15.164:0/2709259426 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f971c003530 con 0x7f971c004d10 2024-03-23T14:56:34.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:34.394+0000 7f9733fff640 1 --2- 172.21.15.164:0/2709259426 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f971c004d10 0x7f971c005190 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f972c007330 tx=0x7f972c01ce40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=812320b9cde8b220 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:56:34.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:34.394+0000 7f97327fc640 1 -- 172.21.15.164:0/2709259426 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f972c0a5d40 con 0x7f971c004d10 2024-03-23T14:56:34.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:34.394+0000 7f97327fc640 1 -- 172.21.15.164:0/2709259426 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f972c0ad440 con 0x7f971c004d10 2024-03-23T14:56:34.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:34.394+0000 7f97327fc640 1 -- 172.21.15.164:0/2709259426 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f972c0a5ea0 con 0x7f971c004d10 2024-03-23T14:56:34.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:34.398+0000 7f973a020640 1 -- 172.21.15.164:0/2709259426 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f971c004d10 msgr2=0x7f971c005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:56:34.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:34.398+0000 7f973a020640 1 --2- 172.21.15.164:0/2709259426 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f971c004d10 0x7f971c005190 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f972c007330 tx=0x7f972c01ce40 comp rx=0 tx=0).stop 2024-03-23T14:56:34.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:34.398+0000 7f9733fff640 1 -- 172.21.15.164:0/2709259426 reap_dead start 2024-03-23T14:56:34.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:34.398+0000 7f973a020640 1 -- 172.21.15.164:0/2709259426 shutdown_connections 2024-03-23T14:56:34.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:34.398+0000 7f973a020640 1 -- 172.21.15.164:0/2709259426 >> 172.21.15.164:0/2709259426 conn(0x7f973402c3c0 msgr2=0x7f973402c7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:56:34.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:34.398+0000 7f973a020640 1 -- 172.21.15.164:0/2709259426 shutdown_connections 2024-03-23T14:56:34.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:34.398+0000 7f973a020640 1 -- 172.21.15.164:0/2709259426 wait complete. 2024-03-23T14:56:34.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:34.398+0000 7f973a020640 1 Processor -- start 2024-03-23T14:56:34.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:34.398+0000 7f973a020640 1 -- start start 2024-03-23T14:56:34.404 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:34.398+0000 7f973a020640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f973414a3c0 0x7f97341df780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:34.404 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:34.398+0000 7f973a020640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f97341477c0 0x7f9734141ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:34.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:34.398+0000 7f973a020640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f97341448c0 0x7f97341420e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:34.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:34.398+0000 7f973a020640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9734142ea0 con 0x7f973414a3c0 2024-03-23T14:56:34.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:34.398+0000 7f973a020640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f9734144430 con 0x7f97341477c0 2024-03-23T14:56:34.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:34.398+0000 7f973a020640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f973414a7a0 con 0x7f97341448c0 2024-03-23T14:56:34.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:34.398+0000 7f97337fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f973414a3c0 0x7f97341df780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:34.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:34.398+0000 7f97337fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f973414a3c0 0x7f97341df780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:37528/0 (socket says 172.21.15.164:37528) 2024-03-23T14:56:34.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:34.398+0000 7f97337fe640 1 -- 172.21.15.164:0/4255131774 learned_addr learned my addr 172.21.15.164:0/4255131774 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:56:34.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:34.398+0000 7f9732ffd640 1 --2- 172.21.15.164:0/4255131774 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f97341477c0 0x7f9734141ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:34.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:34.402+0000 7f9733fff640 1 --2- 172.21.15.164:0/4255131774 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f97341448c0 0x7f97341420e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:34.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:34.402+0000 7f97337fe640 1 -- 172.21.15.164:0/4255131774 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f97341448c0 msgr2=0x7f97341420e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:56:34.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:34.402+0000 7f97337fe640 1 --2- 172.21.15.164:0/4255131774 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f97341448c0 0x7f97341420e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:34.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:34.402+0000 7f97337fe640 1 -- 172.21.15.164:0/4255131774 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f97341477c0 msgr2=0x7f9734141ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:56:34.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:34.402+0000 7f97337fe640 1 --2- 172.21.15.164:0/4255131774 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f97341477c0 0x7f9734141ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:34.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:34.402+0000 7f97337fe640 1 -- 172.21.15.164:0/4255131774 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9734143b00 con 0x7f973414a3c0 2024-03-23T14:56:34.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:34.402+0000 7f97337fe640 1 --2- 172.21.15.164:0/4255131774 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f973414a3c0 0x7f97341df780 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f97240114e0 tx=0x7f97240118e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:56:34.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:34.402+0000 7f971bfff640 1 -- 172.21.15.164:0/4255131774 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9724015890 con 0x7f973414a3c0 2024-03-23T14:56:34.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:34.402+0000 7f971bfff640 1 -- 172.21.15.164:0/4255131774 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97240159f0 con 0x7f973414a3c0 2024-03-23T14:56:34.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:34.402+0000 7f971bfff640 1 -- 172.21.15.164:0/4255131774 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9724015da0 con 0x7f973414a3c0 2024-03-23T14:56:34.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:34.402+0000 7f973a020640 1 -- 172.21.15.164:0/4255131774 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97341440c0 con 0x7f973414a3c0 2024-03-23T14:56:34.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:34.402+0000 7f973a020640 1 -- 172.21.15.164:0/4255131774 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9734147d10 con 0x7f973414a3c0 2024-03-23T14:56:34.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:34.402+0000 7f971bfff640 1 -- 172.21.15.164:0/4255131774 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f972401e450 con 0x7f973414a3c0 2024-03-23T14:56:34.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:34.402+0000 7f971bfff640 1 -- 172.21.15.164:0/4255131774 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f9724013b90 con 0x7f973414a3c0 2024-03-23T14:56:34.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:34.402+0000 7f973a020640 1 -- 172.21.15.164:0/4255131774 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97340690a0 con 0x7f973414a3c0 2024-03-23T14:56:34.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:34.406+0000 7f971bfff640 1 -- 172.21.15.164:0/4255131774 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f9724023060 con 0x7f973414a3c0 2024-03-23T14:56:34.474 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:34.470+0000 7f973a020640 1 -- 172.21.15.164:0/4255131774 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f973406b7b0 con 0x7f973414a3c0 2024-03-23T14:56:34.476 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:34.474+0000 7f971bfff640 1 -- 172.21.15.164:0/4255131774 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7f972402d090 con 0x7f973414a3c0 2024-03-23T14:56:34.476 INFO:teuthology.orchestra.run.smithi164.stderr:adjusted tunables profile to default 2024-03-23T14:56:34.477 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:34.474+0000 7f973a020640 1 -- 172.21.15.164:0/4255131774 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f973414a3c0 msgr2=0x7f97341df780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:56:34.478 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:34.474+0000 7f973a020640 1 --2- 172.21.15.164:0/4255131774 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f973414a3c0 0x7f97341df780 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f97240114e0 tx=0x7f97240118e0 comp rx=0 tx=0).stop 2024-03-23T14:56:34.478 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:34.474+0000 7f973a020640 1 -- 172.21.15.164:0/4255131774 shutdown_connections 2024-03-23T14:56:34.478 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:34.474+0000 7f973a020640 1 --2- 172.21.15.164:0/4255131774 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f97341448c0 0x7f97341420e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:34.478 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:34.474+0000 7f973a020640 1 --2- 172.21.15.164:0/4255131774 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f97341477c0 0x7f9734141ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:34.478 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:34.474+0000 7f973a020640 1 --2- 172.21.15.164:0/4255131774 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f973414a3c0 0x7f97341df780 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:34.478 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:34.474+0000 7f973a020640 1 -- 172.21.15.164:0/4255131774 >> 172.21.15.164:0/4255131774 conn(0x7f973402c3c0 msgr2=0x7f973414b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:56:34.478 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:34.474+0000 7f973a020640 1 -- 172.21.15.164:0/4255131774 shutdown_connections 2024-03-23T14:56:34.478 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:34.474+0000 7f973a020640 1 -- 172.21.15.164:0/4255131774 wait complete. 2024-03-23T14:56:34.495 INFO:tasks.ceph:check_enable_crimson: False 2024-03-23T14:56:34.495 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-03-23T14:56:34.496 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-23T14:56:34.496 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-03-23T14:56:34.508 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-23T14:56:34.508 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-03-23T14:56:34.564 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-23T14:56:34.564 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-03-23T14:56:34.619 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-23T14:56:34.619 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-03-23T14:56:34.672 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-03-23T14:56:34.672 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-03-23T14:56:34.678 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-03-23T14:56:34.673+0000 7f3fc8b03180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-23T14:56:34.686 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-03-23T14:56:34.686 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-03-23T14:56:34.686 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-03-23T14:56:34.682+0000 7efcb3f00180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-23T14:56:34.739 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-03-23T14:56:34.740 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-03-23T14:56:34.795 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-03-23T14:56:34.795 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-03-23T14:56:34.847 DEBUG:teuthology.orchestra.run.smithi173:> sudo ceph --cluster ceph osd new 80abdc62-bd0d-4e60-a554-7ed6fd67cf54 0 2024-03-23T14:56:35.011 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.005+0000 7fcb6e428640 1 Processor -- start 2024-03-23T14:56:35.011 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.005+0000 7fcb6e428640 1 -- start start 2024-03-23T14:56:35.013 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.009+0000 7fcb6e428640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcb680b56f0 0x7fcb6814a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:35.013 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.009+0000 7fcb6e428640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fcb68141640 0x7fcb68151c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:35.013 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.009+0000 7fcb6e428640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb680b4300 con 0x7fcb680b4a70 2024-03-23T14:56:35.013 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.009+0000 7fcb6e428640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fcb680b5fa0 con 0x7fcb68141640 2024-03-23T14:56:35.013 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.009+0000 7fcb6e428640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fcb680b45e0 con 0x7fcb680b56f0 2024-03-23T14:56:35.013 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.009+0000 7fcb67fff640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcb680b56f0 0x7fcb6814a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:35.013 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.009+0000 7fcb67fff640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcb680b56f0 0x7fcb6814a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.173:59358/0 (socket says 172.21.15.173:59358) 2024-03-23T14:56:35.013 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.009+0000 7fcb67fff640 1 -- 172.21.15.173:0/884281161 learned_addr learned my addr 172.21.15.173:0/884281161 (peer_addr_for_me v2:172.21.15.173:0/0) 2024-03-23T14:56:35.013 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.009+0000 7fcb6c9c5640 1 --2- 172.21.15.173:0/884281161 >> v2:172.21.15.164:3300/0 conn(0x7fcb68141640 0x7fcb68151c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:35.013 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.009+0000 7fcb67fff640 1 -- 172.21.15.173:0/884281161 >> v2:172.21.15.164:3300/0 conn(0x7fcb68141640 msgr2=0x7fcb68151c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:56:35.014 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.009+0000 7fcb67fff640 1 --2- 172.21.15.173:0/884281161 >> v2:172.21.15.164:3300/0 conn(0x7fcb68141640 0x7fcb68151c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:35.014 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.009+0000 7fcb67fff640 1 -- 172.21.15.173:0/884281161 >> v1:172.21.15.164:6789/0 conn(0x7fcb680b4a70 legacy=0x7fcb68140f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:56:35.014 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.009+0000 7fcb67fff640 1 -- 172.21.15.173:0/884281161 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb6814ade0 con 0x7fcb680b56f0 2024-03-23T14:56:35.015 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.009+0000 7fcb67fff640 1 --2- 172.21.15.173:0/884281161 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcb680b56f0 0x7fcb6814a8a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c009870 tx=0x7fcb5c01c4f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=525faffc46b86f95 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:56:35.015 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.009+0000 7fcb66ffd640 1 -- 172.21.15.173:0/884281161 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb5c0aa610 con 0x7fcb680b56f0 2024-03-23T14:56:35.015 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.009+0000 7fcb66ffd640 1 -- 172.21.15.173:0/884281161 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb5c0a6040 con 0x7fcb680b56f0 2024-03-23T14:56:35.015 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.009+0000 7fcb66ffd640 1 -- 172.21.15.173:0/884281161 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb5c0aa990 con 0x7fcb680b56f0 2024-03-23T14:56:35.016 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.009+0000 7fcb6e428640 1 -- 172.21.15.173:0/884281161 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcb680b56f0 msgr2=0x7fcb6814a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:56:35.016 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.009+0000 7fcb6e428640 1 --2- 172.21.15.173:0/884281161 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcb680b56f0 0x7fcb6814a8a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c009870 tx=0x7fcb5c01c4f0 comp rx=0 tx=0).stop 2024-03-23T14:56:35.016 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.009+0000 7fcb6e428640 1 -- 172.21.15.173:0/884281161 shutdown_connections 2024-03-23T14:56:35.016 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.009+0000 7fcb6e428640 1 --2- 172.21.15.173:0/884281161 >> v2:172.21.15.164:3300/0 conn(0x7fcb68141640 0x7fcb68151c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:35.016 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.009+0000 7fcb6e428640 1 --2- 172.21.15.173:0/884281161 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcb680b56f0 0x7fcb6814a8a0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:35.017 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.009+0000 7fcb6e428640 1 -- 172.21.15.173:0/884281161 >> 172.21.15.173:0/884281161 conn(0x7fcb6802c3c0 msgr2=0x7fcb6802c7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:56:35.017 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.009+0000 7fcb6e428640 1 -- 172.21.15.173:0/884281161 shutdown_connections 2024-03-23T14:56:35.017 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.009+0000 7fcb6e428640 1 -- 172.21.15.173:0/884281161 wait complete. 2024-03-23T14:56:35.019 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.013+0000 7fcb6e428640 1 Processor -- start 2024-03-23T14:56:35.019 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.013+0000 7fcb6e428640 1 -- start start 2024-03-23T14:56:35.019 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.013+0000 7fcb6e428640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcb680b4a70 0x7fcb68137450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:35.019 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.013+0000 7fcb6e428640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcb680b56f0 0x7fcb68139040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:35.019 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.013+0000 7fcb6e428640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcb68141640 0x7fcb68149620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:35.019 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.013+0000 7fcb6e428640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fcb68136000 con 0x7fcb680b4a70 2024-03-23T14:56:35.020 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.013+0000 7fcb6e428640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fcb6814a8d0 con 0x7fcb68141640 2024-03-23T14:56:35.020 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.013+0000 7fcb6e428640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fcb6814a030 con 0x7fcb680b56f0 2024-03-23T14:56:35.020 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.013+0000 7fcb677fe640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcb680b56f0 0x7fcb68139040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:35.020 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.013+0000 7fcb6c9c5640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcb68141640 0x7fcb68149620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:35.020 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.013+0000 7fcb6c9c5640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcb68141640 0x7fcb68149620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.173:38562/0 (socket says 172.21.15.173:38562) 2024-03-23T14:56:35.020 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.013+0000 7fcb677fe640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcb680b56f0 0x7fcb68139040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.173:59374/0 (socket says 172.21.15.173:59374) 2024-03-23T14:56:35.020 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.013+0000 7fcb6c9c5640 1 -- 172.21.15.173:0/26521068 learned_addr learned my addr 172.21.15.173:0/26521068 (peer_addr_for_me v2:172.21.15.173:0/0) 2024-03-23T14:56:35.020 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.013+0000 7fcb67fff640 1 --2- 172.21.15.173:0/26521068 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcb680b4a70 0x7fcb68137450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:35.020 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.013+0000 7fcb6c9c5640 1 -- 172.21.15.173:0/26521068 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcb680b56f0 msgr2=0x7fcb68139040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:56:35.020 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.013+0000 7fcb6c9c5640 1 --2- 172.21.15.173:0/26521068 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcb680b56f0 0x7fcb68139040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:35.020 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.013+0000 7fcb6c9c5640 1 -- 172.21.15.173:0/26521068 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcb680b4a70 msgr2=0x7fcb68137450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:56:35.020 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.013+0000 7fcb6c9c5640 1 --2- 172.21.15.173:0/26521068 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcb680b4a70 0x7fcb68137450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:35.020 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.013+0000 7fcb6c9c5640 1 -- 172.21.15.173:0/26521068 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb5c0094d0 con 0x7fcb68141640 2024-03-23T14:56:35.020 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.013+0000 7fcb677fe640 1 --2- 172.21.15.173:0/26521068 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcb680b56f0 0x7fcb68139040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T14:56:35.020 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.013+0000 7fcb6c9c5640 1 --2- 172.21.15.173:0/26521068 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcb68141640 0x7fcb68149620 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fcb60007e60 tx=0x7fcb6000e4b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:56:35.020 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.013+0000 7fcb64ff9640 1 -- 172.21.15.173:0/26521068 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb60014420 con 0x7fcb68141640 2024-03-23T14:56:35.021 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.013+0000 7fcb64ff9640 1 -- 172.21.15.173:0/26521068 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb600145a0 con 0x7fcb68141640 2024-03-23T14:56:35.021 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.013+0000 7fcb64ff9640 1 -- 172.21.15.173:0/26521068 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb60014950 con 0x7fcb68141640 2024-03-23T14:56:35.021 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.013+0000 7fcb6e428640 1 -- 172.21.15.173:0/26521068 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb6814a310 con 0x7fcb68141640 2024-03-23T14:56:35.021 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.013+0000 7fcb6e428640 1 -- 172.21.15.173:0/26521068 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb681e2ff0 con 0x7fcb68141640 2024-03-23T14:56:35.021 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.013+0000 7fcb64ff9640 1 -- 172.21.15.173:0/26521068 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fcb60014ab0 con 0x7fcb68141640 2024-03-23T14:56:35.021 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.013+0000 7fcb64ff9640 1 -- 172.21.15.173:0/26521068 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7fcb6001d3c0 con 0x7fcb68141640 2024-03-23T14:56:35.021 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.013+0000 7fcb6e428640 1 -- 172.21.15.173:0/26521068 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb68149340 con 0x7fcb68141640 2024-03-23T14:56:35.032 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.029+0000 7fcb64ff9640 1 -- 172.21.15.173:0/26521068 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fcb68149340 con 0x7fcb68141640 2024-03-23T14:56:35.101 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.097+0000 7fcb6e428640 1 -- 172.21.15.173:0/26521068 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "osd new", "uuid": "80abdc62-bd0d-4e60-a554-7ed6fd67cf54", "id": 0} v 0) -- 0x7fcb680690a0 con 0x7fcb68141640 2024-03-23T14:56:35.108 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.105+0000 7fcb64ff9640 1 -- 172.21.15.173:0/26521068 <== mon.1 v2:172.21.15.173:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "80abdc62-bd0d-4e60-a554-7ed6fd67cf54", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7fcb6002d0b0 con 0x7fcb68141640 2024-03-23T14:56:35.108 INFO:teuthology.orchestra.run.smithi173.stdout:0 2024-03-23T14:56:35.110 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.105+0000 7fcb6e428640 1 -- 172.21.15.173:0/26521068 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcb68141640 msgr2=0x7fcb68149620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:56:35.110 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.105+0000 7fcb6e428640 1 --2- 172.21.15.173:0/26521068 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcb68141640 0x7fcb68149620 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fcb60007e60 tx=0x7fcb6000e4b0 comp rx=0 tx=0).stop 2024-03-23T14:56:35.110 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.105+0000 7fcb6e428640 1 -- 172.21.15.173:0/26521068 shutdown_connections 2024-03-23T14:56:35.110 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.105+0000 7fcb6e428640 1 --2- 172.21.15.173:0/26521068 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcb68141640 0x7fcb68149620 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:35.110 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.105+0000 7fcb6e428640 1 --2- 172.21.15.173:0/26521068 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcb680b56f0 0x7fcb68139040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:35.110 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.105+0000 7fcb6e428640 1 --2- 172.21.15.173:0/26521068 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcb680b4a70 0x7fcb68137450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:35.110 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.105+0000 7fcb6e428640 1 -- 172.21.15.173:0/26521068 >> 172.21.15.173:0/26521068 conn(0x7fcb6802c3c0 msgr2=0x7fcb68139bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:56:35.110 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.105+0000 7fcb6e428640 1 -- 172.21.15.173:0/26521068 shutdown_connections 2024-03-23T14:56:35.110 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.105+0000 7fcb6e428640 1 -- 172.21.15.173:0/26521068 wait complete. 2024-03-23T14:56:35.126 DEBUG:teuthology.orchestra.run.smithi173:> sudo ceph --cluster ceph osd new f9001b06-be4d-4f42-a0e6-bcef2afa0d6d 1 2024-03-23T14:56:35.174 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-03-23T14:56:35.170+0000 7efcb3f00180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-23T14:56:35.206 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-03-23T14:56:35.201+0000 7f3fc8b03180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-23T14:56:35.243 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.237+0000 7f8313b89640 1 Processor -- start 2024-03-23T14:56:35.244 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.237+0000 7f8313b89640 1 -- start start 2024-03-23T14:56:35.245 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.241+0000 7f8313b89640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f830c0725b0 0x7f830c148890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:35.245 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.241+0000 7f8313b89640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f830c06e730 0x7f830c0b3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:35.245 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.241+0000 7f8313b89640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f830c06d770 0x7f830c0b37c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:35.245 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.241+0000 7f8313b89640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f830c0b0ca0 con 0x7f830c06d770 2024-03-23T14:56:35.245 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.241+0000 7f8313b89640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 -- 0x7f830c072990 con 0x7f830c0725b0 2024-03-23T14:56:35.246 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.241+0000 7f8313b89640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f830c0b95f0 con 0x7f830c06e730 2024-03-23T14:56:35.246 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.241+0000 7f8311925640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f830c0725b0 0x7f830c148890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:35.246 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.241+0000 7f8311925640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f830c0725b0 0x7f830c148890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.173:38564/0 (socket says 172.21.15.173:38564) 2024-03-23T14:56:35.246 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.241+0000 7f8311925640 1 -- 172.21.15.173:0/4269967856 learned_addr learned my addr 172.21.15.173:0/4269967856 (peer_addr_for_me v2:172.21.15.173:0/0) 2024-03-23T14:56:35.246 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.241+0000 7f8311124640 1 --2- 172.21.15.173:0/4269967856 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f830c06e730 0x7f830c0b3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:35.246 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.241+0000 7f8312126640 1 --2- 172.21.15.173:0/4269967856 >> v2:172.21.15.164:3300/0 conn(0x7f830c06d770 0x7f830c0b37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:35.246 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.241+0000 7f8311925640 1 -- 172.21.15.173:0/4269967856 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f830c06e730 msgr2=0x7f830c0b3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:56:35.246 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.241+0000 7f8311925640 1 --2- 172.21.15.173:0/4269967856 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f830c06e730 0x7f830c0b3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:35.246 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.241+0000 7f8311925640 1 -- 172.21.15.173:0/4269967856 >> v2:172.21.15.164:3300/0 conn(0x7f830c06d770 msgr2=0x7f830c0b37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:56:35.246 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.241+0000 7f8311925640 1 --2- 172.21.15.173:0/4269967856 >> v2:172.21.15.164:3300/0 conn(0x7f830c06d770 0x7f830c0b37c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:35.246 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.241+0000 7f8311925640 1 -- 172.21.15.173:0/4269967856 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f830c148dd0 con 0x7f830c0725b0 2024-03-23T14:56:35.246 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.241+0000 7f8311124640 1 --2- 172.21.15.173:0/4269967856 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f830c06e730 0x7f830c0b3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T14:56:35.246 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.241+0000 7f8312126640 1 --2- 172.21.15.173:0/4269967856 >> v2:172.21.15.164:3300/0 conn(0x7f830c06d770 0x7f830c0b37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T14:56:35.247 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.241+0000 7f8311925640 1 --2- 172.21.15.173:0/4269967856 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f830c0725b0 0x7f830c148890 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f830c06cc00 tx=0x7f830401c960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d583f3773dbdf28c server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:56:35.247 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.241+0000 7f8310923640 1 -- 172.21.15.173:0/4269967856 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83040a56f0 con 0x7f830c0725b0 2024-03-23T14:56:35.247 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.241+0000 7f8310923640 1 -- 172.21.15.173:0/4269967856 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83040a5850 con 0x7f830c0725b0 2024-03-23T14:56:35.247 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.241+0000 7f8310923640 1 -- 172.21.15.173:0/4269967856 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83040a5be0 con 0x7f830c0725b0 2024-03-23T14:56:35.248 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.241+0000 7f8313b89640 1 -- 172.21.15.173:0/4269967856 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f830c0725b0 msgr2=0x7f830c148890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:56:35.248 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.241+0000 7f8313b89640 1 --2- 172.21.15.173:0/4269967856 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f830c0725b0 0x7f830c148890 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f830c06cc00 tx=0x7f830401c960 comp rx=0 tx=0).stop 2024-03-23T14:56:35.248 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.241+0000 7f8313b89640 1 -- 172.21.15.173:0/4269967856 shutdown_connections 2024-03-23T14:56:35.248 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.241+0000 7f8313b89640 1 --2- 172.21.15.173:0/4269967856 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f830c06e730 0x7f830c0b3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:35.249 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.241+0000 7f8313b89640 1 --2- 172.21.15.173:0/4269967856 >> v2:172.21.15.164:3300/0 conn(0x7f830c06d770 0x7f830c0b37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:35.249 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.241+0000 7f8313b89640 1 --2- 172.21.15.173:0/4269967856 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f830c0725b0 0x7f830c148890 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:35.249 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.241+0000 7f8313b89640 1 -- 172.21.15.173:0/4269967856 >> 172.21.15.173:0/4269967856 conn(0x7f830c0b7970 msgr2=0x7f830c0b7d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:56:35.249 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.245+0000 7f8313b89640 1 -- 172.21.15.173:0/4269967856 shutdown_connections 2024-03-23T14:56:35.249 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.245+0000 7f8313b89640 1 -- 172.21.15.173:0/4269967856 wait complete. 2024-03-23T14:56:35.251 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.245+0000 7f8313b89640 1 Processor -- start 2024-03-23T14:56:35.251 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.245+0000 7f8313b89640 1 -- start start 2024-03-23T14:56:35.251 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.245+0000 7f8313b89640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f830c06d770 0x7f830c1466e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:35.251 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.245+0000 7f8313b89640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f830c06e730 0x7f830c138c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:35.251 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.245+0000 7f8313b89640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f830c0725b0 0x7f830c1391c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:35.252 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.245+0000 7f8313b89640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f830c147530 con 0x7f830c06e730 2024-03-23T14:56:35.252 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.245+0000 7f8313b89640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f830c1481f0 con 0x7f830c0725b0 2024-03-23T14:56:35.252 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.245+0000 7f8313b89640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f830c148840 con 0x7f830c06d770 2024-03-23T14:56:35.252 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.245+0000 7f8311925640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f830c06d770 0x7f830c1466e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:35.252 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.245+0000 7f8312126640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f830c0725b0 0x7f830c1391c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:35.252 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.245+0000 7f8311925640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f830c06d770 0x7f830c1466e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.173:59392/0 (socket says 172.21.15.173:59392) 2024-03-23T14:56:35.252 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.245+0000 7f8311925640 1 -- 172.21.15.173:0/3329614705 learned_addr learned my addr 172.21.15.173:0/3329614705 (peer_addr_for_me v2:172.21.15.173:0/0) 2024-03-23T14:56:35.252 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.245+0000 7f8311124640 1 --2- 172.21.15.173:0/3329614705 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f830c06e730 0x7f830c138c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:35.252 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.245+0000 7f8311925640 1 -- 172.21.15.173:0/3329614705 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f830c0725b0 msgr2=0x7f830c1391c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:56:35.252 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.245+0000 7f8311925640 1 --2- 172.21.15.173:0/3329614705 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f830c0725b0 0x7f830c1391c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:35.252 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.245+0000 7f8311925640 1 -- 172.21.15.173:0/3329614705 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f830c06e730 msgr2=0x7f830c138c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:56:35.252 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.245+0000 7f8311925640 1 --2- 172.21.15.173:0/3329614705 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f830c06e730 0x7f830c138c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:35.252 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.245+0000 7f8311925640 1 -- 172.21.15.173:0/3329614705 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82fc009420 con 0x7f830c06d770 2024-03-23T14:56:35.252 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.245+0000 7f8312126640 1 --2- 172.21.15.173:0/3329614705 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f830c0725b0 0x7f830c1391c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T14:56:35.252 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.245+0000 7f8311925640 1 --2- 172.21.15.173:0/3329614705 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f830c06d770 0x7f830c1466e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f83040af040 tx=0x7f8304019700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:56:35.253 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.245+0000 7f83027fc640 1 -- 172.21.15.173:0/3329614705 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83040b8670 con 0x7f830c06d770 2024-03-23T14:56:35.253 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.245+0000 7f83027fc640 1 -- 172.21.15.173:0/3329614705 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83040b87f0 con 0x7f830c06d770 2024-03-23T14:56:35.253 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.245+0000 7f8313b89640 1 -- 172.21.15.173:0/3329614705 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8304009b30 con 0x7f830c06d770 2024-03-23T14:56:35.253 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.245+0000 7f83027fc640 1 -- 172.21.15.173:0/3329614705 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83040b8ba0 con 0x7f830c06d770 2024-03-23T14:56:35.253 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.245+0000 7f8313b89640 1 -- 172.21.15.173:0/3329614705 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f830c13e240 con 0x7f830c06d770 2024-03-23T14:56:35.253 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.245+0000 7f83027fc640 1 -- 172.21.15.173:0/3329614705 <== mon.2 v2:172.21.15.173:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f8304003d30 con 0x7f830c06d770 2024-03-23T14:56:35.253 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.245+0000 7f83027fc640 1 -- 172.21.15.173:0/3329614705 <== mon.2 v2:172.21.15.173:3301/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7f83040c17c0 con 0x7f830c06d770 2024-03-23T14:56:35.253 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.249+0000 7f8313b89640 1 -- 172.21.15.173:0/3329614705 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f830c0690a0 con 0x7f830c06d770 2024-03-23T14:56:35.254 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.249+0000 7f83027fc640 1 -- 172.21.15.173:0/3329614705 <== mon.2 v2:172.21.15.173:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f83040bc020 con 0x7f830c06d770 2024-03-23T14:56:35.324 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.317+0000 7f8313b89640 1 -- 172.21.15.173:0/3329614705 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "osd new", "uuid": "f9001b06-be4d-4f42-a0e6-bcef2afa0d6d", "id": 1} v 0) -- 0x7f830c13ed60 con 0x7f830c06d770 2024-03-23T14:56:35.333 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.329+0000 7f83027fc640 1 -- 172.21.15.173:0/3329614705 <== mon.2 v2:172.21.15.173:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f9001b06-be4d-4f42-a0e6-bcef2afa0d6d", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7f8304004250 con 0x7f830c06d770 2024-03-23T14:56:35.333 INFO:teuthology.orchestra.run.smithi173.stdout:1 2024-03-23T14:56:35.335 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.329+0000 7f8313b89640 1 -- 172.21.15.173:0/3329614705 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f830c06d770 msgr2=0x7f830c1466e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:56:35.335 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.329+0000 7f8313b89640 1 --2- 172.21.15.173:0/3329614705 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f830c06d770 0x7f830c1466e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f83040af040 tx=0x7f8304019700 comp rx=0 tx=0).stop 2024-03-23T14:56:35.335 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.329+0000 7f8313b89640 1 -- 172.21.15.173:0/3329614705 shutdown_connections 2024-03-23T14:56:35.335 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.329+0000 7f8313b89640 1 --2- 172.21.15.173:0/3329614705 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f830c0725b0 0x7f830c1391c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:35.335 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.329+0000 7f8313b89640 1 --2- 172.21.15.173:0/3329614705 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f830c06e730 0x7f830c138c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:35.335 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.329+0000 7f8313b89640 1 --2- 172.21.15.173:0/3329614705 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f830c06d770 0x7f830c1466e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:35.335 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.329+0000 7f8313b89640 1 -- 172.21.15.173:0/3329614705 >> 172.21.15.173:0/3329614705 conn(0x7f830c0b7970 msgr2=0x7f830c0b5d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:56:35.335 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.329+0000 7f8313b89640 1 -- 172.21.15.173:0/3329614705 shutdown_connections 2024-03-23T14:56:35.335 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.329+0000 7f8313b89640 1 -- 172.21.15.173:0/3329614705 wait complete. 2024-03-23T14:56:35.350 DEBUG:teuthology.orchestra.run.smithi173:> sudo ceph --cluster ceph osd new fa0aeb9f-8b54-4831-a188-3e361c7980d7 2 2024-03-23T14:56:35.388 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-03-23T14:56:35.386+0000 7efcb3f00180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-23T14:56:35.428 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-03-23T14:56:35.425+0000 7f3fc8b03180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-23T14:56:35.467 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.461+0000 7fa127083640 1 Processor -- start 2024-03-23T14:56:35.467 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.461+0000 7fa127083640 1 -- start start 2024-03-23T14:56:35.468 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.461+0000 7fa127083640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fa1200b9ac0 0x7fa12013f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:35.468 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.461+0000 7fa127083640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa12014db90 0x7fa12014ff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:35.468 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.461+0000 7fa127083640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa120145240 con 0x7fa12006f8e0 2024-03-23T14:56:35.468 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.461+0000 7fa127083640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fa1200b3b50 con 0x7fa1200b9ac0 2024-03-23T14:56:35.468 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.461+0000 7fa127083640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fa120070050 con 0x7fa12014db90 2024-03-23T14:56:35.468 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.461+0000 7fa125620640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa12014db90 0x7fa12014ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:35.468 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.461+0000 7fa11ffff640 1 --1- >> v1:172.21.15.173:6789/0 conn(0x7fa12006f8e0 0x7fa1201459b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.173:6789/0 says I am v1:172.21.15.173:52828/0 (socket says 172.21.15.173:52828) 2024-03-23T14:56:35.468 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.465+0000 7fa11ffff640 1 -- 172.21.15.173:0/3648023245 learned_addr learned my addr 172.21.15.173:0/3648023245 (peer_addr_for_me v1:172.21.15.173:0/0) 2024-03-23T14:56:35.469 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.465+0000 7fa124e1f640 1 --2- 172.21.15.173:0/3648023245 >> v2:172.21.15.164:3300/0 conn(0x7fa1200b9ac0 0x7fa12013f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:35.469 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.465+0000 7fa125620640 1 -- 172.21.15.173:0/3648023245 >> v2:172.21.15.164:3300/0 conn(0x7fa1200b9ac0 msgr2=0x7fa12013f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:56:35.469 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.465+0000 7fa125620640 1 --2- 172.21.15.173:0/3648023245 >> v2:172.21.15.164:3300/0 conn(0x7fa1200b9ac0 0x7fa12013f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:35.469 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.465+0000 7fa125620640 1 -- 172.21.15.173:0/3648023245 >> v1:172.21.15.173:6789/0 conn(0x7fa12006f8e0 legacy=0x7fa1201459b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:56:35.469 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.465+0000 7fa11f7fe640 1 -- 172.21.15.173:0/3648023245 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3847023588 0 0) 0x7fa120145240 con 0x7fa12006f8e0 2024-03-23T14:56:35.469 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.465+0000 7fa125620640 1 -- 172.21.15.173:0/3648023245 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1200b9ea0 con 0x7fa12014db90 2024-03-23T14:56:35.470 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.465+0000 7fa125620640 1 --2- 172.21.15.173:0/3648023245 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa12014db90 0x7fa12014ff80 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa1080097c0 tx=0x7fa10801c350 comp rx=0 tx=0).ready entity=mon.2 client_cookie=348be92b48c36246 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:56:35.471 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.465+0000 7fa11f7fe640 1 -- 172.21.15.173:0/3648023245 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa108004230 con 0x7fa12014db90 2024-03-23T14:56:35.471 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.465+0000 7fa11f7fe640 1 -- 172.21.15.173:0/3648023245 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1080a7050 con 0x7fa12014db90 2024-03-23T14:56:35.471 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.465+0000 7fa11f7fe640 1 -- 172.21.15.173:0/3648023245 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa108003620 con 0x7fa12014db90 2024-03-23T14:56:35.471 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.465+0000 7fa127083640 1 -- 172.21.15.173:0/3648023245 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa12014db90 msgr2=0x7fa12014ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:56:35.471 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.465+0000 7fa127083640 1 --2- 172.21.15.173:0/3648023245 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa12014db90 0x7fa12014ff80 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa1080097c0 tx=0x7fa10801c350 comp rx=0 tx=0).stop 2024-03-23T14:56:35.472 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.465+0000 7fa127083640 1 -- 172.21.15.173:0/3648023245 shutdown_connections 2024-03-23T14:56:35.472 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.465+0000 7fa127083640 1 --2- 172.21.15.173:0/3648023245 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa12014db90 0x7fa12014ff80 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:35.472 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.465+0000 7fa127083640 1 --2- 172.21.15.173:0/3648023245 >> v2:172.21.15.164:3300/0 conn(0x7fa1200b9ac0 0x7fa12013f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:35.472 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.465+0000 7fa127083640 1 -- 172.21.15.173:0/3648023245 >> 172.21.15.173:0/3648023245 conn(0x7fa12006b930 msgr2=0x7fa1200b9150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:56:35.472 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.465+0000 7fa127083640 1 -- 172.21.15.173:0/3648023245 shutdown_connections 2024-03-23T14:56:35.472 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.465+0000 7fa127083640 1 -- 172.21.15.173:0/3648023245 wait complete. 2024-03-23T14:56:35.472 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.469+0000 7fa127083640 1 Processor -- start 2024-03-23T14:56:35.472 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.469+0000 7fa127083640 1 -- start start 2024-03-23T14:56:35.474 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.469+0000 7fa127083640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa1200b9ac0 0x7fa12013fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:35.474 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.469+0000 7fa127083640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa12014db90 0x7fa120072c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:35.474 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.469+0000 7fa127083640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa120140c30 0x7fa1201e0e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:35.474 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.469+0000 7fa127083640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa120140810 con 0x7fa1200b9ac0 2024-03-23T14:56:35.475 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.469+0000 7fa127083640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa120141410 con 0x7fa12014db90 2024-03-23T14:56:35.475 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.469+0000 7fa127083640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fa1201420e0 con 0x7fa120140c30 2024-03-23T14:56:35.475 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.469+0000 7fa125620640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa120140c30 0x7fa1201e0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:35.475 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.469+0000 7fa11ffff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa12014db90 0x7fa120072c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:35.475 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.469+0000 7fa125620640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa120140c30 0x7fa1201e0e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.173:59406/0 (socket says 172.21.15.173:59406) 2024-03-23T14:56:35.475 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.469+0000 7fa125620640 1 -- 172.21.15.173:0/4084481683 learned_addr learned my addr 172.21.15.173:0/4084481683 (peer_addr_for_me v2:172.21.15.173:0/0) 2024-03-23T14:56:35.475 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.469+0000 7fa124e1f640 1 --2- 172.21.15.173:0/4084481683 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa1200b9ac0 0x7fa12013fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:35.475 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.469+0000 7fa11ffff640 1 -- 172.21.15.173:0/4084481683 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa120140c30 msgr2=0x7fa1201e0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:56:35.475 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.469+0000 7fa11ffff640 1 --2- 172.21.15.173:0/4084481683 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa120140c30 0x7fa1201e0e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:35.475 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.469+0000 7fa11ffff640 1 -- 172.21.15.173:0/4084481683 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa1200b9ac0 msgr2=0x7fa12013fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:56:35.475 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.469+0000 7fa11ffff640 1 --2- 172.21.15.173:0/4084481683 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa1200b9ac0 0x7fa12013fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:35.475 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.469+0000 7fa11ffff640 1 -- 172.21.15.173:0/4084481683 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa108009420 con 0x7fa12014db90 2024-03-23T14:56:35.475 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.469+0000 7fa125620640 1 --2- 172.21.15.173:0/4084481683 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa120140c30 0x7fa1201e0e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T14:56:35.475 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.469+0000 7fa11ffff640 1 --2- 172.21.15.173:0/4084481683 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa12014db90 0x7fa120072c70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa11000c7e0 tx=0x7fa11000cca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:56:35.475 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.469+0000 7fa11d7fa640 1 -- 172.21.15.173:0/4084481683 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa11000a640 con 0x7fa12014db90 2024-03-23T14:56:35.475 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.469+0000 7fa11d7fa640 1 -- 172.21.15.173:0/4084481683 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa11000a7a0 con 0x7fa12014db90 2024-03-23T14:56:35.476 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.469+0000 7fa127083640 1 -- 172.21.15.173:0/4084481683 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa120140470 con 0x7fa12014db90 2024-03-23T14:56:35.476 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.469+0000 7fa11d7fa640 1 -- 172.21.15.173:0/4084481683 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa11000ab50 con 0x7fa12014db90 2024-03-23T14:56:35.476 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.469+0000 7fa127083640 1 -- 172.21.15.173:0/4084481683 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa120140130 con 0x7fa12014db90 2024-03-23T14:56:35.476 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.469+0000 7fa11d7fa640 1 -- 172.21.15.173:0/4084481683 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fa11000acb0 con 0x7fa12014db90 2024-03-23T14:56:35.476 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.469+0000 7fa11d7fa640 1 -- 172.21.15.173:0/4084481683 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7fa110012820 con 0x7fa12014db90 2024-03-23T14:56:35.476 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.469+0000 7fa127083640 1 -- 172.21.15.173:0/4084481683 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa120140950 con 0x7fa12014db90 2024-03-23T14:56:35.477 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.473+0000 7fa11d7fa640 1 -- 172.21.15.173:0/4084481683 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fa120140950 con 0x7fa12014db90 2024-03-23T14:56:35.546 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.541+0000 7fa127083640 1 -- 172.21.15.173:0/4084481683 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "osd new", "uuid": "fa0aeb9f-8b54-4831-a188-3e361c7980d7", "id": 2} v 0) -- 0x7fa12013ddc0 con 0x7fa12014db90 2024-03-23T14:56:35.554 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.549+0000 7fa11d7fa640 1 -- 172.21.15.173:0/4084481683 <== mon.1 v2:172.21.15.173:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "fa0aeb9f-8b54-4831-a188-3e361c7980d7", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7fa110018070 con 0x7fa12014db90 2024-03-23T14:56:35.554 INFO:teuthology.orchestra.run.smithi173.stdout:2 2024-03-23T14:56:35.555 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.549+0000 7fa127083640 1 -- 172.21.15.173:0/4084481683 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa12014db90 msgr2=0x7fa120072c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:56:35.555 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.549+0000 7fa127083640 1 --2- 172.21.15.173:0/4084481683 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa12014db90 0x7fa120072c70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa11000c7e0 tx=0x7fa11000cca0 comp rx=0 tx=0).stop 2024-03-23T14:56:35.555 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.549+0000 7fa127083640 1 -- 172.21.15.173:0/4084481683 shutdown_connections 2024-03-23T14:56:35.555 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.549+0000 7fa127083640 1 --2- 172.21.15.173:0/4084481683 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa120140c30 0x7fa1201e0e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:35.556 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.549+0000 7fa127083640 1 --2- 172.21.15.173:0/4084481683 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa12014db90 0x7fa120072c70 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:35.556 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.549+0000 7fa127083640 1 --2- 172.21.15.173:0/4084481683 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa1200b9ac0 0x7fa12013fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:35.556 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.549+0000 7fa127083640 1 -- 172.21.15.173:0/4084481683 >> 172.21.15.173:0/4084481683 conn(0x7fa12006b930 msgr2=0x7fa1200b3ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:56:35.556 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.549+0000 7fa127083640 1 -- 172.21.15.173:0/4084481683 shutdown_connections 2024-03-23T14:56:35.556 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.549+0000 7fa127083640 1 -- 172.21.15.173:0/4084481683 wait complete. 2024-03-23T14:56:35.571 DEBUG:teuthology.orchestra.run.smithi173:> sudo ceph --cluster ceph osd new 7e18c957-3091-46d5-8e17-8f3354c1726c 3 2024-03-23T14:56:35.690 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.685+0000 7f7dba280640 1 Processor -- start 2024-03-23T14:56:35.690 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.685+0000 7f7dba280640 1 -- start start 2024-03-23T14:56:35.692 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.685+0000 7f7dba280640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7db40b9ac0 0x7f7db40ae920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:35.692 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.685+0000 7f7dba280640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f7db40adc70 0x7f7db41445e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:35.692 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.685+0000 7f7dba280640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f7db4144b20 0x7f7db41477b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:35.692 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.685+0000 7f7dba280640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f7db40af400 con 0x7f7db40adc70 2024-03-23T14:56:35.692 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.685+0000 7f7dba280640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 -- 0x7f7db4144fd0 con 0x7f7db4144b20 2024-03-23T14:56:35.692 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.685+0000 7f7dba280640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f7db40b9ea0 con 0x7f7db40b9ac0 2024-03-23T14:56:35.693 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.685+0000 7f7db881d640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f7db4144b20 0x7f7db41477b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:35.693 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.685+0000 7f7db881d640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f7db4144b20 0x7f7db41477b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.173:38590/0 (socket says 172.21.15.173:38590) 2024-03-23T14:56:35.693 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.685+0000 7f7db881d640 1 -- 172.21.15.173:0/1372095746 learned_addr learned my addr 172.21.15.173:0/1372095746 (peer_addr_for_me v2:172.21.15.173:0/0) 2024-03-23T14:56:35.693 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.685+0000 7f7db3fff640 1 --2- 172.21.15.173:0/1372095746 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7db40b9ac0 0x7f7db40ae920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:35.693 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.685+0000 7f7db37fe640 1 --2- 172.21.15.173:0/1372095746 >> v2:172.21.15.164:3300/0 conn(0x7f7db40adc70 0x7f7db41445e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:35.693 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.685+0000 7f7db881d640 1 -- 172.21.15.173:0/1372095746 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7db40b9ac0 msgr2=0x7f7db40ae920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:56:35.693 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.685+0000 7f7db881d640 1 --2- 172.21.15.173:0/1372095746 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7db40b9ac0 0x7f7db40ae920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:35.693 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.685+0000 7f7db881d640 1 -- 172.21.15.173:0/1372095746 >> v2:172.21.15.164:3300/0 conn(0x7f7db40adc70 msgr2=0x7f7db41445e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:56:35.693 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.685+0000 7f7db881d640 1 --2- 172.21.15.173:0/1372095746 >> v2:172.21.15.164:3300/0 conn(0x7f7db40adc70 0x7f7db41445e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:35.693 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.685+0000 7f7db881d640 1 -- 172.21.15.173:0/1372095746 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7db40b53e0 con 0x7f7db4144b20 2024-03-23T14:56:35.693 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.685+0000 7f7db3fff640 1 --2- 172.21.15.173:0/1372095746 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7db40b9ac0 0x7f7db40ae920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T14:56:35.693 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.685+0000 7f7db37fe640 1 --2- 172.21.15.173:0/1372095746 >> v2:172.21.15.164:3300/0 conn(0x7f7db40adc70 0x7f7db41445e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T14:56:35.694 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.689+0000 7f7db881d640 1 --2- 172.21.15.173:0/1372095746 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7db4144b20 0x7f7db41477b0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f7db41474d0 tx=0x7f7da401c460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a16adcb657d7a15f server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:56:35.694 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.689+0000 7f7db2ffd640 1 -- 172.21.15.173:0/1372095746 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7da40ab620 con 0x7f7db4144b20 2024-03-23T14:56:35.694 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.689+0000 7f7db2ffd640 1 -- 172.21.15.173:0/1372095746 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7da40ab780 con 0x7f7db4144b20 2024-03-23T14:56:35.694 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.689+0000 7f7db2ffd640 1 -- 172.21.15.173:0/1372095746 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7da40abb10 con 0x7f7db4144b20 2024-03-23T14:56:35.695 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.689+0000 7f7dba280640 1 -- 172.21.15.173:0/1372095746 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7db4144b20 msgr2=0x7f7db41477b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:56:35.695 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.689+0000 7f7dba280640 1 --2- 172.21.15.173:0/1372095746 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7db4144b20 0x7f7db41477b0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f7db41474d0 tx=0x7f7da401c460 comp rx=0 tx=0).stop 2024-03-23T14:56:35.695 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.689+0000 7f7dba280640 1 -- 172.21.15.173:0/1372095746 shutdown_connections 2024-03-23T14:56:35.695 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.689+0000 7f7dba280640 1 --2- 172.21.15.173:0/1372095746 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7db4144b20 0x7f7db41477b0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:35.695 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.689+0000 7f7dba280640 1 --2- 172.21.15.173:0/1372095746 >> v2:172.21.15.164:3300/0 conn(0x7f7db40adc70 0x7f7db41445e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:35.695 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.689+0000 7f7dba280640 1 --2- 172.21.15.173:0/1372095746 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7db40b9ac0 0x7f7db40ae920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:35.695 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.689+0000 7f7dba280640 1 -- 172.21.15.173:0/1372095746 >> 172.21.15.173:0/1372095746 conn(0x7f7db406b930 msgr2=0x7f7db40b9150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:56:35.695 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.689+0000 7f7dba280640 1 -- 172.21.15.173:0/1372095746 shutdown_connections 2024-03-23T14:56:35.696 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.689+0000 7f7dba280640 1 -- 172.21.15.173:0/1372095746 wait complete. 2024-03-23T14:56:35.698 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.689+0000 7f7dba280640 1 Processor -- start 2024-03-23T14:56:35.698 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.693+0000 7f7dba280640 1 -- start start 2024-03-23T14:56:35.698 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.693+0000 7f7dba280640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7db40adc70 0x7f7db413c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:35.698 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.693+0000 7f7dba280640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7db40b9ac0 0x7f7db4140e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:35.698 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.693+0000 7f7dba280640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7db4144b20 0x7f7db4141380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:35.698 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.693+0000 7f7dba280640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f7db413d610 con 0x7f7db40b9ac0 2024-03-23T14:56:35.698 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.693+0000 7f7dba280640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f7db40b1150 con 0x7f7db40adc70 2024-03-23T14:56:35.698 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.693+0000 7f7dba280640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f7db413d2a0 con 0x7f7db4144b20 2024-03-23T14:56:35.699 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.693+0000 7f7db3fff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7db40adc70 0x7f7db413c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:35.699 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.693+0000 7f7db881d640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7db4144b20 0x7f7db4141380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:35.699 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.693+0000 7f7db3fff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7db40adc70 0x7f7db413c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.173:38596/0 (socket says 172.21.15.173:38596) 2024-03-23T14:56:35.699 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.693+0000 7f7db3fff640 1 -- 172.21.15.173:0/1417961900 learned_addr learned my addr 172.21.15.173:0/1417961900 (peer_addr_for_me v2:172.21.15.173:0/0) 2024-03-23T14:56:35.699 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.693+0000 7f7db37fe640 1 --2- 172.21.15.173:0/1417961900 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7db40b9ac0 0x7f7db4140e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:35.699 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.693+0000 7f7db881d640 1 -- 172.21.15.173:0/1417961900 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7db40adc70 msgr2=0x7f7db413c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:56:35.699 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.693+0000 7f7db881d640 1 --2- 172.21.15.173:0/1417961900 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7db40adc70 0x7f7db413c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:35.699 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.693+0000 7f7db881d640 1 -- 172.21.15.173:0/1417961900 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7db40b9ac0 msgr2=0x7f7db4140e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:56:35.699 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.693+0000 7f7db881d640 1 --2- 172.21.15.173:0/1417961900 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7db40b9ac0 0x7f7db4140e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:35.699 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.693+0000 7f7db881d640 1 -- 172.21.15.173:0/1417961900 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dac009a70 con 0x7f7db4144b20 2024-03-23T14:56:35.699 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.693+0000 7f7db3fff640 1 --2- 172.21.15.173:0/1417961900 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7db40adc70 0x7f7db413c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T14:56:35.699 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.693+0000 7f7db881d640 1 --2- 172.21.15.173:0/1417961900 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7db4144b20 0x7f7db4141380 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7db4067960 tx=0x7f7da401aca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:56:35.699 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.693+0000 7f7db0ff9640 1 -- 172.21.15.173:0/1417961900 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7da40abe40 con 0x7f7db4144b20 2024-03-23T14:56:35.699 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.693+0000 7f7db0ff9640 1 -- 172.21.15.173:0/1417961900 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7da40be3f0 con 0x7f7db4144b20 2024-03-23T14:56:35.700 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.693+0000 7f7dba280640 1 -- 172.21.15.173:0/1417961900 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7da40094d0 con 0x7f7db4144b20 2024-03-23T14:56:35.700 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.693+0000 7f7db0ff9640 1 -- 172.21.15.173:0/1417961900 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7da40be7a0 con 0x7f7db4144b20 2024-03-23T14:56:35.700 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.693+0000 7f7dba280640 1 -- 172.21.15.173:0/1417961900 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f7db40b0dd0 con 0x7f7db4144b20 2024-03-23T14:56:35.700 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.693+0000 7f7db0ff9640 1 -- 172.21.15.173:0/1417961900 <== mon.2 v2:172.21.15.173:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7da40bec90 con 0x7f7db4144b20 2024-03-23T14:56:35.700 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.693+0000 7f7db0ff9640 1 -- 172.21.15.173:0/1417961900 <== mon.2 v2:172.21.15.173:3301/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7f7da40c7820 con 0x7f7db4144b20 2024-03-23T14:56:35.700 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.693+0000 7f7dba280640 1 -- 172.21.15.173:0/1417961900 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7db413d460 con 0x7f7db4144b20 2024-03-23T14:56:35.700 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.697+0000 7f7db0ff9640 1 -- 172.21.15.173:0/1417961900 <== mon.2 v2:172.21.15.173:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f7da40c7390 con 0x7f7db4144b20 2024-03-23T14:56:35.769 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.765+0000 7f7dba280640 1 -- 172.21.15.173:0/1417961900 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "osd new", "uuid": "7e18c957-3091-46d5-8e17-8f3354c1726c", "id": 3} v 0) -- 0x7f7db40b1930 con 0x7f7db4144b20 2024-03-23T14:56:35.776 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.773+0000 7f7db0ff9640 1 -- 172.21.15.173:0/1417961900 <== mon.2 v2:172.21.15.173:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "7e18c957-3091-46d5-8e17-8f3354c1726c", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7f7da40cd020 con 0x7f7db4144b20 2024-03-23T14:56:35.776 INFO:teuthology.orchestra.run.smithi173.stdout:3 2024-03-23T14:56:35.778 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.773+0000 7f7dba280640 1 -- 172.21.15.173:0/1417961900 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7db4144b20 msgr2=0x7f7db4141380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:56:35.778 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.773+0000 7f7dba280640 1 --2- 172.21.15.173:0/1417961900 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7db4144b20 0x7f7db4141380 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7db4067960 tx=0x7f7da401aca0 comp rx=0 tx=0).stop 2024-03-23T14:56:35.778 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.773+0000 7f7dba280640 1 -- 172.21.15.173:0/1417961900 shutdown_connections 2024-03-23T14:56:35.778 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.773+0000 7f7dba280640 1 --2- 172.21.15.173:0/1417961900 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7db4144b20 0x7f7db4141380 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:35.778 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.773+0000 7f7dba280640 1 --2- 172.21.15.173:0/1417961900 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7db40b9ac0 0x7f7db4140e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:35.778 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.773+0000 7f7dba280640 1 --2- 172.21.15.173:0/1417961900 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7db40adc70 0x7f7db413c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:35.778 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.773+0000 7f7dba280640 1 -- 172.21.15.173:0/1417961900 >> 172.21.15.173:0/1417961900 conn(0x7f7db406b930 msgr2=0x7f7db40b38f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:56:35.778 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.773+0000 7f7dba280640 1 -- 172.21.15.173:0/1417961900 shutdown_connections 2024-03-23T14:56:35.779 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.773+0000 7f7dba280640 1 -- 172.21.15.173:0/1417961900 wait complete. 2024-03-23T14:56:35.794 DEBUG:teuthology.orchestra.run.smithi173:> sudo ceph --cluster ceph osd new 8e83f315-42bf-4471-9ce8-d81948da0f9f 4 2024-03-23T14:56:35.903 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.897+0000 7f370d30e640 1 Processor -- start 2024-03-23T14:56:35.903 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.897+0000 7f370d30e640 1 -- start start 2024-03-23T14:56:35.903 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.897+0000 7f370d30e640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3708147890 0x7f37080b0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:35.903 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.897+0000 7f370d30e640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37081480c0 con 0x7f37080707c0 2024-03-23T14:56:35.903 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.897+0000 7f370d30e640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37080725c0 con 0x7f3708070c00 2024-03-23T14:56:35.903 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.897+0000 7f370d30e640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f37080705c0 con 0x7f3708147890 2024-03-23T14:56:35.903 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.897+0000 7f3706ffd640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3708147890 0x7f37080b0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:35.903 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.897+0000 7f37077fe640 1 --1- >> v1:172.21.15.173:6789/0 conn(0x7f3708070c00 0x7f370814e980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.173:6789/0 says I am v1:172.21.15.173:52836/0 (socket says 172.21.15.173:52836) 2024-03-23T14:56:35.903 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.897+0000 7f3706ffd640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3708147890 0x7f37080b0910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.173:59438/0 (socket says 172.21.15.173:59438) 2024-03-23T14:56:35.903 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.897+0000 7f37077fe640 1 -- 172.21.15.173:0/2118710455 learned_addr learned my addr 172.21.15.173:0/2118710455 (peer_addr_for_me v1:172.21.15.173:0/0) 2024-03-23T14:56:35.903 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.897+0000 7f3705ffb640 1 -- 172.21.15.173:0/2118710455 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 928383517 0 0) 0x7f37080725c0 con 0x7f3708070c00 2024-03-23T14:56:35.904 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.897+0000 7f3706ffd640 1 -- 172.21.15.173:0/2118710455 >> v1:172.21.15.173:6789/0 conn(0x7f3708070c00 legacy=0x7f370814e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:56:35.904 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.897+0000 7f3706ffd640 1 -- 172.21.15.173:0/2118710455 >> v1:172.21.15.164:6789/0 conn(0x7f37080707c0 legacy=0x7f370813df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:56:35.904 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.897+0000 7f3706ffd640 1 -- 172.21.15.173:0/2118710455 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3708072350 con 0x7f3708147890 2024-03-23T14:56:35.905 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.901+0000 7f3706ffd640 1 --2- 172.21.15.173:0/2118710455 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3708147890 0x7f37080b0910 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f36f80097f0 tx=0x7f36f801c380 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e8e1ca5cdbdfd73 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:56:35.905 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.901+0000 7f3705ffb640 1 -- 172.21.15.173:0/2118710455 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36f80ab600 con 0x7f3708147890 2024-03-23T14:56:35.905 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.901+0000 7f3705ffb640 1 -- 172.21.15.173:0/2118710455 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36f80a7050 con 0x7f3708147890 2024-03-23T14:56:35.905 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.901+0000 7f3705ffb640 1 -- 172.21.15.173:0/2118710455 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36f80ab980 con 0x7f3708147890 2024-03-23T14:56:35.909 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.901+0000 7f370d30e640 1 -- 172.21.15.173:0/2118710455 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3708147890 msgr2=0x7f37080b0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:56:35.909 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.901+0000 7f370d30e640 1 --2- 172.21.15.173:0/2118710455 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3708147890 0x7f37080b0910 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f36f80097f0 tx=0x7f36f801c380 comp rx=0 tx=0).stop 2024-03-23T14:56:35.909 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.901+0000 7f370d30e640 1 -- 172.21.15.173:0/2118710455 shutdown_connections 2024-03-23T14:56:35.909 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.901+0000 7f370d30e640 1 --2- 172.21.15.173:0/2118710455 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3708147890 0x7f37080b0910 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:35.909 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.901+0000 7f370d30e640 1 -- 172.21.15.173:0/2118710455 >> 172.21.15.173:0/2118710455 conn(0x7f37080b7970 msgr2=0x7f37080b7d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:56:35.909 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.901+0000 7f370d30e640 1 -- 172.21.15.173:0/2118710455 shutdown_connections 2024-03-23T14:56:35.909 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.901+0000 7f370d30e640 1 -- 172.21.15.173:0/2118710455 wait complete. 2024-03-23T14:56:35.909 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.901+0000 7f370d30e640 1 Processor -- start 2024-03-23T14:56:35.909 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.901+0000 7f370d30e640 1 -- start start 2024-03-23T14:56:35.909 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.901+0000 7f370d30e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f37080707c0 0x7f37080b05a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:35.910 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.901+0000 7f370d30e640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3708070c00 0x7f3708154460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:35.910 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.901+0000 7f370d30e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3708147890 0x7f3708143c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:35.910 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.901+0000 7f370d30e640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f37081426b0 con 0x7f3708147890 2024-03-23T14:56:35.910 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.901+0000 7f370d30e640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f3708143120 con 0x7f37080707c0 2024-03-23T14:56:35.910 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.901+0000 7f370d30e640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f3708143390 con 0x7f3708070c00 2024-03-23T14:56:35.910 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.901+0000 7f37067fc640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3708070c00 0x7f3708154460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:35.910 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.901+0000 7f3706ffd640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f37080707c0 0x7f37080b05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:35.910 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.901+0000 7f37067fc640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3708070c00 0x7f3708154460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.173:59454/0 (socket says 172.21.15.173:59454) 2024-03-23T14:56:35.910 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.901+0000 7f37067fc640 1 -- 172.21.15.173:0/1978037743 learned_addr learned my addr 172.21.15.173:0/1978037743 (peer_addr_for_me v2:172.21.15.173:0/0) 2024-03-23T14:56:35.910 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.901+0000 7f37077fe640 1 --2- 172.21.15.173:0/1978037743 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3708147890 0x7f3708143c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:35.910 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.901+0000 7f37067fc640 1 -- 172.21.15.173:0/1978037743 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f37080707c0 msgr2=0x7f37080b05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:56:35.910 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.901+0000 7f37067fc640 1 --2- 172.21.15.173:0/1978037743 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f37080707c0 0x7f37080b05a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:35.910 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.901+0000 7f37067fc640 1 -- 172.21.15.173:0/1978037743 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3708147890 msgr2=0x7f3708143c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:56:35.910 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.901+0000 7f37067fc640 1 --2- 172.21.15.173:0/1978037743 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3708147890 0x7f3708143c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:35.911 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.901+0000 7f37067fc640 1 -- 172.21.15.173:0/1978037743 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36f8009420 con 0x7f3708070c00 2024-03-23T14:56:35.911 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.901+0000 7f3706ffd640 1 --2- 172.21.15.173:0/1978037743 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f37080707c0 0x7f37080b05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T14:56:35.911 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.905+0000 7f37067fc640 1 --2- 172.21.15.173:0/1978037743 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3708070c00 0x7f3708154460 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f36fc00c480 tx=0x7f36fc00c940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:56:35.911 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.905+0000 7f36ef7fe640 1 -- 172.21.15.173:0/1978037743 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36fc012830 con 0x7f3708070c00 2024-03-23T14:56:35.911 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.905+0000 7f36ef7fe640 1 -- 172.21.15.173:0/1978037743 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36fc0129b0 con 0x7f3708070c00 2024-03-23T14:56:35.911 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.905+0000 7f36ef7fe640 1 -- 172.21.15.173:0/1978037743 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36fc012d60 con 0x7f3708070c00 2024-03-23T14:56:35.911 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.905+0000 7f370d30e640 1 -- 172.21.15.173:0/1978037743 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3708141d10 con 0x7f3708070c00 2024-03-23T14:56:35.911 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.905+0000 7f370d30e640 1 -- 172.21.15.173:0/1978037743 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f37081528c0 con 0x7f3708070c00 2024-03-23T14:56:35.911 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.905+0000 7f36ef7fe640 1 -- 172.21.15.173:0/1978037743 <== mon.2 v2:172.21.15.173:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f36fc01c460 con 0x7f3708070c00 2024-03-23T14:56:35.911 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.905+0000 7f36ef7fe640 1 -- 172.21.15.173:0/1978037743 <== mon.2 v2:172.21.15.173:3301/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7f36fc01bce0 con 0x7f3708070c00 2024-03-23T14:56:35.911 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.905+0000 7f370d30e640 1 -- 172.21.15.173:0/1978037743 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3708142510 con 0x7f3708070c00 2024-03-23T14:56:35.912 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.905+0000 7f36ef7fe640 1 -- 172.21.15.173:0/1978037743 <== mon.2 v2:172.21.15.173:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f3708142510 con 0x7f3708070c00 2024-03-23T14:56:35.951 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-03-23T14:56:35.946+0000 7efcb3f00180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-23T14:56:35.979 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.973+0000 7f370d30e640 1 -- 172.21.15.173:0/1978037743 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "osd new", "uuid": "8e83f315-42bf-4471-9ce8-d81948da0f9f", "id": 4} v 0) -- 0x7f37080690a0 con 0x7f3708070c00 2024-03-23T14:56:35.985 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.981+0000 7f36ef7fe640 1 -- 172.21.15.173:0/1978037743 <== mon.2 v2:172.21.15.173:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "8e83f315-42bf-4471-9ce8-d81948da0f9f", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7f36fc01b3c0 con 0x7f3708070c00 2024-03-23T14:56:35.986 INFO:teuthology.orchestra.run.smithi173.stdout:4 2024-03-23T14:56:35.987 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.981+0000 7f370d30e640 1 -- 172.21.15.173:0/1978037743 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3708070c00 msgr2=0x7f3708154460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:56:35.987 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.981+0000 7f370d30e640 1 --2- 172.21.15.173:0/1978037743 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3708070c00 0x7f3708154460 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f36fc00c480 tx=0x7f36fc00c940 comp rx=0 tx=0).stop 2024-03-23T14:56:35.987 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.981+0000 7f370d30e640 1 -- 172.21.15.173:0/1978037743 shutdown_connections 2024-03-23T14:56:35.987 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.981+0000 7f370d30e640 1 --2- 172.21.15.173:0/1978037743 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3708147890 0x7f3708143c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:35.987 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.981+0000 7f370d30e640 1 --2- 172.21.15.173:0/1978037743 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3708070c00 0x7f3708154460 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:35.987 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.981+0000 7f370d30e640 1 --2- 172.21.15.173:0/1978037743 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f37080707c0 0x7f37080b05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:35.988 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.981+0000 7f370d30e640 1 -- 172.21.15.173:0/1978037743 >> 172.21.15.173:0/1978037743 conn(0x7f37080b7970 msgr2=0x7f370802c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:56:35.988 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.981+0000 7f370d30e640 1 -- 172.21.15.173:0/1978037743 shutdown_connections 2024-03-23T14:56:35.988 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:35.981+0000 7f370d30e640 1 -- 172.21.15.173:0/1978037743 wait complete. 2024-03-23T14:56:36.005 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-03-23T14:56:36.002+0000 7efcb3f00180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-23T14:56:36.006 DEBUG:teuthology.orchestra.run.smithi173:> sudo ceph --cluster ceph osd new a230c872-71f9-40ab-8999-defd0162d4ed 5 2024-03-23T14:56:36.026 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-03-23T14:56:36.021+0000 7f3fc8b03180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-23T14:56:36.056 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-03-23T14:56:36.054+0000 7efcb3f00180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-23T14:56:36.081 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-03-23T14:56:36.077+0000 7f3fc8b03180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-23T14:56:36.114 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.109+0000 7f620108e640 1 Processor -- start 2024-03-23T14:56:36.114 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.109+0000 7f620108e640 1 -- start start 2024-03-23T14:56:36.116 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.109+0000 7f620108e640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f61fc0b6740 0x7f61fc0b41d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:36.117 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.109+0000 7f620108e640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61fc0b5530 con 0x7f61fc0b4710 2024-03-23T14:56:36.117 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.109+0000 7f620108e640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61fc0b2a70 con 0x7f61fc0b6360 2024-03-23T14:56:36.117 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.109+0000 7f620108e640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f61fc0b5a60 con 0x7f61fc0b6740 2024-03-23T14:56:36.117 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.109+0000 7f61fad9d640 1 --1- >> v1:172.21.15.173:6789/0 conn(0x7f61fc0b6360 0x7f61fc145a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.173:6789/0 says I am v1:172.21.15.173:52838/0 (socket says 172.21.15.173:52838) 2024-03-23T14:56:36.117 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.109+0000 7f61fad9d640 1 -- 172.21.15.173:0/1451596630 learned_addr learned my addr 172.21.15.173:0/1451596630 (peer_addr_for_me v1:172.21.15.173:0/0) 2024-03-23T14:56:36.117 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.109+0000 7f61f9d9b640 1 -- 172.21.15.173:0/1451596630 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2213520775 0 0) 0x7f61fc0b2a70 con 0x7f61fc0b6360 2024-03-23T14:56:36.117 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.109+0000 7f61fa59c640 1 --2- 172.21.15.173:0/1451596630 >> v2:172.21.15.164:3300/0 conn(0x7f61fc0b6740 0x7f61fc0b41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:36.117 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.109+0000 7f61f9d9b640 1 -- 172.21.15.173:0/1451596630 --> v1:172.21.15.173:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f61e8003530 con 0x7f61fc0b6360 2024-03-23T14:56:36.117 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.109+0000 7f61f9d9b640 1 -- 172.21.15.173:0/1451596630 <== mon.1 v1:172.21.15.173:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3084487338 0 0) 0x7f61e8003530 con 0x7f61fc0b6360 2024-03-23T14:56:36.117 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.109+0000 7f61f9d9b640 1 -- 172.21.15.173:0/1451596630 >> v2:172.21.15.164:3300/0 conn(0x7f61fc0b6740 msgr2=0x7f61fc0b41d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:56:36.117 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.109+0000 7f61f9d9b640 1 --2- 172.21.15.173:0/1451596630 >> v2:172.21.15.164:3300/0 conn(0x7f61fc0b6740 0x7f61fc0b41d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:36.117 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.109+0000 7f61f9d9b640 1 -- 172.21.15.173:0/1451596630 >> v1:172.21.15.164:6789/0 conn(0x7f61fc0b4710 legacy=0x7f61fc14fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:56:36.117 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.109+0000 7f61f9d9b640 1 -- 172.21.15.173:0/1451596630 --> v1:172.21.15.173:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61fc0b2800 con 0x7f61fc0b6360 2024-03-23T14:56:36.118 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.109+0000 7f61f9d9b640 1 -- 172.21.15.173:0/1451596630 <== mon.1 v1:172.21.15.173:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2526390548 0 0) 0x7f61e4001680 con 0x7f61fc0b6360 2024-03-23T14:56:36.118 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.109+0000 7f61f9d9b640 1 -- 172.21.15.173:0/1451596630 >> v1:172.21.15.173:6789/0 conn(0x7f61fc0b6360 legacy=0x7f61fc145a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:56:36.118 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.109+0000 7f61f9d9b640 1 --2- 172.21.15.173:0/1451596630 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f61e8003980 0x7f61e8003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:36.118 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.109+0000 7f61f9d9b640 1 --2- 172.21.15.173:0/1451596630 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f61e80043a0 0x7f61e8004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:36.118 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.109+0000 7f61f9d9b640 1 --2- 172.21.15.173:0/1451596630 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f61e8005110 0x7f61e800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:36.118 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.109+0000 7f61f9d9b640 1 -- 172.21.15.173:0/1451596630 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f61fc0b5a60 con 0x7f61e8003980 2024-03-23T14:56:36.118 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.109+0000 7f61f9d9b640 1 -- 172.21.15.173:0/1451596630 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f61e800c060 con 0x7f61e80043a0 2024-03-23T14:56:36.118 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.109+0000 7f61f9d9b640 1 -- 172.21.15.173:0/1451596630 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f61e800c220 con 0x7f61e8005110 2024-03-23T14:56:36.118 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.109+0000 7f61fa59c640 1 --2- 172.21.15.173:0/1451596630 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f61e80043a0 0x7f61e8004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:36.118 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.109+0000 7f61fb59e640 1 --2- 172.21.15.173:0/1451596630 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f61e8005110 0x7f61e800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:36.118 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.109+0000 7f61fad9d640 1 --2- 172.21.15.173:0/1451596630 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f61e8003980 0x7f61e8003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:36.118 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.109+0000 7f61fb59e640 1 -- 172.21.15.173:0/1451596630 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f61e80043a0 msgr2=0x7f61e8004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:56:36.118 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.113+0000 7f61fb59e640 1 --2- 172.21.15.173:0/1451596630 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f61e80043a0 0x7f61e8004870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:36.118 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.113+0000 7f61fb59e640 1 -- 172.21.15.173:0/1451596630 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f61e8003980 msgr2=0x7f61e8003d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:56:36.119 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.113+0000 7f61fb59e640 1 --2- 172.21.15.173:0/1451596630 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f61e8003980 0x7f61e8003d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:36.119 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.113+0000 7f61fb59e640 1 -- 172.21.15.173:0/1451596630 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61e8003530 con 0x7f61e8005110 2024-03-23T14:56:36.119 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.113+0000 7f61fad9d640 1 --2- 172.21.15.173:0/1451596630 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f61e8003980 0x7f61e8003d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T14:56:36.119 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.113+0000 7f61fa59c640 1 --2- 172.21.15.173:0/1451596630 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f61e80043a0 0x7f61e8004870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T14:56:36.120 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.113+0000 7f61fb59e640 1 -- 172.21.15.173:0/1451596630 reap_dead start 2024-03-23T14:56:36.120 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.113+0000 7f61fb59e640 1 --2- 172.21.15.173:0/1451596630 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f61e8005110 0x7f61e800b990 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f61f4001eb0 tx=0x7f61f401c7a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c16745a3c086089 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:56:36.120 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.113+0000 7f61f9d9b640 1 -- 172.21.15.173:0/1451596630 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61f4003b80 con 0x7f61e8005110 2024-03-23T14:56:36.120 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.113+0000 7f61f9d9b640 1 -- 172.21.15.173:0/1451596630 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61f4003d00 con 0x7f61e8005110 2024-03-23T14:56:36.120 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.113+0000 7f61f9d9b640 1 -- 172.21.15.173:0/1451596630 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61f4005310 con 0x7f61e8005110 2024-03-23T14:56:36.120 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.117+0000 7f620108e640 1 -- 172.21.15.173:0/1451596630 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f61e8005110 msgr2=0x7f61e800b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:56:36.121 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.117+0000 7f620108e640 1 --2- 172.21.15.173:0/1451596630 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f61e8005110 0x7f61e800b990 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f61f4001eb0 tx=0x7f61f401c7a0 comp rx=0 tx=0).stop 2024-03-23T14:56:36.121 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.117+0000 7f620108e640 1 -- 172.21.15.173:0/1451596630 shutdown_connections 2024-03-23T14:56:36.121 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.117+0000 7f620108e640 1 --2- 172.21.15.173:0/1451596630 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f61e8005110 0x7f61e800b990 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:36.121 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.117+0000 7f620108e640 1 -- 172.21.15.173:0/1451596630 >> 172.21.15.173:0/1451596630 conn(0x7f61fc0b7630 msgr2=0x7f61fc0b7a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:56:36.121 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.117+0000 7f620108e640 1 -- 172.21.15.173:0/1451596630 shutdown_connections 2024-03-23T14:56:36.121 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.117+0000 7f620108e640 1 -- 172.21.15.173:0/1451596630 wait complete. 2024-03-23T14:56:36.121 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.117+0000 7f620108e640 1 Processor -- start 2024-03-23T14:56:36.122 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.117+0000 7f620108e640 1 -- start start 2024-03-23T14:56:36.124 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.117+0000 7f620108e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f61e8005110 0x7f61fc13b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:36.125 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.117+0000 7f620108e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f61fc142ab0 0x7f61fc13d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:36.125 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.117+0000 7f620108e640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f61fc13db00 0x7f61fc131b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:36.125 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.117+0000 7f620108e640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f61fc1428b0 con 0x7f61fc142ab0 2024-03-23T14:56:36.125 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.117+0000 7f620108e640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f61fc13bca0 con 0x7f61e8005110 2024-03-23T14:56:36.125 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.117+0000 7f620108e640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f61fc13b070 con 0x7f61fc13db00 2024-03-23T14:56:36.125 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.117+0000 7f61fad9d640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f61e8005110 0x7f61fc13b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:36.125 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.117+0000 7f61fad9d640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f61e8005110 0x7f61fc13b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.173:38606/0 (socket says 172.21.15.173:38606) 2024-03-23T14:56:36.125 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.117+0000 7f61fad9d640 1 -- 172.21.15.173:0/2092281733 learned_addr learned my addr 172.21.15.173:0/2092281733 (peer_addr_for_me v2:172.21.15.173:0/0) 2024-03-23T14:56:36.125 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.117+0000 7f61fb59e640 1 --2- 172.21.15.173:0/2092281733 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f61fc13db00 0x7f61fc131b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:36.125 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.117+0000 7f61fa59c640 1 --2- 172.21.15.173:0/2092281733 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f61fc142ab0 0x7f61fc13d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:36.125 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.117+0000 7f61fb59e640 1 -- 172.21.15.173:0/2092281733 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f61e8005110 msgr2=0x7f61fc13b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:56:36.125 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.117+0000 7f61fb59e640 1 --2- 172.21.15.173:0/2092281733 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f61e8005110 0x7f61fc13b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:36.126 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.117+0000 7f61fb59e640 1 -- 172.21.15.173:0/2092281733 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f61fc142ab0 msgr2=0x7f61fc13d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:56:36.126 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.117+0000 7f61fb59e640 1 --2- 172.21.15.173:0/2092281733 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f61fc142ab0 0x7f61fc13d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:36.126 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.117+0000 7f61fb59e640 1 -- 172.21.15.173:0/2092281733 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61fc13ce50 con 0x7f61fc13db00 2024-03-23T14:56:36.126 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.117+0000 7f61fa59c640 1 --2- 172.21.15.173:0/2092281733 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f61fc142ab0 0x7f61fc13d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T14:56:36.126 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.117+0000 7f61fad9d640 1 --2- 172.21.15.173:0/2092281733 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f61e8005110 0x7f61fc13b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T14:56:36.126 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.117+0000 7f61fb59e640 1 --2- 172.21.15.173:0/2092281733 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f61fc13db00 0x7f61fc131b70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f61fc072dc0 tx=0x7f61e8005800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:56:36.126 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.117+0000 7f61f37fe640 1 -- 172.21.15.173:0/2092281733 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61f4007e00 con 0x7f61fc13db00 2024-03-23T14:56:36.126 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.117+0000 7f61f37fe640 1 -- 172.21.15.173:0/2092281733 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61f4003980 con 0x7f61fc13db00 2024-03-23T14:56:36.126 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.117+0000 7f620108e640 1 -- 172.21.15.173:0/2092281733 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61fc13bf00 con 0x7f61fc13db00 2024-03-23T14:56:36.126 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.117+0000 7f61f37fe640 1 -- 172.21.15.173:0/2092281733 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61f40a9470 con 0x7f61fc13db00 2024-03-23T14:56:36.126 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.117+0000 7f620108e640 1 -- 172.21.15.173:0/2092281733 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f61fc141980 con 0x7f61fc13db00 2024-03-23T14:56:36.126 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.121+0000 7f61f37fe640 1 -- 172.21.15.173:0/2092281733 <== mon.2 v2:172.21.15.173:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f61f40a9960 con 0x7f61fc13db00 2024-03-23T14:56:36.126 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.121+0000 7f61f37fe640 1 -- 172.21.15.173:0/2092281733 <== mon.2 v2:172.21.15.173:3301/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7f61f40b5bf0 con 0x7f61fc13db00 2024-03-23T14:56:36.127 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.121+0000 7f620108e640 1 -- 172.21.15.173:0/2092281733 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61fc13ad70 con 0x7f61fc13db00 2024-03-23T14:56:36.127 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.121+0000 7f61f37fe640 1 -- 172.21.15.173:0/2092281733 <== mon.2 v2:172.21.15.173:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f61ec0096b0 con 0x7f61fc13db00 2024-03-23T14:56:36.134 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-03-23T14:56:36.130+0000 7efcb3f00180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-23T14:56:36.149 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-03-23T14:56:36.145+0000 7f3fc8b03180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-23T14:56:36.184 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-03-23T14:56:36.182+0000 7efcb3f00180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-23T14:56:36.207 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.201+0000 7f620108e640 1 -- 172.21.15.173:0/2092281733 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "osd new", "uuid": "a230c872-71f9-40ab-8999-defd0162d4ed", "id": 5} v 0) -- 0x7f61fc13b050 con 0x7f61fc13db00 2024-03-23T14:56:36.213 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.209+0000 7f61f37fe640 1 -- 172.21.15.173:0/2092281733 <== mon.2 v2:172.21.15.173:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a230c872-71f9-40ab-8999-defd0162d4ed", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7f61f4001a80 con 0x7f61fc13db00 2024-03-23T14:56:36.213 INFO:teuthology.orchestra.run.smithi173.stdout:5 2024-03-23T14:56:36.215 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.209+0000 7f620108e640 1 -- 172.21.15.173:0/2092281733 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f61fc13db00 msgr2=0x7f61fc131b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:56:36.215 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.209+0000 7f620108e640 1 --2- 172.21.15.173:0/2092281733 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f61fc13db00 0x7f61fc131b70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f61fc072dc0 tx=0x7f61e8005800 comp rx=0 tx=0).stop 2024-03-23T14:56:36.215 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.209+0000 7f620108e640 1 -- 172.21.15.173:0/2092281733 shutdown_connections 2024-03-23T14:56:36.215 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.209+0000 7f620108e640 1 --2- 172.21.15.173:0/2092281733 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f61fc13db00 0x7f61fc131b70 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:36.215 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.209+0000 7f620108e640 1 --2- 172.21.15.173:0/2092281733 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f61fc142ab0 0x7f61fc13d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:36.215 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.209+0000 7f620108e640 1 --2- 172.21.15.173:0/2092281733 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f61e8005110 0x7f61fc13b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:36.215 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.209+0000 7f620108e640 1 -- 172.21.15.173:0/2092281733 >> 172.21.15.173:0/2092281733 conn(0x7f61fc0b7630 msgr2=0x7f61fc071350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:56:36.215 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.209+0000 7f620108e640 1 -- 172.21.15.173:0/2092281733 shutdown_connections 2024-03-23T14:56:36.215 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.209+0000 7f620108e640 1 -- 172.21.15.173:0/2092281733 wait complete. 2024-03-23T14:56:36.229 DEBUG:teuthology.orchestra.run.smithi173:> sudo ceph --cluster ceph osd new 6193bfd7-8432-4f7f-a117-a3fcee21af99 6 2024-03-23T14:56:36.237 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-03-23T14:56:36.233+0000 7f3fc8b03180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-23T14:56:36.261 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-03-23T14:56:36.258+0000 7efcb3f00180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-23T14:56:36.294 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-03-23T14:56:36.289+0000 7f3fc8b03180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-23T14:56:36.314 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-03-23T14:56:36.310+0000 7efcb3f00180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-23T14:56:36.344 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.341+0000 7f4ec1ea8640 1 Processor -- start 2024-03-23T14:56:36.344 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.341+0000 7f4ec1ea8640 1 -- start start 2024-03-23T14:56:36.346 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.341+0000 7f4ec1ea8640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f4ebc06e8e0 0x7f4ebc141380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:36.346 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.341+0000 7f4ec1ea8640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f4ebc02c2a0 0x7f4ebc02c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:36.346 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.341+0000 7f4ec1ea8640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ebc0b9470 con 0x7f4ebc14a880 2024-03-23T14:56:36.346 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.341+0000 7f4ec1ea8640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f4ebc072af0 con 0x7f4ebc06e8e0 2024-03-23T14:56:36.346 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.341+0000 7f4ec1ea8640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 -- 0x7f4ebc06ecc0 con 0x7f4ebc02c2a0 2024-03-23T14:56:36.346 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.341+0000 7f4eb2ffd640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f4ebc02c2a0 0x7f4ebc02c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:36.346 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.341+0000 7f4eb2ffd640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f4ebc02c2a0 0x7f4ebc02c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.173:49642/0 (socket says 172.21.15.173:49642) 2024-03-23T14:56:36.346 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.341+0000 7f4ebbfff640 1 --1- >> v1:172.21.15.173:6789/0 conn(0x7f4ebc14a880 0x7f4ebc14acd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.173:6789/0 says I am v1:172.21.15.173:59612/0 (socket says 172.21.15.173:59612) 2024-03-23T14:56:36.346 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.341+0000 7f4eb2ffd640 1 -- 172.21.15.173:0/1593554071 learned_addr learned my addr 172.21.15.173:0/1593554071 (peer_addr_for_me v2:172.21.15.173:0/0) 2024-03-23T14:56:36.346 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.341+0000 7f4ebb7fe640 1 --2- 172.21.15.173:0/1593554071 >> v2:172.21.15.164:3300/0 conn(0x7f4ebc06e8e0 0x7f4ebc141380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:36.347 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.341+0000 7f4eb2ffd640 1 -- 172.21.15.173:0/1593554071 >> v2:172.21.15.164:3300/0 conn(0x7f4ebc06e8e0 msgr2=0x7f4ebc141380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:56:36.347 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.341+0000 7f4eb2ffd640 1 --2- 172.21.15.173:0/1593554071 >> v2:172.21.15.164:3300/0 conn(0x7f4ebc06e8e0 0x7f4ebc141380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:36.347 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.341+0000 7f4eb2ffd640 1 -- 172.21.15.173:0/1593554071 >> v1:172.21.15.173:6789/0 conn(0x7f4ebc14a880 legacy=0x7f4ebc14acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:56:36.347 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.341+0000 7f4eb2ffd640 1 -- 172.21.15.173:0/1593554071 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ebc072040 con 0x7f4ebc02c2a0 2024-03-23T14:56:36.347 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.341+0000 7f4ebaffd640 1 -- 172.21.15.173:0/1593554071 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1748184374 0 0) 0x7f4ebc0b9470 con 0x7f4ebc14a880 2024-03-23T14:56:36.348 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.345+0000 7f4eb2ffd640 1 --2- 172.21.15.173:0/1593554071 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4ebc02c2a0 0x7f4ebc02c6e0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc0b0dc0 tx=0x7f4ea401ed10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9bccd9f3dcc147e3 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:56:36.348 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.345+0000 7f4ebaffd640 1 -- 172.21.15.173:0/1593554071 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea40a8a10 con 0x7f4ebc02c2a0 2024-03-23T14:56:36.348 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.345+0000 7f4ebaffd640 1 -- 172.21.15.173:0/1593554071 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ea40a8b70 con 0x7f4ebc02c2a0 2024-03-23T14:56:36.348 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.345+0000 7f4ebaffd640 1 -- 172.21.15.173:0/1593554071 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea40b0440 con 0x7f4ebc02c2a0 2024-03-23T14:56:36.349 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.345+0000 7f4ec1ea8640 1 -- 172.21.15.173:0/1593554071 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4ebc02c2a0 msgr2=0x7f4ebc02c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:56:36.349 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.345+0000 7f4ec1ea8640 1 --2- 172.21.15.173:0/1593554071 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4ebc02c2a0 0x7f4ebc02c6e0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc0b0dc0 tx=0x7f4ea401ed10 comp rx=0 tx=0).stop 2024-03-23T14:56:36.349 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.345+0000 7f4ec1ea8640 1 -- 172.21.15.173:0/1593554071 shutdown_connections 2024-03-23T14:56:36.349 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.345+0000 7f4ec1ea8640 1 --2- 172.21.15.173:0/1593554071 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4ebc02c2a0 0x7f4ebc02c6e0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:36.350 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.345+0000 7f4ec1ea8640 1 --2- 172.21.15.173:0/1593554071 >> v2:172.21.15.164:3300/0 conn(0x7f4ebc06e8e0 0x7f4ebc141380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:36.350 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.345+0000 7f4ec1ea8640 1 -- 172.21.15.173:0/1593554071 >> 172.21.15.173:0/1593554071 conn(0x7f4ebc0b7970 msgr2=0x7f4ebc0b7d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:56:36.350 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.345+0000 7f4ec1ea8640 1 -- 172.21.15.173:0/1593554071 shutdown_connections 2024-03-23T14:56:36.350 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.345+0000 7f4ec1ea8640 1 -- 172.21.15.173:0/1593554071 wait complete. 2024-03-23T14:56:36.350 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.345+0000 7f4ec1ea8640 1 Processor -- start 2024-03-23T14:56:36.350 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.345+0000 7f4ec1ea8640 1 -- start start 2024-03-23T14:56:36.353 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.345+0000 7f4ec1ea8640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4ebc02c2a0 0x7f4ebc149e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:36.353 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.345+0000 7f4ec1ea8640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4ebc06e8e0 0x7f4ebc1dfac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:36.353 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.345+0000 7f4ec1ea8640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4ebc14a880 0x7f4ebc0b50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:36.353 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.345+0000 7f4ec1ea8640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f4ebc0b36c0 con 0x7f4ebc14a880 2024-03-23T14:56:36.353 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.345+0000 7f4ec1ea8640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f4ebc0b3d70 con 0x7f4ebc06e8e0 2024-03-23T14:56:36.353 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.345+0000 7f4ec1ea8640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f4ebc1d0170 con 0x7f4ebc02c2a0 2024-03-23T14:56:36.353 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.345+0000 7f4ebb7fe640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4ebc02c2a0 0x7f4ebc149e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:36.353 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.345+0000 7f4eb2ffd640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4ebc06e8e0 0x7f4ebc1dfac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:36.353 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.345+0000 7f4ebb7fe640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4ebc02c2a0 0x7f4ebc149e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.173:56964/0 (socket says 172.21.15.173:56964) 2024-03-23T14:56:36.353 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.345+0000 7f4ebb7fe640 1 -- 172.21.15.173:0/1448880053 learned_addr learned my addr 172.21.15.173:0/1448880053 (peer_addr_for_me v2:172.21.15.173:0/0) 2024-03-23T14:56:36.353 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.345+0000 7f4ebbfff640 1 --2- 172.21.15.173:0/1448880053 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4ebc14a880 0x7f4ebc0b50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:36.353 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.345+0000 7f4ebb7fe640 1 -- 172.21.15.173:0/1448880053 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4ebc06e8e0 msgr2=0x7f4ebc1dfac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:56:36.354 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.345+0000 7f4ebb7fe640 1 --2- 172.21.15.173:0/1448880053 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4ebc06e8e0 0x7f4ebc1dfac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:36.354 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.345+0000 7f4ebb7fe640 1 -- 172.21.15.173:0/1448880053 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4ebc14a880 msgr2=0x7f4ebc0b50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:56:36.354 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.345+0000 7f4ebb7fe640 1 --2- 172.21.15.173:0/1448880053 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4ebc14a880 0x7f4ebc0b50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:36.354 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.345+0000 7f4ebb7fe640 1 -- 172.21.15.173:0/1448880053 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ea40094d0 con 0x7f4ebc02c2a0 2024-03-23T14:56:36.354 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.345+0000 7f4ebbfff640 1 --2- 172.21.15.173:0/1448880053 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4ebc14a880 0x7f4ebc0b50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T14:56:36.354 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.345+0000 7f4eb2ffd640 1 --2- 172.21.15.173:0/1448880053 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4ebc06e8e0 0x7f4ebc1dfac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T14:56:36.354 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.345+0000 7f4ebb7fe640 1 --2- 172.21.15.173:0/1448880053 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4ebc02c2a0 0x7f4ebc149e60 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f4eac00d800 tx=0x7f4eac00dcc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:56:36.354 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.345+0000 7f4eb8ff9640 1 -- 172.21.15.173:0/1448880053 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eac00b6c0 con 0x7f4ebc02c2a0 2024-03-23T14:56:36.354 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.345+0000 7f4eb8ff9640 1 -- 172.21.15.173:0/1448880053 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4eac00b820 con 0x7f4ebc02c2a0 2024-03-23T14:56:36.354 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.345+0000 7f4ec1ea8640 1 -- 172.21.15.173:0/1448880053 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ebc1d0450 con 0x7f4ebc02c2a0 2024-03-23T14:56:36.354 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.345+0000 7f4eb8ff9640 1 -- 172.21.15.173:0/1448880053 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eac00bb90 con 0x7f4ebc02c2a0 2024-03-23T14:56:36.354 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.345+0000 7f4ec1ea8640 1 -- 172.21.15.173:0/1448880053 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ebc0b2ec0 con 0x7f4ebc02c2a0 2024-03-23T14:56:36.354 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.349+0000 7f4eb8ff9640 1 -- 172.21.15.173:0/1448880053 <== mon.2 v2:172.21.15.173:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f4eac00bcf0 con 0x7f4ebc02c2a0 2024-03-23T14:56:36.354 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.349+0000 7f4eb8ff9640 1 -- 172.21.15.173:0/1448880053 <== mon.2 v2:172.21.15.173:3301/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7f4eac014bb0 con 0x7f4ebc02c2a0 2024-03-23T14:56:36.355 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.349+0000 7f4ec1ea8640 1 -- 172.21.15.173:0/1448880053 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ebc0690a0 con 0x7f4ebc02c2a0 2024-03-23T14:56:36.355 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.349+0000 7f4eb8ff9640 1 -- 172.21.15.173:0/1448880053 <== mon.2 v2:172.21.15.173:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f4eac014e30 con 0x7f4ebc02c2a0 2024-03-23T14:56:36.379 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-03-23T14:56:36.373+0000 7f3fc8b03180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-23T14:56:36.421 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.417+0000 7f4ec1ea8640 1 -- 172.21.15.173:0/1448880053 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "osd new", "uuid": "6193bfd7-8432-4f7f-a117-a3fcee21af99", "id": 6} v 0) -- 0x7f4ebc06e460 con 0x7f4ebc02c2a0 2024-03-23T14:56:36.427 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.421+0000 7f4eb8ff9640 1 -- 172.21.15.173:0/1448880053 <== mon.2 v2:172.21.15.173:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "6193bfd7-8432-4f7f-a117-a3fcee21af99", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7f4eac0146f0 con 0x7f4ebc02c2a0 2024-03-23T14:56:36.427 INFO:teuthology.orchestra.run.smithi173.stdout:6 2024-03-23T14:56:36.429 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.425+0000 7f4ec1ea8640 1 -- 172.21.15.173:0/1448880053 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4ebc02c2a0 msgr2=0x7f4ebc149e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:56:36.429 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.425+0000 7f4ec1ea8640 1 --2- 172.21.15.173:0/1448880053 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4ebc02c2a0 0x7f4ebc149e60 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f4eac00d800 tx=0x7f4eac00dcc0 comp rx=0 tx=0).stop 2024-03-23T14:56:36.429 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.425+0000 7f4ec1ea8640 1 -- 172.21.15.173:0/1448880053 shutdown_connections 2024-03-23T14:56:36.429 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.425+0000 7f4ec1ea8640 1 --2- 172.21.15.173:0/1448880053 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4ebc14a880 0x7f4ebc0b50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:36.429 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.425+0000 7f4ec1ea8640 1 --2- 172.21.15.173:0/1448880053 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4ebc06e8e0 0x7f4ebc1dfac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:36.429 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.425+0000 7f4ec1ea8640 1 --2- 172.21.15.173:0/1448880053 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4ebc02c2a0 0x7f4ebc149e60 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:36.429 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.425+0000 7f4ec1ea8640 1 -- 172.21.15.173:0/1448880053 >> 172.21.15.173:0/1448880053 conn(0x7f4ebc0b7970 msgr2=0x7f4ebc140320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:56:36.429 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.425+0000 7f4ec1ea8640 1 -- 172.21.15.173:0/1448880053 shutdown_connections 2024-03-23T14:56:36.429 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.425+0000 7f4ec1ea8640 1 -- 172.21.15.173:0/1448880053 wait complete. 2024-03-23T14:56:36.436 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-03-23T14:56:36.433+0000 7f3fc8b03180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-23T14:56:36.444 DEBUG:teuthology.orchestra.run.smithi173:> sudo ceph --cluster ceph osd new b0f1b80a-afc4-4538-83b9-f081d2dcb8cb 7 2024-03-23T14:56:36.463 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-03-23T14:56:36.458+0000 7efcb3f00180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-23T14:56:36.523 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-03-23T14:56:36.518+0000 7efcb3f00180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-23T14:56:36.555 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.549+0000 7f7ede4a2640 1 Processor -- start 2024-03-23T14:56:36.555 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.549+0000 7f7ede4a2640 1 -- start start 2024-03-23T14:56:36.556 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.549+0000 7f7ede4a2640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f7ed80b9ac0 0x7f7ed80b0650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:36.556 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.549+0000 7f7ede4a2640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f7ed8145bb0 0x7f7ed8156220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:36.557 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.549+0000 7f7ede4a2640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ed80b2b80 con 0x7f7ed8145700 2024-03-23T14:56:36.557 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.549+0000 7f7ede4a2640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f7ed80b5990 con 0x7f7ed8145bb0 2024-03-23T14:56:36.557 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.549+0000 7f7ede4a2640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 -- 0x7f7ed80b2dd0 con 0x7f7ed80b9ac0 2024-03-23T14:56:36.557 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.553+0000 7f7ed7fff640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f7ed80b9ac0 0x7f7ed80b0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:36.557 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.553+0000 7f7ed7fff640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f7ed80b9ac0 0x7f7ed80b0650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.173:49654/0 (socket says 172.21.15.173:49654) 2024-03-23T14:56:36.557 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.553+0000 7f7ed7fff640 1 -- 172.21.15.173:0/1959175181 learned_addr learned my addr 172.21.15.173:0/1959175181 (peer_addr_for_me v2:172.21.15.173:0/0) 2024-03-23T14:56:36.557 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.553+0000 7f7edca3f640 1 --2- 172.21.15.173:0/1959175181 >> v2:172.21.15.164:3300/0 conn(0x7f7ed8145bb0 0x7f7ed8156220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:36.557 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.553+0000 7f7ed7fff640 1 -- 172.21.15.173:0/1959175181 >> v2:172.21.15.164:3300/0 conn(0x7f7ed8145bb0 msgr2=0x7f7ed8156220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:56:36.557 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.553+0000 7f7ed7fff640 1 --2- 172.21.15.173:0/1959175181 >> v2:172.21.15.164:3300/0 conn(0x7f7ed8145bb0 0x7f7ed8156220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:36.557 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.553+0000 7f7ed7fff640 1 -- 172.21.15.173:0/1959175181 >> v1:172.21.15.164:6789/0 conn(0x7f7ed8145700 legacy=0x7f7ed80b14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:56:36.557 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.553+0000 7f7ed7fff640 1 -- 172.21.15.173:0/1959175181 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ed8145fe0 con 0x7f7ed80b9ac0 2024-03-23T14:56:36.558 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.553+0000 7f7ed7fff640 1 --2- 172.21.15.173:0/1959175181 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7ed80b9ac0 0x7f7ed80b0650 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f7ed80ba1a0 tx=0x7f7ec801c4e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b3abfa2ef0e38e3 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:56:36.558 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.553+0000 7f7ed77fe640 1 -- 172.21.15.173:0/1959175181 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec80ac630 con 0x7f7ed80b9ac0 2024-03-23T14:56:36.559 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.553+0000 7f7ed77fe640 1 -- 172.21.15.173:0/1959175181 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ec80ac790 con 0x7f7ed80b9ac0 2024-03-23T14:56:36.559 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.553+0000 7f7ed77fe640 1 -- 172.21.15.173:0/1959175181 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec80acb20 con 0x7f7ed80b9ac0 2024-03-23T14:56:36.559 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.553+0000 7f7ede4a2640 1 -- 172.21.15.173:0/1959175181 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7ed80b9ac0 msgr2=0x7f7ed80b0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:56:36.559 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.553+0000 7f7ede4a2640 1 --2- 172.21.15.173:0/1959175181 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7ed80b9ac0 0x7f7ed80b0650 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f7ed80ba1a0 tx=0x7f7ec801c4e0 comp rx=0 tx=0).stop 2024-03-23T14:56:36.559 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.553+0000 7f7ede4a2640 1 -- 172.21.15.173:0/1959175181 shutdown_connections 2024-03-23T14:56:36.559 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.553+0000 7f7ede4a2640 1 --2- 172.21.15.173:0/1959175181 >> v2:172.21.15.164:3300/0 conn(0x7f7ed8145bb0 0x7f7ed8156220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:36.559 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.553+0000 7f7ede4a2640 1 --2- 172.21.15.173:0/1959175181 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7ed80b9ac0 0x7f7ed80b0650 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:36.560 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.553+0000 7f7ede4a2640 1 -- 172.21.15.173:0/1959175181 >> 172.21.15.173:0/1959175181 conn(0x7f7ed806b930 msgr2=0x7f7ed80b9150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:56:36.560 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.553+0000 7f7ede4a2640 1 -- 172.21.15.173:0/1959175181 shutdown_connections 2024-03-23T14:56:36.560 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.553+0000 7f7ede4a2640 1 -- 172.21.15.173:0/1959175181 wait complete. 2024-03-23T14:56:36.560 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.557+0000 7f7ede4a2640 1 Processor -- start 2024-03-23T14:56:36.560 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.557+0000 7f7ede4a2640 1 -- start start 2024-03-23T14:56:36.562 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.557+0000 7f7ede4a2640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7ed80b9ac0 0x7f7ed8144af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:36.562 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.557+0000 7f7ede4a2640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7ed8145700 0x7f7ed8070270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:36.562 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.557+0000 7f7ede4a2640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7ed8145bb0 0x7f7ed81e79a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:36.562 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.557+0000 7f7ede4a2640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f7ed8139640 con 0x7f7ed8145bb0 2024-03-23T14:56:36.562 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.557+0000 7f7ede4a2640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f7ed806fdb0 con 0x7f7ed8145700 2024-03-23T14:56:36.562 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.557+0000 7f7ede4a2640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f7ed806fa80 con 0x7f7ed80b9ac0 2024-03-23T14:56:36.562 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.557+0000 7f7ed7fff640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7ed80b9ac0 0x7f7ed8144af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:36.562 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.557+0000 7f7ecffff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7ed8145700 0x7f7ed8070270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:36.562 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.557+0000 7f7ed7fff640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7ed80b9ac0 0x7f7ed8144af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.173:56978/0 (socket says 172.21.15.173:56978) 2024-03-23T14:56:36.562 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.557+0000 7f7ed7fff640 1 -- 172.21.15.173:0/2834601794 learned_addr learned my addr 172.21.15.173:0/2834601794 (peer_addr_for_me v2:172.21.15.173:0/0) 2024-03-23T14:56:36.562 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.557+0000 7f7edca3f640 1 --2- 172.21.15.173:0/2834601794 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7ed8145bb0 0x7f7ed81e79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:36.563 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.557+0000 7f7ed7fff640 1 -- 172.21.15.173:0/2834601794 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7ed8145700 msgr2=0x7f7ed8070270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:56:36.563 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.557+0000 7f7ed7fff640 1 --2- 172.21.15.173:0/2834601794 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7ed8145700 0x7f7ed8070270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:36.563 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.557+0000 7f7ed7fff640 1 -- 172.21.15.173:0/2834601794 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7ed8145bb0 msgr2=0x7f7ed81e79a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:56:36.563 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.557+0000 7f7ed7fff640 1 --2- 172.21.15.173:0/2834601794 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7ed8145bb0 0x7f7ed81e79a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:36.563 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.557+0000 7f7ed7fff640 1 -- 172.21.15.173:0/2834601794 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ec80094d0 con 0x7f7ed80b9ac0 2024-03-23T14:56:36.563 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.557+0000 7f7ed7fff640 1 --2- 172.21.15.173:0/2834601794 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7ed80b9ac0 0x7f7ed8144af0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f7ec801ab90 tx=0x7f7ec8019280 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:56:36.563 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.557+0000 7f7ed57fa640 1 -- 172.21.15.173:0/2834601794 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec80b5850 con 0x7f7ed80b9ac0 2024-03-23T14:56:36.563 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.557+0000 7f7ed57fa640 1 -- 172.21.15.173:0/2834601794 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ec80b59b0 con 0x7f7ed80b9ac0 2024-03-23T14:56:36.563 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.557+0000 7f7ede4a2640 1 -- 172.21.15.173:0/2834601794 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ed8070ff0 con 0x7f7ed80b9ac0 2024-03-23T14:56:36.563 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.557+0000 7f7ed57fa640 1 -- 172.21.15.173:0/2834601794 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec80b5d40 con 0x7f7ed80b9ac0 2024-03-23T14:56:36.563 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.557+0000 7f7ede4a2640 1 -- 172.21.15.173:0/2834601794 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ed80707b0 con 0x7f7ed80b9ac0 2024-03-23T14:56:36.564 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.557+0000 7f7ed57fa640 1 -- 172.21.15.173:0/2834601794 <== mon.2 v2:172.21.15.173:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7ec80b3070 con 0x7f7ed80b9ac0 2024-03-23T14:56:36.564 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.557+0000 7f7ede4a2640 1 -- 172.21.15.173:0/2834601794 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ed80b2300 con 0x7f7ed80b9ac0 2024-03-23T14:56:36.564 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.557+0000 7f7ed57fa640 1 -- 172.21.15.173:0/2834601794 <== mon.2 v2:172.21.15.173:3301/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7f7ec80abd00 con 0x7f7ed80b9ac0 2024-03-23T14:56:36.564 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.561+0000 7f7ed57fa640 1 -- 172.21.15.173:0/2834601794 <== mon.2 v2:172.21.15.173:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f7ec80b9020 con 0x7f7ed80b9ac0 2024-03-23T14:56:36.591 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-03-23T14:56:36.585+0000 7f3fc8b03180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-23T14:56:36.628 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.625+0000 7f7ede4a2640 1 -- 172.21.15.173:0/2834601794 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "osd new", "uuid": "b0f1b80a-afc4-4538-83b9-f081d2dcb8cb", "id": 7} v 0) -- 0x7f7ed806f8e0 con 0x7f7ed80b9ac0 2024-03-23T14:56:36.637 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.633+0000 7f7ed57fa640 1 -- 172.21.15.173:0/2834601794 <== mon.2 v2:172.21.15.173:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "b0f1b80a-afc4-4538-83b9-f081d2dcb8cb", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7f7ed806f8e0 con 0x7f7ed80b9ac0 2024-03-23T14:56:36.637 INFO:teuthology.orchestra.run.smithi173.stdout:7 2024-03-23T14:56:36.638 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.633+0000 7f7ede4a2640 1 -- 172.21.15.173:0/2834601794 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7ed80b9ac0 msgr2=0x7f7ed8144af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:56:36.638 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.633+0000 7f7ede4a2640 1 --2- 172.21.15.173:0/2834601794 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7ed80b9ac0 0x7f7ed8144af0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f7ec801ab90 tx=0x7f7ec8019280 comp rx=0 tx=0).stop 2024-03-23T14:56:36.639 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.633+0000 7f7ede4a2640 1 -- 172.21.15.173:0/2834601794 shutdown_connections 2024-03-23T14:56:36.639 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.633+0000 7f7ede4a2640 1 --2- 172.21.15.173:0/2834601794 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7ed8145bb0 0x7f7ed81e79a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:36.639 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.633+0000 7f7ede4a2640 1 --2- 172.21.15.173:0/2834601794 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7ed8145700 0x7f7ed8070270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:36.639 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.633+0000 7f7ede4a2640 1 --2- 172.21.15.173:0/2834601794 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7ed80b9ac0 0x7f7ed8144af0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:36.639 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.633+0000 7f7ede4a2640 1 -- 172.21.15.173:0/2834601794 >> 172.21.15.173:0/2834601794 conn(0x7f7ed806b930 msgr2=0x7f7ed814a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:56:36.639 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.633+0000 7f7ede4a2640 1 -- 172.21.15.173:0/2834601794 shutdown_connections 2024-03-23T14:56:36.639 INFO:teuthology.orchestra.run.smithi173.stderr:2024-03-23T14:56:36.633+0000 7f7ede4a2640 1 -- 172.21.15.173:0/2834601794 wait complete. 2024-03-23T14:56:36.654 INFO:tasks.ceph.osd.0:Restarting daemon 2024-03-23T14:56:36.654 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-03-23T14:56:36.655 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-03-23T14:56:36.649+0000 7f3fc8b03180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-23T14:56:36.656 INFO:tasks.ceph.osd.0:Started 2024-03-23T14:56:36.657 INFO:tasks.ceph.osd.1:Restarting daemon 2024-03-23T14:56:36.657 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-03-23T14:56:36.659 INFO:tasks.ceph.osd.1:Started 2024-03-23T14:56:36.659 INFO:tasks.ceph.osd.2:Restarting daemon 2024-03-23T14:56:36.659 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-03-23T14:56:36.661 INFO:tasks.ceph.osd.2:Started 2024-03-23T14:56:36.662 INFO:tasks.ceph.osd.3:Restarting daemon 2024-03-23T14:56:36.662 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-03-23T14:56:36.664 INFO:tasks.ceph.osd.3:Started 2024-03-23T14:56:36.664 INFO:tasks.ceph.osd.4:Restarting daemon 2024-03-23T14:56:36.664 DEBUG:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-03-23T14:56:36.667 INFO:tasks.ceph.osd.4:Started 2024-03-23T14:56:36.667 INFO:tasks.ceph.osd.5:Restarting daemon 2024-03-23T14:56:36.667 DEBUG:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-03-23T14:56:36.669 INFO:tasks.ceph.osd.5:Started 2024-03-23T14:56:36.669 INFO:tasks.ceph.osd.6:Restarting daemon 2024-03-23T14:56:36.669 DEBUG:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-03-23T14:56:36.671 INFO:tasks.ceph.osd.6:Started 2024-03-23T14:56:36.671 INFO:tasks.ceph.osd.7:Restarting daemon 2024-03-23T14:56:36.671 DEBUG:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-03-23T14:56:36.673 INFO:tasks.ceph.osd.7:Started 2024-03-23T14:56:36.673 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T14:56:36.798 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:36.794+0000 7f280fea4640 1 Processor -- start 2024-03-23T14:56:36.798 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:36.794+0000 7f280fea4640 1 -- start start 2024-03-23T14:56:36.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:36.794+0000 7f280fea4640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f2808147890 0x7f28080b0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:36.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:36.794+0000 7f280fea4640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f28080707c0 0x7f2808070c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:36.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:36.794+0000 7f280fea4640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f280813df90 0x7f280813e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:36.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:36.794+0000 7f280fea4640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f28080705c0 con 0x7f28080707c0 2024-03-23T14:56:36.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:36.794+0000 7f280fea4640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 -- 0x7f2808148550 con 0x7f2808147890 2024-03-23T14:56:36.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:36.794+0000 7f280fea4640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f28081480c0 con 0x7f280813df90 2024-03-23T14:56:36.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:36.794+0000 7f280d43f640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f28080707c0 0x7f2808070c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:36.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:36.794+0000 7f280d43f640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f28080707c0 0x7f2808070c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:37586/0 (socket says 172.21.15.164:37586) 2024-03-23T14:56:36.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:36.794+0000 7f280d43f640 1 -- 172.21.15.164:0/4170365403 learned_addr learned my addr 172.21.15.164:0/4170365403 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:56:36.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:36.794+0000 7f280dc40640 1 --2- 172.21.15.164:0/4170365403 >> v2:172.21.15.173:3300/0 conn(0x7f2808147890 0x7f28080b0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:36.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:36.794+0000 7f280e441640 1 --2- 172.21.15.164:0/4170365403 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f280813df90 0x7f280813e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:36.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:36.794+0000 7f280d43f640 1 -- 172.21.15.164:0/4170365403 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f280813df90 msgr2=0x7f280813e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:56:36.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:36.794+0000 7f280d43f640 1 --2- 172.21.15.164:0/4170365403 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f280813df90 0x7f280813e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:36.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:36.794+0000 7f280d43f640 1 -- 172.21.15.164:0/4170365403 >> v2:172.21.15.173:3300/0 conn(0x7f2808147890 msgr2=0x7f28080b0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:56:36.801 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:36.794+0000 7f280d43f640 1 --2- 172.21.15.164:0/4170365403 >> v2:172.21.15.173:3300/0 conn(0x7f2808147890 0x7f28080b0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:36.801 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:36.794+0000 7f280d43f640 1 -- 172.21.15.164:0/4170365403 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28080725c0 con 0x7f28080707c0 2024-03-23T14:56:36.801 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:36.798+0000 7f280d43f640 1 --2- 172.21.15.164:0/4170365403 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f28080707c0 0x7f2808070c30 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f2808072750 tx=0x7f27fc01c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79a0e0607b5692f server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:56:36.801 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:36.798+0000 7f280cc3e640 1 -- 172.21.15.164:0/4170365403 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27fc0ab600 con 0x7f28080707c0 2024-03-23T14:56:36.801 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:36.798+0000 7f280cc3e640 1 -- 172.21.15.164:0/4170365403 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27fc0a7050 con 0x7f28080707c0 2024-03-23T14:56:36.801 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:36.798+0000 7f280cc3e640 1 -- 172.21.15.164:0/4170365403 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27fc0ab980 con 0x7f28080707c0 2024-03-23T14:56:36.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:36.798+0000 7f280fea4640 1 -- 172.21.15.164:0/4170365403 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f28080707c0 msgr2=0x7f2808070c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:56:36.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:36.798+0000 7f280fea4640 1 --2- 172.21.15.164:0/4170365403 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f28080707c0 0x7f2808070c30 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f2808072750 tx=0x7f27fc01c3e0 comp rx=0 tx=0).stop 2024-03-23T14:56:36.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:36.798+0000 7f280fea4640 1 -- 172.21.15.164:0/4170365403 shutdown_connections 2024-03-23T14:56:36.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:36.798+0000 7f280fea4640 1 --2- 172.21.15.164:0/4170365403 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f280813df90 0x7f280813e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:36.803 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:36.798+0000 7f280fea4640 1 --2- 172.21.15.164:0/4170365403 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f28080707c0 0x7f2808070c30 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:36.803 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:36.798+0000 7f280fea4640 1 --2- 172.21.15.164:0/4170365403 >> v2:172.21.15.173:3300/0 conn(0x7f2808147890 0x7f28080b0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:36.803 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:36.798+0000 7f280fea4640 1 -- 172.21.15.164:0/4170365403 >> 172.21.15.164:0/4170365403 conn(0x7f28080b7970 msgr2=0x7f28080b7d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:56:36.803 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:36.798+0000 7f280fea4640 1 -- 172.21.15.164:0/4170365403 shutdown_connections 2024-03-23T14:56:36.803 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:36.798+0000 7f280fea4640 1 -- 172.21.15.164:0/4170365403 wait complete. 2024-03-23T14:56:36.805 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:36.798+0000 7f280fea4640 1 Processor -- start 2024-03-23T14:56:36.805 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:36.798+0000 7f280fea4640 1 -- start start 2024-03-23T14:56:36.805 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:36.798+0000 7f280fea4640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f28080707c0 0x7f28081e0f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:36.805 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:36.798+0000 7f280fea4640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f280813df90 0x7f28081e1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:36.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:36.798+0000 7f280fea4640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2808147890 0x7f280813e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:36.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:36.798+0000 7f280fea4640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2808141670 con 0x7f280813df90 2024-03-23T14:56:36.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:36.798+0000 7f280fea4640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f280814e450 con 0x7f28080707c0 2024-03-23T14:56:36.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:36.798+0000 7f280fea4640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f2808140620 con 0x7f2808147890 2024-03-23T14:56:36.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:36.798+0000 7f280d43f640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f280813df90 0x7f28081e1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:36.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:36.798+0000 7f280d43f640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f280813df90 0x7f28081e1690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:37590/0 (socket says 172.21.15.164:37590) 2024-03-23T14:56:36.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:36.798+0000 7f280d43f640 1 -- 172.21.15.164:0/4257200156 learned_addr learned my addr 172.21.15.164:0/4257200156 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:56:36.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:36.802+0000 7f280dc40640 1 --2- 172.21.15.164:0/4257200156 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f28080707c0 0x7f28081e0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:36.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:36.802+0000 7f280e441640 1 --2- 172.21.15.164:0/4257200156 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2808147890 0x7f280813e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:36.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:36.802+0000 7f280d43f640 1 -- 172.21.15.164:0/4257200156 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2808147890 msgr2=0x7f280813e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:56:36.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:36.802+0000 7f280d43f640 1 --2- 172.21.15.164:0/4257200156 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2808147890 0x7f280813e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:36.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:36.802+0000 7f280d43f640 1 -- 172.21.15.164:0/4257200156 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f28080707c0 msgr2=0x7f28081e0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:56:36.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:36.802+0000 7f280d43f640 1 --2- 172.21.15.164:0/4257200156 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f28080707c0 0x7f28081e0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:36.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:36.802+0000 7f280d43f640 1 -- 172.21.15.164:0/4257200156 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27fc0094d0 con 0x7f280813df90 2024-03-23T14:56:36.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:36.802+0000 7f280e441640 1 --2- 172.21.15.164:0/4257200156 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2808147890 0x7f280813e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T14:56:36.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:36.802+0000 7f280d43f640 1 --2- 172.21.15.164:0/4257200156 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f280813df90 0x7f28081e1690 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f27fc0ae040 tx=0x7f28081450d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:56:36.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:36.802+0000 7f27fa7fc640 1 -- 172.21.15.164:0/4257200156 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27fc0b8070 con 0x7f280813df90 2024-03-23T14:56:36.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:36.802+0000 7f27fa7fc640 1 -- 172.21.15.164:0/4257200156 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27fc0b4460 con 0x7f280813df90 2024-03-23T14:56:36.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:36.802+0000 7f280fea4640 1 -- 172.21.15.164:0/4257200156 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2808141950 con 0x7f280813df90 2024-03-23T14:56:36.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:36.802+0000 7f27fa7fc640 1 -- 172.21.15.164:0/4257200156 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27fc0b4810 con 0x7f280813df90 2024-03-23T14:56:36.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:36.802+0000 7f280fea4640 1 -- 172.21.15.164:0/4257200156 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2808070c30 con 0x7f280813df90 2024-03-23T14:56:36.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:36.802+0000 7f27fa7fc640 1 -- 172.21.15.164:0/4257200156 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f27fc0b2070 con 0x7f280813df90 2024-03-23T14:56:36.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:36.802+0000 7f27fa7fc640 1 -- 172.21.15.164:0/4257200156 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f27fc0bde10 con 0x7f280813df90 2024-03-23T14:56:36.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:36.802+0000 7f280fea4640 1 -- 172.21.15.164:0/4257200156 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28080690a0 con 0x7f280813df90 2024-03-23T14:56:36.816 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:36.814+0000 7f27fa7fc640 1 -- 172.21.15.164:0/4257200156 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f27fc0b0070 con 0x7f280813df90 2024-03-23T14:56:36.917 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:36.914+0000 7f280fea4640 1 -- 172.21.15.164:0/4257200156 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2808072000 con 0x7f280813df90 2024-03-23T14:56:36.918 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:36.914+0000 7f27fa7fc640 1 -- 172.21.15.164:0/4257200156 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f27fc0cb090 con 0x7f280813df90 2024-03-23T14:56:36.918 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-23T14:56:36.918 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":10,"fsid":"45c7911b-b69d-4e68-907b-fbfeb780aa3b","created":"2024-03-23T14:56:34.367633+0000","modified":"2024-03-23T14:56:36.629884+0000","last_up_change":"0.000000","last_in_change":"2024-03-23T14:56:36.629884+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"80abdc62-bd0d-4e60-a554-7ed6fd67cf54","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"f9001b06-be4d-4f42-a0e6-bcef2afa0d6d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"fa0aeb9f-8b54-4831-a188-3e361c7980d7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"7e18c957-3091-46d5-8e17-8f3354c1726c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"8e83f315-42bf-4471-9ce8-d81948da0f9f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"a230c872-71f9-40ab-8999-defd0162d4ed","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"6193bfd7-8432-4f7f-a117-a3fcee21af99","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"b0f1b80a-afc4-4538-83b9-f081d2dcb8cb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T14:56:36.919 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:36.914+0000 7f280fea4640 1 -- 172.21.15.164:0/4257200156 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f280813df90 msgr2=0x7f28081e1690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:56:36.919 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:36.914+0000 7f280fea4640 1 --2- 172.21.15.164:0/4257200156 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f280813df90 0x7f28081e1690 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f27fc0ae040 tx=0x7f28081450d0 comp rx=0 tx=0).stop 2024-03-23T14:56:36.919 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:36.914+0000 7f280fea4640 1 -- 172.21.15.164:0/4257200156 shutdown_connections 2024-03-23T14:56:36.919 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:36.914+0000 7f280fea4640 1 --2- 172.21.15.164:0/4257200156 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2808147890 0x7f280813e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:36.919 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:36.914+0000 7f280fea4640 1 --2- 172.21.15.164:0/4257200156 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f280813df90 0x7f28081e1690 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:36.920 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:36.914+0000 7f280fea4640 1 --2- 172.21.15.164:0/4257200156 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f28080707c0 0x7f28081e0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:36.920 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:36.914+0000 7f280fea4640 1 -- 172.21.15.164:0/4257200156 >> 172.21.15.164:0/4257200156 conn(0x7f28080b7970 msgr2=0x7f2808143950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:56:36.920 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:36.914+0000 7f280fea4640 1 -- 172.21.15.164:0/4257200156 shutdown_connections 2024-03-23T14:56:36.920 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:36.914+0000 7f280fea4640 1 -- 172.21.15.164:0/4257200156 wait complete. 2024-03-23T14:56:36.934 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-03-23T14:56:36.934 INFO:tasks.ceph:Waiting for OSDs to come up 2024-03-23T14:56:37.197 INFO:tasks.ceph.mgr.y.smithi164.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-03-23T14:56:37.197 INFO:tasks.ceph.mgr.y.smithi164.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-03-23T14:56:37.197 INFO:tasks.ceph.mgr.y.smithi164.stderr: from numpy import show_config as show_numpy_config 2024-03-23T14:56:37.270 INFO:tasks.ceph.osd.0.smithi164.stderr:2024-03-23T14:56:37.266+0000 7f9ce3f927c0 -1 Falling back to public interface 2024-03-23T14:56:37.271 INFO:tasks.ceph.osd.2.smithi164.stderr:2024-03-23T14:56:37.266+0000 7f1f7673a7c0 -1 Falling back to public interface 2024-03-23T14:56:37.273 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-03-23T14:56:37.270+0000 7efcb3f00180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-23T14:56:37.285 INFO:tasks.ceph.osd.1.smithi164.stderr:2024-03-23T14:56:37.282+0000 7f113b4317c0 -1 Falling back to public interface 2024-03-23T14:56:37.293 INFO:tasks.ceph.osd.3.smithi164.stderr:2024-03-23T14:56:37.290+0000 7ffa748227c0 -1 Falling back to public interface 2024-03-23T14:56:37.294 INFO:tasks.ceph.osd.5.smithi173.stderr:2024-03-23T14:56:37.289+0000 7f9f944187c0 -1 Falling back to public interface 2024-03-23T14:56:37.297 INFO:tasks.ceph.osd.6.smithi173.stderr:2024-03-23T14:56:37.293+0000 7f9ead26b7c0 -1 Falling back to public interface 2024-03-23T14:56:37.305 INFO:tasks.ceph.osd.7.smithi173.stderr:2024-03-23T14:56:37.301+0000 7f7db1d7b7c0 -1 Falling back to public interface 2024-03-23T14:56:37.321 INFO:tasks.ceph.osd.4.smithi173.stderr:2024-03-23T14:56:37.317+0000 7ff3cdeb67c0 -1 Falling back to public interface 2024-03-23T14:56:37.332 INFO:tasks.ceph.mgr.x.smithi173.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-03-23T14:56:37.332 INFO:tasks.ceph.mgr.x.smithi173.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-03-23T14:56:37.332 INFO:tasks.ceph.mgr.x.smithi173.stderr: from numpy import show_config as show_numpy_config 2024-03-23T14:56:37.371 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-03-23T14:56:37.365+0000 7f3fc8b03180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-23T14:56:37.425 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-03-23T14:56:37.422+0000 7efcb3f00180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-23T14:56:37.507 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-03-23T14:56:37.501+0000 7f3fc8b03180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-23T14:56:37.571 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-03-23T14:56:37.566+0000 7efcb3f00180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-23T14:56:37.635 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-03-23T14:56:37.630+0000 7efcb3f00180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-23T14:56:37.673 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-03-23T14:56:37.669+0000 7f3fc8b03180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-23T14:56:37.736 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-03-23T14:56:37.729+0000 7f3fc8b03180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-23T14:56:37.740 DEBUG:teuthology.orchestra.run.smithi164:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-03-23T14:56:37.768 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-03-23T14:56:37.766+0000 7efcb3f00180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-23T14:56:37.853 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:37.850+0000 7f9430fc4640 1 Processor -- start 2024-03-23T14:56:37.853 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:37.850+0000 7f9430fc4640 1 -- start start 2024-03-23T14:56:37.854 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:37.850+0000 7f9430fc4640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f942c0b7700 0x7f942c1413f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:37.854 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:37.850+0000 7f9430fc4640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f942c0b7ae0 con 0x7f942c071250 2024-03-23T14:56:37.854 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:37.850+0000 7f9430fc4640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f942c0b5b50 con 0x7f942c14fc90 2024-03-23T14:56:37.854 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:37.850+0000 7f9430fc4640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f942c0721c0 con 0x7f942c0b7700 2024-03-23T14:56:37.854 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:37.850+0000 7f942a59c640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f942c0b7700 0x7f942c1413f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:37.854 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:37.850+0000 7f942a59c640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f942c0b7700 0x7f942c1413f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:37602/0 (socket says 172.21.15.164:37602) 2024-03-23T14:56:37.854 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:37.850+0000 7f942a59c640 1 -- 172.21.15.164:0/2433890369 learned_addr learned my addr 172.21.15.164:0/2433890369 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:56:37.854 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:37.850+0000 7f942a59c640 1 -- 172.21.15.164:0/2433890369 >> v1:172.21.15.173:6789/0 conn(0x7f942c14fc90 legacy=0x7f942c152080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:56:37.854 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:37.850+0000 7f942a59c640 1 -- 172.21.15.164:0/2433890369 >> v1:172.21.15.164:6789/0 conn(0x7f942c071250 legacy=0x7f942c0b9170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:56:37.854 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:37.850+0000 7f942a59c640 1 -- 172.21.15.164:0/2433890369 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f942c0716e0 con 0x7f942c0b7700 2024-03-23T14:56:37.856 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:37.854+0000 7f942a59c640 1 --2- 172.21.15.164:0/2433890369 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f942c0b7700 0x7f942c1413f0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f941c009870 tx=0x7f941c01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=184dc1475061cfc0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:56:37.856 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:37.854+0000 7f942959a640 1 -- 172.21.15.164:0/2433890369 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f941c0ad680 con 0x7f942c0b7700 2024-03-23T14:56:37.856 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:37.854+0000 7f942959a640 1 -- 172.21.15.164:0/2433890369 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f941c0ad7e0 con 0x7f942c0b7700 2024-03-23T14:56:37.856 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:37.854+0000 7f942959a640 1 -- 172.21.15.164:0/2433890369 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f941c0adb70 con 0x7f942c0b7700 2024-03-23T14:56:37.857 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:37.854+0000 7f9430fc4640 1 -- 172.21.15.164:0/2433890369 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f942c0b7700 msgr2=0x7f942c1413f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:56:37.857 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:37.854+0000 7f9430fc4640 1 --2- 172.21.15.164:0/2433890369 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f942c0b7700 0x7f942c1413f0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f941c009870 tx=0x7f941c01c490 comp rx=0 tx=0).stop 2024-03-23T14:56:37.857 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:37.854+0000 7f9430fc4640 1 -- 172.21.15.164:0/2433890369 shutdown_connections 2024-03-23T14:56:37.857 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:37.854+0000 7f9430fc4640 1 --2- 172.21.15.164:0/2433890369 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f942c0b7700 0x7f942c1413f0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:37.857 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:37.854+0000 7f9430fc4640 1 -- 172.21.15.164:0/2433890369 >> 172.21.15.164:0/2433890369 conn(0x7f942c06f060 msgr2=0x7f942c06f470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:56:37.858 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:37.854+0000 7f9430fc4640 1 -- 172.21.15.164:0/2433890369 shutdown_connections 2024-03-23T14:56:37.858 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:37.854+0000 7f9430fc4640 1 -- 172.21.15.164:0/2433890369 wait complete. 2024-03-23T14:56:37.860 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:37.854+0000 7f9430fc4640 1 Processor -- start 2024-03-23T14:56:37.860 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:37.854+0000 7f9430fc4640 1 -- start start 2024-03-23T14:56:37.860 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:37.854+0000 7f9430fc4640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f942c0b7700 0x7f942c1e34f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:37.860 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:37.854+0000 7f9430fc4640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f942c14fc90 0x7f942c150070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:37.860 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:37.854+0000 7f9430fc4640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f942c152080 0x7f942c13c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:37.860 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:37.854+0000 7f9430fc4640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f942c144360 con 0x7f942c14fc90 2024-03-23T14:56:37.860 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:37.854+0000 7f9430fc4640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f942c1e3bb0 con 0x7f942c0b7700 2024-03-23T14:56:37.861 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:37.854+0000 7f9430fc4640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f942c152590 con 0x7f942c152080 2024-03-23T14:56:37.861 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:37.854+0000 7f9429d9b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f942c14fc90 0x7f942c150070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:37.861 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:37.854+0000 7f9429d9b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f942c14fc90 0x7f942c150070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:37608/0 (socket says 172.21.15.164:37608) 2024-03-23T14:56:37.861 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:37.854+0000 7f9429d9b640 1 -- 172.21.15.164:0/261034113 learned_addr learned my addr 172.21.15.164:0/261034113 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:56:37.861 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:37.854+0000 7f942a59c640 1 --2- 172.21.15.164:0/261034113 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f942c0b7700 0x7f942c1e34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:37.861 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:37.854+0000 7f942ad9d640 1 --2- 172.21.15.164:0/261034113 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f942c152080 0x7f942c13c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:37.861 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:37.854+0000 7f9429d9b640 1 -- 172.21.15.164:0/261034113 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f942c152080 msgr2=0x7f942c13c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:56:37.861 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:37.854+0000 7f9429d9b640 1 --2- 172.21.15.164:0/261034113 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f942c152080 0x7f942c13c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:37.861 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:37.854+0000 7f9429d9b640 1 -- 172.21.15.164:0/261034113 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f942c0b7700 msgr2=0x7f942c1e34f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:56:37.861 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:37.854+0000 7f9429d9b640 1 --2- 172.21.15.164:0/261034113 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f942c0b7700 0x7f942c1e34f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:37.861 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:37.854+0000 7f9429d9b640 1 -- 172.21.15.164:0/261034113 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f941c0094d0 con 0x7f942c14fc90 2024-03-23T14:56:37.861 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:37.854+0000 7f9429d9b640 1 --2- 172.21.15.164:0/261034113 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f942c14fc90 0x7f942c150070 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f941400c370 tx=0x7f941400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:56:37.861 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:37.854+0000 7f9422ffd640 1 -- 172.21.15.164:0/261034113 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94140052d0 con 0x7f942c14fc90 2024-03-23T14:56:37.861 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:37.854+0000 7f9422ffd640 1 -- 172.21.15.164:0/261034113 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9414005430 con 0x7f942c14fc90 2024-03-23T14:56:37.861 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:37.854+0000 7f9430fc4640 1 -- 172.21.15.164:0/261034113 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f942c0b9a00 con 0x7f942c14fc90 2024-03-23T14:56:37.862 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:37.854+0000 7f9422ffd640 1 -- 172.21.15.164:0/261034113 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9414007260 con 0x7f942c14fc90 2024-03-23T14:56:37.862 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:37.854+0000 7f9430fc4640 1 -- 172.21.15.164:0/261034113 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f942c14a650 con 0x7f942c14fc90 2024-03-23T14:56:37.862 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:37.854+0000 7f9422ffd640 1 -- 172.21.15.164:0/261034113 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f94140073c0 con 0x7f942c14fc90 2024-03-23T14:56:37.862 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:37.854+0000 7f9422ffd640 1 -- 172.21.15.164:0/261034113 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f9414011c60 con 0x7f942c14fc90 2024-03-23T14:56:37.862 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:37.858+0000 7f9430fc4640 1 -- 172.21.15.164:0/261034113 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f942c0b72f0 con 0x7f942c14fc90 2024-03-23T14:56:37.862 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:37.858+0000 7f9422ffd640 1 -- 172.21.15.164:0/261034113 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f9414020060 con 0x7f942c14fc90 2024-03-23T14:56:37.867 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-03-23T14:56:37.861+0000 7f3fc8b03180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-23T14:56:37.920 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:37.918+0000 7f9430fc4640 1 -- 172.21.15.164:0/261034113 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f942c04f4f0 con 0x7f942c14fc90 2024-03-23T14:56:37.920 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:37.918+0000 7f9422ffd640 1 -- 172.21.15.164:0/261034113 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f9414003970 con 0x7f942c14fc90 2024-03-23T14:56:37.921 INFO:teuthology.misc.health.smithi164.stdout: 2024-03-23T14:56:37.921 INFO:teuthology.misc.health.smithi164.stdout:{"epoch":10,"fsid":"45c7911b-b69d-4e68-907b-fbfeb780aa3b","created":"2024-03-23T14:56:34.367633+0000","modified":"2024-03-23T14:56:36.629884+0000","last_up_change":"0.000000","last_in_change":"2024-03-23T14:56:36.629884+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"80abdc62-bd0d-4e60-a554-7ed6fd67cf54","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"f9001b06-be4d-4f42-a0e6-bcef2afa0d6d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"fa0aeb9f-8b54-4831-a188-3e361c7980d7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"7e18c957-3091-46d5-8e17-8f3354c1726c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"8e83f315-42bf-4471-9ce8-d81948da0f9f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"a230c872-71f9-40ab-8999-defd0162d4ed","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"6193bfd7-8432-4f7f-a117-a3fcee21af99","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"b0f1b80a-afc4-4538-83b9-f081d2dcb8cb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T14:56:37.922 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:37.918+0000 7f9430fc4640 1 -- 172.21.15.164:0/261034113 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f942c14fc90 msgr2=0x7f942c150070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:56:37.922 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:37.918+0000 7f9430fc4640 1 --2- 172.21.15.164:0/261034113 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f942c14fc90 0x7f942c150070 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f941400c370 tx=0x7f941400c830 comp rx=0 tx=0).stop 2024-03-23T14:56:37.922 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:37.918+0000 7f9430fc4640 1 -- 172.21.15.164:0/261034113 shutdown_connections 2024-03-23T14:56:37.922 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:37.918+0000 7f9430fc4640 1 --2- 172.21.15.164:0/261034113 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f942c152080 0x7f942c13c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:37.922 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:37.918+0000 7f9430fc4640 1 --2- 172.21.15.164:0/261034113 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f942c14fc90 0x7f942c150070 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:37.922 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:37.918+0000 7f9430fc4640 1 --2- 172.21.15.164:0/261034113 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f942c0b7700 0x7f942c1e34f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:37.922 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:37.918+0000 7f9430fc4640 1 -- 172.21.15.164:0/261034113 >> 172.21.15.164:0/261034113 conn(0x7f942c06f060 msgr2=0x7f942c0b8ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:56:37.922 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:37.918+0000 7f9430fc4640 1 -- 172.21.15.164:0/261034113 shutdown_connections 2024-03-23T14:56:37.922 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:37.918+0000 7f9430fc4640 1 -- 172.21.15.164:0/261034113 wait complete. 2024-03-23T14:56:37.935 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-03-23T14:56:38.399 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-03-23T14:56:38.394+0000 7efcb3f00180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-23T14:56:38.474 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-03-23T14:56:38.470+0000 7efcb3f00180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-23T14:56:38.488 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-03-23T14:56:38.481+0000 7f3fc8b03180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-23T14:56:38.563 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-03-23T14:56:38.557+0000 7f3fc8b03180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-23T14:56:38.654 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-03-23T14:56:38.650+0000 7efcb3f00180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-23T14:56:38.727 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-03-23T14:56:38.722+0000 7efcb3f00180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-23T14:56:38.748 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-03-23T14:56:38.741+0000 7f3fc8b03180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-23T14:56:38.822 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-03-23T14:56:38.817+0000 7f3fc8b03180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-23T14:56:39.015 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-03-23T14:56:39.010+0000 7efcb3f00180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-23T14:56:39.076 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-03-23T14:56:39.070+0000 7efcb3f00180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-23T14:56:39.113 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-03-23T14:56:39.109+0000 7f3fc8b03180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-23T14:56:39.179 INFO:tasks.ceph.mgr.x.smithi173.stderr:2024-03-23T14:56:39.173+0000 7f3fc8b03180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-23T14:56:40.112 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-03-23T14:56:40.110+0000 7efc5b349640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-03-23T14:56:40.112 INFO:tasks.ceph.mgr.y.smithi164.stderr:2024-03-23T14:56:40.110+0000 7efc5b349640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-03-23T14:56:41.110 INFO:tasks.ceph.osd.7.smithi173.stderr:2024-03-23T14:56:41.105+0000 7f7db1d7b7c0 -1 osd.7 0 log_to_monitors true 2024-03-23T14:56:41.121 INFO:tasks.ceph.osd.6.smithi173.stderr:2024-03-23T14:56:41.117+0000 7f9ead26b7c0 -1 osd.6 0 log_to_monitors true 2024-03-23T14:56:41.143 INFO:tasks.ceph.osd.5.smithi173.stderr:2024-03-23T14:56:41.137+0000 7f9f944187c0 -1 osd.5 0 log_to_monitors true 2024-03-23T14:56:41.145 INFO:tasks.ceph.osd.1.smithi164.stderr:2024-03-23T14:56:41.142+0000 7f113b4317c0 -1 osd.1 0 log_to_monitors true 2024-03-23T14:56:41.198 INFO:tasks.ceph.osd.3.smithi164.stderr:2024-03-23T14:56:41.194+0000 7ffa748227c0 -1 osd.3 0 log_to_monitors true 2024-03-23T14:56:41.219 INFO:tasks.ceph.osd.2.smithi164.stderr:2024-03-23T14:56:41.214+0000 7f1f7673a7c0 -1 osd.2 0 log_to_monitors true 2024-03-23T14:56:41.230 INFO:tasks.ceph.osd.0.smithi164.stderr:2024-03-23T14:56:41.226+0000 7f9ce3f927c0 -1 osd.0 0 log_to_monitors true 2024-03-23T14:56:41.237 INFO:tasks.ceph.osd.4.smithi173.stderr:2024-03-23T14:56:41.233+0000 7ff3cdeb67c0 -1 osd.4 0 log_to_monitors true 2024-03-23T14:56:44.742 DEBUG:teuthology.orchestra.run.smithi164:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-03-23T14:56:44.914 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:44.910+0000 7f5be2aa7640 1 Processor -- start 2024-03-23T14:56:44.915 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:44.910+0000 7f5be2aa7640 1 -- start start 2024-03-23T14:56:44.915 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:44.910+0000 7f5be2aa7640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5bd4006490 0x7f5bd4031560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:44.915 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:44.910+0000 7f5be2aa7640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5bd401d2e0 con 0x7f5bd401b8c0 2024-03-23T14:56:44.916 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:44.910+0000 7f5be2aa7640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5bd401c920 con 0x7f5bd4039e30 2024-03-23T14:56:44.916 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:44.910+0000 7f5be2aa7640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f5bd401d880 con 0x7f5bd4006490 2024-03-23T14:56:44.916 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:44.914+0000 7f5be1aa5640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5bd4006490 0x7f5bd4031560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:44.916 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:44.914+0000 7f5be22a6640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f5bd401b8c0 0x7f5bd40db3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:52612/0 (socket says 172.21.15.164:52612) 2024-03-23T14:56:44.916 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:44.914+0000 7f5be22a6640 1 -- 172.21.15.164:0/2998937635 learned_addr learned my addr 172.21.15.164:0/2998937635 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-03-23T14:56:44.917 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:44.914+0000 7f5be0aa3640 1 -- 172.21.15.164:0/2998937635 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2884373903 0 0) 0x7f5bd401c920 con 0x7f5bd4039e30 2024-03-23T14:56:44.917 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:44.914+0000 7f5be0aa3640 1 -- 172.21.15.164:0/2998937635 --> v1:172.21.15.173:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5bc8003530 con 0x7f5bd4039e30 2024-03-23T14:56:44.920 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:44.914+0000 7f5be0aa3640 1 -- 172.21.15.164:0/2998937635 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3394747044 0 0) 0x7f5bd401d2e0 con 0x7f5bd401b8c0 2024-03-23T14:56:44.921 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:44.914+0000 7f5be0aa3640 1 -- 172.21.15.164:0/2998937635 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5bd401c920 con 0x7f5bd401b8c0 2024-03-23T14:56:44.921 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:44.914+0000 7f5be1aa5640 1 -- 172.21.15.164:0/2998937635 >> v1:172.21.15.173:6789/0 conn(0x7f5bd4039e30 legacy=0x7f5bd40d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:56:44.921 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:44.914+0000 7f5be1aa5640 1 -- 172.21.15.164:0/2998937635 >> v1:172.21.15.164:6789/0 conn(0x7f5bd401b8c0 legacy=0x7f5bd40db3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:56:44.921 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:44.914+0000 7f5be1aa5640 1 -- 172.21.15.164:0/2998937635 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bd401cb70 con 0x7f5bd4006490 2024-03-23T14:56:44.921 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:44.918+0000 7f5be1aa5640 1 --2- 172.21.15.164:0/2998937635 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5bd4006490 0x7f5bd4031560 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc0097f0 tx=0x7f5bcc01c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78d34d268d90c4cf server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:56:44.921 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:44.918+0000 7f5be0aa3640 1 -- 172.21.15.164:0/2998937635 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bcc0a7070 con 0x7f5bd4006490 2024-03-23T14:56:44.921 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:44.918+0000 7f5be0aa3640 1 -- 172.21.15.164:0/2998937635 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bcc0ab430 con 0x7f5bd4006490 2024-03-23T14:56:44.921 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:44.918+0000 7f5be0aa3640 1 -- 172.21.15.164:0/2998937635 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bcc0ab7a0 con 0x7f5bd4006490 2024-03-23T14:56:44.922 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:44.918+0000 7f5be2aa7640 1 -- 172.21.15.164:0/2998937635 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5bd4006490 msgr2=0x7f5bd4031560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:56:44.922 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:44.918+0000 7f5be2aa7640 1 --2- 172.21.15.164:0/2998937635 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5bd4006490 0x7f5bd4031560 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc0097f0 tx=0x7f5bcc01c410 comp rx=0 tx=0).stop 2024-03-23T14:56:44.922 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:44.918+0000 7f5be2aa7640 1 -- 172.21.15.164:0/2998937635 shutdown_connections 2024-03-23T14:56:44.922 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:44.918+0000 7f5be2aa7640 1 --2- 172.21.15.164:0/2998937635 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5bd4006490 0x7f5bd4031560 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:44.923 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:44.918+0000 7f5be2aa7640 1 -- 172.21.15.164:0/2998937635 >> 172.21.15.164:0/2998937635 conn(0x7f5bd4009c10 msgr2=0x7f5bd400a020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:56:44.923 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:44.918+0000 7f5be2aa7640 1 -- 172.21.15.164:0/2998937635 shutdown_connections 2024-03-23T14:56:44.923 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:44.918+0000 7f5be2aa7640 1 -- 172.21.15.164:0/2998937635 wait complete. 2024-03-23T14:56:44.924 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:44.922+0000 7f5be2aa7640 1 Processor -- start 2024-03-23T14:56:44.924 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:44.922+0000 7f5be2aa7640 1 -- start start 2024-03-23T14:56:44.925 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:44.922+0000 7f5be2aa7640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5bd401b8c0 0x7f5bd4034440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:44.926 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:44.922+0000 7f5be2aa7640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5bd4039e30 0x7f5bd4022490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:44.930 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:44.922+0000 7f5be2aa7640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5bd416d230 0x7f5bd416d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:44.930 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:44.922+0000 7f5be2aa7640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5bd416dc20 con 0x7f5bd4039e30 2024-03-23T14:56:44.930 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:44.922+0000 7f5be2aa7640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f5bd4022b70 con 0x7f5bd401b8c0 2024-03-23T14:56:44.930 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:44.922+0000 7f5be2aa7640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f5bd4022e50 con 0x7f5bd416d230 2024-03-23T14:56:44.930 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:44.922+0000 7f5be22a6640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5bd416d230 0x7f5bd416d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:44.933 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:44.922+0000 7f5be22a6640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5bd416d230 0x7f5bd416d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.164:49310/0 (socket says 172.21.15.164:49310) 2024-03-23T14:56:44.933 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:44.922+0000 7f5be22a6640 1 -- 172.21.15.164:0/1098637523 learned_addr learned my addr 172.21.15.164:0/1098637523 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:56:44.933 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:44.922+0000 7f5be1aa5640 1 --2- 172.21.15.164:0/1098637523 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5bd401b8c0 0x7f5bd4034440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:44.933 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:44.922+0000 7f5be12a4640 1 --2- 172.21.15.164:0/1098637523 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5bd4039e30 0x7f5bd4022490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:44.933 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:44.922+0000 7f5be22a6640 1 -- 172.21.15.164:0/1098637523 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5bd401b8c0 msgr2=0x7f5bd4034440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:56:44.933 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:44.922+0000 7f5be22a6640 1 --2- 172.21.15.164:0/1098637523 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5bd401b8c0 0x7f5bd4034440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:44.933 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:44.922+0000 7f5be22a6640 1 -- 172.21.15.164:0/1098637523 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5bd4039e30 msgr2=0x7f5bd4022490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:56:44.933 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:44.922+0000 7f5be22a6640 1 --2- 172.21.15.164:0/1098637523 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5bd4039e30 0x7f5bd4022490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:44.933 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:44.922+0000 7f5be22a6640 1 -- 172.21.15.164:0/1098637523 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bcc009420 con 0x7f5bd416d230 2024-03-23T14:56:44.933 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:44.922+0000 7f5be12a4640 1 --2- 172.21.15.164:0/1098637523 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5bd4039e30 0x7f5bd4022490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T14:56:44.934 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:44.922+0000 7f5be22a6640 1 --2- 172.21.15.164:0/1098637523 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5bd416d230 0x7f5bd416d6e0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f5bd800c430 tx=0x7f5bd800c8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:56:44.934 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:44.922+0000 7f5bd27fc640 1 -- 172.21.15.164:0/1098637523 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bd80074e0 con 0x7f5bd416d230 2024-03-23T14:56:44.934 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:44.922+0000 7f5be2aa7640 1 -- 172.21.15.164:0/1098637523 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bd4023480 con 0x7f5bd416d230 2024-03-23T14:56:44.934 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:44.922+0000 7f5be2aa7640 1 -- 172.21.15.164:0/1098637523 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bd4023a40 con 0x7f5bd416d230 2024-03-23T14:56:44.934 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:44.922+0000 7f5bd27fc640 1 -- 172.21.15.164:0/1098637523 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bd8007640 con 0x7f5bd416d230 2024-03-23T14:56:44.934 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:44.922+0000 7f5bd27fc640 1 -- 172.21.15.164:0/1098637523 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bd8004a30 con 0x7f5bd416d230 2024-03-23T14:56:44.934 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:44.926+0000 7f5be2aa7640 1 -- 172.21.15.164:0/1098637523 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5bc40052a0 con 0x7f5bd416d230 2024-03-23T14:56:44.934 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:44.926+0000 7f5bd27fc640 1 -- 172.21.15.164:0/1098637523 <== mon.2 v2:172.21.15.173:3301/0 4 ==== mgrmap(e 4) ==== 86671+0+0 (secure 0 0 0) 0x7f5bd8004b90 con 0x7f5bd416d230 2024-03-23T14:56:44.934 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:44.926+0000 7f5bd27fc640 1 --2- 172.21.15.164:0/1098637523 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f5bc0067240 0x7f5bc0069700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:44.934 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:44.926+0000 7f5be1aa5640 1 --2- 172.21.15.164:0/1098637523 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f5bc0067240 0x7f5bc0069700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:44.934 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:44.926+0000 7f5bd27fc640 1 -- 172.21.15.164:0/1098637523 <== mon.2 v2:172.21.15.173:3301/0 5 ==== osd_map(12..12 src has 1..12) ==== 2231+0+0 (secure 0 0 0) 0x7f5bd8083da0 con 0x7f5bd416d230 2024-03-23T14:56:44.934 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:44.926+0000 7f5be1aa5640 1 --2- 172.21.15.164:0/1098637523 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f5bc0067240 0x7f5bc0069700 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc0af000 tx=0x7f5bcc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:56:44.939 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:44.934+0000 7f5bd27fc640 1 -- 172.21.15.164:0/1098637523 <== mon.2 v2:172.21.15.173:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7f5bd8051870 con 0x7f5bd416d230 2024-03-23T14:56:45.276 INFO:tasks.ceph.osd.5.smithi173.stderr:2024-03-23T14:56:45.269+0000 7f9f8fb9e640 -1 osd.5 0 waiting for initial osdmap 2024-03-23T14:56:45.276 INFO:tasks.ceph.osd.7.smithi173.stderr:2024-03-23T14:56:45.273+0000 7f7dad501640 -1 osd.7 0 waiting for initial osdmap 2024-03-23T14:56:45.293 INFO:tasks.ceph.osd.4.smithi173.stderr:2024-03-23T14:56:45.289+0000 7ff3cae72640 -1 osd.4 0 waiting for initial osdmap 2024-03-23T14:56:45.295 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:45.290+0000 7f5be2aa7640 1 -- 172.21.15.164:0/1098637523 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5bc4005550 con 0x7f5bd416d230 2024-03-23T14:56:45.304 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:45.302+0000 7f5bd27fc640 1 -- 172.21.15.164:0/1098637523 <== mon.2 v2:172.21.15.173:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) ==== 74+0+7058 (secure 0 0 0) 0x7f5bd8054f30 con 0x7f5bd416d230 2024-03-23T14:56:45.304 INFO:teuthology.misc.health.smithi164.stdout: 2024-03-23T14:56:45.304 INFO:teuthology.misc.health.smithi164.stdout:{"epoch":12,"fsid":"45c7911b-b69d-4e68-907b-fbfeb780aa3b","created":"2024-03-23T14:56:34.367633+0000","modified":"2024-03-23T14:56:43.121534+0000","last_up_change":"0.000000","last_in_change":"2024-03-23T14:56:36.629884+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"80abdc62-bd0d-4e60-a554-7ed6fd67cf54","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"f9001b06-be4d-4f42-a0e6-bcef2afa0d6d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"fa0aeb9f-8b54-4831-a188-3e361c7980d7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"7e18c957-3091-46d5-8e17-8f3354c1726c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"8e83f315-42bf-4471-9ce8-d81948da0f9f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"a230c872-71f9-40ab-8999-defd0162d4ed","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"6193bfd7-8432-4f7f-a117-a3fcee21af99","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"b0f1b80a-afc4-4538-83b9-f081d2dcb8cb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T14:56:45.305 INFO:tasks.ceph.osd.5.smithi173.stderr:2024-03-23T14:56:45.301+0000 7f9f881e5640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-23T14:56:45.308 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:45.306+0000 7f5be2aa7640 1 -- 172.21.15.164:0/1098637523 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f5bc0067240 msgr2=0x7f5bc0069700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:56:45.308 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:45.306+0000 7f5be2aa7640 1 --2- 172.21.15.164:0/1098637523 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f5bc0067240 0x7f5bc0069700 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc0af000 tx=0x7f5bcc008040 comp rx=0 tx=0).stop 2024-03-23T14:56:45.308 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:45.306+0000 7f5be2aa7640 1 -- 172.21.15.164:0/1098637523 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5bd416d230 msgr2=0x7f5bd416d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:56:45.309 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:45.306+0000 7f5be2aa7640 1 --2- 172.21.15.164:0/1098637523 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5bd416d230 0x7f5bd416d6e0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f5bd800c430 tx=0x7f5bd800c8f0 comp rx=0 tx=0).stop 2024-03-23T14:56:45.309 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:45.306+0000 7f5be2aa7640 1 -- 172.21.15.164:0/1098637523 shutdown_connections 2024-03-23T14:56:45.309 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:45.306+0000 7f5be2aa7640 1 --2- 172.21.15.164:0/1098637523 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5bd416d230 0x7f5bd416d6e0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:45.309 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:45.306+0000 7f5be2aa7640 1 --2- 172.21.15.164:0/1098637523 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5bd4039e30 0x7f5bd4022490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:45.309 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:45.306+0000 7f5be2aa7640 1 --2- 172.21.15.164:0/1098637523 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f5bc0067240 0x7f5bc0069700 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:45.309 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:45.306+0000 7f5be2aa7640 1 --2- 172.21.15.164:0/1098637523 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5bd401b8c0 0x7f5bd4034440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:45.309 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:45.306+0000 7f5be2aa7640 1 -- 172.21.15.164:0/1098637523 >> 172.21.15.164:0/1098637523 conn(0x7f5bd4009c10 msgr2=0x7f5bd403c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:56:45.309 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:45.306+0000 7f5be2aa7640 1 -- 172.21.15.164:0/1098637523 shutdown_connections 2024-03-23T14:56:45.309 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:45.306+0000 7f5be2aa7640 1 -- 172.21.15.164:0/1098637523 wait complete. 2024-03-23T14:56:45.311 INFO:tasks.ceph.osd.6.smithi173.stderr:2024-03-23T14:56:45.305+0000 7f9ea89f1640 -1 osd.6 0 waiting for initial osdmap 2024-03-23T14:56:45.313 INFO:tasks.ceph.osd.4.smithi173.stderr:2024-03-23T14:56:45.309+0000 7ff3c1c83640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-23T14:56:45.313 INFO:tasks.ceph.osd.7.smithi173.stderr:2024-03-23T14:56:45.309+0000 7f7da5b48640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-23T14:56:45.336 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-03-23T14:56:45.340 INFO:tasks.ceph.osd.1.smithi164.stderr:2024-03-23T14:56:45.338+0000 7f11383ed640 -1 osd.1 0 waiting for initial osdmap 2024-03-23T14:56:45.341 INFO:tasks.ceph.osd.6.smithi173.stderr:2024-03-23T14:56:45.337+0000 7f9ea1038640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-23T14:56:45.373 INFO:tasks.ceph.osd.1.smithi164.stderr:2024-03-23T14:56:45.370+0000 7f112f1fe640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-23T14:56:45.373 INFO:tasks.ceph.osd.0.smithi164.stderr:2024-03-23T14:56:45.370+0000 7f9cdf718640 -1 osd.0 0 waiting for initial osdmap 2024-03-23T14:56:45.404 INFO:tasks.ceph.osd.0.smithi164.stderr:2024-03-23T14:56:45.402+0000 7f9cd7d5f640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-23T14:56:45.472 INFO:tasks.ceph.osd.3.smithi164.stderr:2024-03-23T14:56:45.470+0000 7ffa6ffa8640 -1 osd.3 0 waiting for initial osdmap 2024-03-23T14:56:45.474 INFO:tasks.ceph.osd.2.smithi164.stderr:2024-03-23T14:56:45.470+0000 7f1f71ec0640 -1 osd.2 0 waiting for initial osdmap 2024-03-23T14:56:45.501 INFO:tasks.ceph.osd.3.smithi164.stderr:2024-03-23T14:56:45.498+0000 7ffa685ef640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-23T14:56:45.513 INFO:tasks.ceph.osd.2.smithi164.stderr:2024-03-23T14:56:45.510+0000 7f1f6a507640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-23T14:56:52.143 DEBUG:teuthology.orchestra.run.smithi164:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-03-23T14:56:52.265 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:52.262+0000 7fe006629640 1 Processor -- start 2024-03-23T14:56:52.265 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:52.262+0000 7fe006629640 1 -- start start 2024-03-23T14:56:52.267 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:52.262+0000 7fe006629640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe000148d70 0x7fe0000af5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:52.267 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:52.262+0000 7fe006629640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fe000149150 0x7fe00014b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:52.267 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:52.262+0000 7fe006629640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0000b00a0 con 0x7fe00014b880 2024-03-23T14:56:52.268 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:52.262+0000 7fe006629640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 -- 0x7fe0000afb20 con 0x7fe000149150 2024-03-23T14:56:52.268 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:52.262+0000 7fe006629640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fe0000adf90 con 0x7fe000148d70 2024-03-23T14:56:52.268 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:52.262+0000 7fe004bc6640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fe00014b880 0x7fe0000ae310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:52618/0 (socket says 172.21.15.164:52618) 2024-03-23T14:56:52.268 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:52.262+0000 7fe004bc6640 1 -- 172.21.15.164:0/2811437431 learned_addr learned my addr 172.21.15.164:0/2811437431 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-03-23T14:56:52.268 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:52.262+0000 7fdffffff640 1 --2- 172.21.15.164:0/2811437431 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe000148d70 0x7fe0000af5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:52.268 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:52.262+0000 7fdfff7fe640 1 --2- 172.21.15.164:0/2811437431 >> v2:172.21.15.173:3300/0 conn(0x7fe000149150 0x7fe00014b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:52.268 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:52.262+0000 7fdffeffd640 1 -- 172.21.15.164:0/2811437431 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2242354899 0 0) 0x7fe0000b00a0 con 0x7fe00014b880 2024-03-23T14:56:52.268 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:52.262+0000 7fdffeffd640 1 -- 172.21.15.164:0/2811437431 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdfe8003530 con 0x7fe00014b880 2024-03-23T14:56:52.268 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:52.262+0000 7fdffeffd640 1 -- 172.21.15.164:0/2811437431 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4265214088 0 0) 0x7fdfe8003530 con 0x7fe00014b880 2024-03-23T14:56:52.268 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:52.262+0000 7fdffeffd640 1 -- 172.21.15.164:0/2811437431 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe000148d70 msgr2=0x7fe0000af5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:56:52.268 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:52.262+0000 7fdffeffd640 1 --2- 172.21.15.164:0/2811437431 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe000148d70 0x7fe0000af5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:52.268 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:52.262+0000 7fdffeffd640 1 -- 172.21.15.164:0/2811437431 >> v2:172.21.15.173:3300/0 conn(0x7fe000149150 msgr2=0x7fe00014b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:56:52.268 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:52.262+0000 7fdffeffd640 1 --2- 172.21.15.164:0/2811437431 >> v2:172.21.15.173:3300/0 conn(0x7fe000149150 0x7fe00014b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:52.268 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:52.262+0000 7fdffeffd640 1 -- 172.21.15.164:0/2811437431 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0000b4ce0 con 0x7fe00014b880 2024-03-23T14:56:52.269 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:52.262+0000 7fdffeffd640 1 -- 172.21.15.164:0/2811437431 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2526390548 0 0) 0x7fdff8001a00 con 0x7fe00014b880 2024-03-23T14:56:52.269 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:52.262+0000 7fdffeffd640 1 -- 172.21.15.164:0/2811437431 >> v1:172.21.15.164:6789/0 conn(0x7fe00014b880 legacy=0x7fe0000ae310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:56:52.269 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:52.262+0000 7fdffeffd640 1 --2- 172.21.15.164:0/2811437431 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdfe8003980 0x7fdfe8003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:52.269 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:52.262+0000 7fdffeffd640 1 --2- 172.21.15.164:0/2811437431 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe000148d70 0x7fdfe80047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:52.269 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:52.262+0000 7fdffeffd640 1 --2- 172.21.15.164:0/2811437431 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdfe8004d10 0x7fdfe8005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:52.269 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:52.262+0000 7fdffeffd640 1 -- 172.21.15.164:0/2811437431 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe0000afb20 con 0x7fdfe8004d10 2024-03-23T14:56:52.269 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:52.262+0000 7fdffeffd640 1 -- 172.21.15.164:0/2811437431 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fdfe800b990 con 0x7fdfe8003980 2024-03-23T14:56:52.269 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:52.262+0000 7fdffeffd640 1 -- 172.21.15.164:0/2811437431 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fdfe800bad0 con 0x7fe000148d70 2024-03-23T14:56:52.269 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:52.262+0000 7fe004bc6640 1 --2- 172.21.15.164:0/2811437431 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdfe8004d10 0x7fdfe8005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:52.269 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:52.262+0000 7fdffffff640 1 --2- 172.21.15.164:0/2811437431 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdfe8003980 0x7fdfe8003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:52.269 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:52.262+0000 7fe004bc6640 1 -- 172.21.15.164:0/2811437431 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe000148d70 msgr2=0x7fdfe80047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:56:52.269 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:52.262+0000 7fdfff7fe640 1 --2- 172.21.15.164:0/2811437431 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe000148d70 0x7fdfe80047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:52.269 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:52.262+0000 7fe004bc6640 1 --2- 172.21.15.164:0/2811437431 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe000148d70 0x7fdfe80047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:52.269 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:52.262+0000 7fe004bc6640 1 -- 172.21.15.164:0/2811437431 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdfe8003980 msgr2=0x7fdfe8003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:56:52.269 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:52.262+0000 7fe004bc6640 1 --2- 172.21.15.164:0/2811437431 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdfe8003980 0x7fdfe8003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:52.269 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:52.262+0000 7fe004bc6640 1 -- 172.21.15.164:0/2811437431 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfe8003530 con 0x7fdfe8004d10 2024-03-23T14:56:52.270 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:52.262+0000 7fdfff7fe640 1 --2- 172.21.15.164:0/2811437431 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe000148d70 0x7fdfe80047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T14:56:52.271 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:52.266+0000 7fe004bc6640 1 --2- 172.21.15.164:0/2811437431 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdfe8004d10 0x7fdfe8005190 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fdff800a830 tx=0x7fdff801fae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb578565f0928ed7 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:56:52.271 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:52.266+0000 7fdffeffd640 1 -- 172.21.15.164:0/2811437431 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdff80aa710 con 0x7fdfe8004d10 2024-03-23T14:56:52.271 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:52.266+0000 7fdffeffd640 1 -- 172.21.15.164:0/2811437431 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdff80aa870 con 0x7fdfe8004d10 2024-03-23T14:56:52.271 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:52.266+0000 7fdffeffd640 1 -- 172.21.15.164:0/2811437431 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdff80aac40 con 0x7fdfe8004d10 2024-03-23T14:56:52.271 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:52.266+0000 7fe006629640 1 -- 172.21.15.164:0/2811437431 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdfe8004d10 msgr2=0x7fdfe8005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:56:52.271 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:52.266+0000 7fe006629640 1 --2- 172.21.15.164:0/2811437431 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdfe8004d10 0x7fdfe8005190 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fdff800a830 tx=0x7fdff801fae0 comp rx=0 tx=0).stop 2024-03-23T14:56:52.271 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:52.266+0000 7fe004bc6640 1 -- 172.21.15.164:0/2811437431 reap_dead start 2024-03-23T14:56:52.271 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:52.266+0000 7fe006629640 1 -- 172.21.15.164:0/2811437431 shutdown_connections 2024-03-23T14:56:52.271 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:52.266+0000 7fe006629640 1 -- 172.21.15.164:0/2811437431 >> 172.21.15.164:0/2811437431 conn(0x7fe0000b8860 msgr2=0x7fe0000b8c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:56:52.271 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:52.266+0000 7fe006629640 1 -- 172.21.15.164:0/2811437431 shutdown_connections 2024-03-23T14:56:52.272 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:52.266+0000 7fe006629640 1 -- 172.21.15.164:0/2811437431 wait complete. 2024-03-23T14:56:52.272 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:52.266+0000 7fe006629640 1 Processor -- start 2024-03-23T14:56:52.272 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:52.266+0000 7fe006629640 1 -- start start 2024-03-23T14:56:52.274 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:52.266+0000 7fe006629640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe000120700 0x7fe0001e0150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:52.274 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:52.266+0000 7fe006629640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe0001e0690 0x7fe000143520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:52.274 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:52.266+0000 7fe006629640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe000143a60 0x7fe000143ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:52.274 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:52.266+0000 7fe006629640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe0001e0b10 con 0x7fe0001e0690 2024-03-23T14:56:52.274 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:52.266+0000 7fe006629640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fe000120ae0 con 0x7fe000143a60 2024-03-23T14:56:52.274 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:52.266+0000 7fe006629640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fe000122d70 con 0x7fe000120700 2024-03-23T14:56:52.274 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:52.270+0000 7fdfff7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe0001e0690 0x7fe000143520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:52.274 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:52.270+0000 7fdfff7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe0001e0690 0x7fe000143520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:54206/0 (socket says 172.21.15.164:54206) 2024-03-23T14:56:52.275 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:52.270+0000 7fdfff7fe640 1 -- 172.21.15.164:0/1272801194 learned_addr learned my addr 172.21.15.164:0/1272801194 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:56:52.275 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:52.270+0000 7fdffffff640 1 --2- 172.21.15.164:0/1272801194 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe000120700 0x7fe0001e0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:52.275 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:52.270+0000 7fe004bc6640 1 --2- 172.21.15.164:0/1272801194 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe000143a60 0x7fe000143ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:52.275 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:52.270+0000 7fdfff7fe640 1 -- 172.21.15.164:0/1272801194 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe000120700 msgr2=0x7fe0001e0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:56:52.275 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:52.270+0000 7fdfff7fe640 1 --2- 172.21.15.164:0/1272801194 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe000120700 0x7fe0001e0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:52.275 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:52.270+0000 7fdfff7fe640 1 -- 172.21.15.164:0/1272801194 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe000143a60 msgr2=0x7fe000143ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:56:52.275 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:52.270+0000 7fdfff7fe640 1 --2- 172.21.15.164:0/1272801194 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe000143a60 0x7fe000143ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:52.275 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:52.270+0000 7fdfff7fe640 1 -- 172.21.15.164:0/1272801194 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe000121820 con 0x7fe0001e0690 2024-03-23T14:56:52.275 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:52.270+0000 7fdffffff640 1 --2- 172.21.15.164:0/1272801194 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe000120700 0x7fe0001e0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T14:56:52.275 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:52.270+0000 7fdfff7fe640 1 --2- 172.21.15.164:0/1272801194 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe0001e0690 0x7fe000143520 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fdff4010950 tx=0x7fdff4010e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:56:52.275 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:52.270+0000 7fdffcff9640 1 -- 172.21.15.164:0/1272801194 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdff400ec20 con 0x7fe0001e0690 2024-03-23T14:56:52.275 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:52.270+0000 7fdffcff9640 1 -- 172.21.15.164:0/1272801194 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdff400ed80 con 0x7fe0001e0690 2024-03-23T14:56:52.275 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:52.270+0000 7fe006629640 1 -- 172.21.15.164:0/1272801194 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe000121b00 con 0x7fe0001e0690 2024-03-23T14:56:52.275 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:52.270+0000 7fdffcff9640 1 -- 172.21.15.164:0/1272801194 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdff4004220 con 0x7fe0001e0690 2024-03-23T14:56:52.275 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:52.270+0000 7fe006629640 1 -- 172.21.15.164:0/1272801194 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0000710a0 con 0x7fe0001e0690 2024-03-23T14:56:52.276 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:52.270+0000 7fe006629640 1 -- 172.21.15.164:0/1272801194 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe00004f4f0 con 0x7fe0001e0690 2024-03-23T14:56:52.276 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:52.270+0000 7fdffcff9640 1 -- 172.21.15.164:0/1272801194 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7fdff4004380 con 0x7fe0001e0690 2024-03-23T14:56:52.276 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:52.270+0000 7fdffcff9640 1 --2- 172.21.15.164:0/1272801194 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fdfec0672a0 0x7fdfec069760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:52.276 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:52.270+0000 7fdffcff9640 1 -- 172.21.15.164:0/1272801194 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fdff4084c30 con 0x7fe0001e0690 2024-03-23T14:56:52.276 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:52.270+0000 7fdffffff640 1 --2- 172.21.15.164:0/1272801194 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fdfec0672a0 0x7fdfec069760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:52.277 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:52.270+0000 7fdffffff640 1 --2- 172.21.15.164:0/1272801194 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fdfec0672a0 0x7fdfec069760 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fdff0009af0 tx=0x7fdff0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:56:52.279 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:52.274+0000 7fdffcff9640 1 -- 172.21.15.164:0/1272801194 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7fdff4051bc0 con 0x7fe0001e0690 2024-03-23T14:56:52.426 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:52.422+0000 7fe006629640 1 -- 172.21.15.164:0/1272801194 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe0000b2b20 con 0x7fe0001e0690 2024-03-23T14:56:52.428 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:52.426+0000 7fdffcff9640 1 -- 172.21.15.164:0/1272801194 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12908 (secure 0 0 0) 0x7fdff4055280 con 0x7fe0001e0690 2024-03-23T14:56:52.428 INFO:teuthology.misc.health.smithi164.stdout: 2024-03-23T14:56:52.429 INFO:teuthology.misc.health.smithi164.stdout:{"epoch":16,"fsid":"45c7911b-b69d-4e68-907b-fbfeb780aa3b","created":"2024-03-23T14:56:34.367633+0000","modified":"2024-03-23T14:56:49.497757+0000","last_up_change":"2024-03-23T14:56:46.297403+0000","last_in_change":"2024-03-23T14:56:36.629884+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T14:56:47.186157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"80abdc62-bd0d-4e60-a554-7ed6fd67cf54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":984716690},{"type":"v1","addr":"172.21.15.164:6801","nonce":984716690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":984716690},{"type":"v1","addr":"172.21.15.164:6803","nonce":984716690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":984716690},{"type":"v1","addr":"172.21.15.164:6807","nonce":984716690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":984716690},{"type":"v1","addr":"172.21.15.164:6805","nonce":984716690}]},"public_addr":"172.21.15.164:6801/984716690","cluster_addr":"172.21.15.164:6803/984716690","heartbeat_back_addr":"172.21.15.164:6807/984716690","heartbeat_front_addr":"172.21.15.164:6805/984716690","state":["exists","up"]},{"osd":1,"uuid":"f9001b06-be4d-4f42-a0e6-bcef2afa0d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":1349657239},{"type":"v1","addr":"172.21.15.164:6817","nonce":1349657239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":1349657239},{"type":"v1","addr":"172.21.15.164:6819","nonce":1349657239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":1349657239},{"type":"v1","addr":"172.21.15.164:6823","nonce":1349657239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":1349657239},{"type":"v1","addr":"172.21.15.164:6821","nonce":1349657239}]},"public_addr":"172.21.15.164:6817/1349657239","cluster_addr":"172.21.15.164:6819/1349657239","heartbeat_back_addr":"172.21.15.164:6823/1349657239","heartbeat_front_addr":"172.21.15.164:6821/1349657239","state":["exists","up"]},{"osd":2,"uuid":"fa0aeb9f-8b54-4831-a188-3e361c7980d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":887861252},{"type":"v1","addr":"172.21.15.164:6809","nonce":887861252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":887861252},{"type":"v1","addr":"172.21.15.164:6811","nonce":887861252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":887861252},{"type":"v1","addr":"172.21.15.164:6815","nonce":887861252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":887861252},{"type":"v1","addr":"172.21.15.164:6813","nonce":887861252}]},"public_addr":"172.21.15.164:6809/887861252","cluster_addr":"172.21.15.164:6811/887861252","heartbeat_back_addr":"172.21.15.164:6815/887861252","heartbeat_front_addr":"172.21.15.164:6813/887861252","state":["exists","up"]},{"osd":3,"uuid":"7e18c957-3091-46d5-8e17-8f3354c1726c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":812540271},{"type":"v1","addr":"172.21.15.164:6825","nonce":812540271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":812540271},{"type":"v1","addr":"172.21.15.164:6827","nonce":812540271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":812540271},{"type":"v1","addr":"172.21.15.164:6831","nonce":812540271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":812540271},{"type":"v1","addr":"172.21.15.164:6829","nonce":812540271}]},"public_addr":"172.21.15.164:6825/812540271","cluster_addr":"172.21.15.164:6827/812540271","heartbeat_back_addr":"172.21.15.164:6831/812540271","heartbeat_front_addr":"172.21.15.164:6829/812540271","state":["exists","up"]},{"osd":4,"uuid":"8e83f315-42bf-4471-9ce8-d81948da0f9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":2656857421},{"type":"v1","addr":"172.21.15.173:6825","nonce":2656857421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2656857421},{"type":"v1","addr":"172.21.15.173:6827","nonce":2656857421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2656857421},{"type":"v1","addr":"172.21.15.173:6831","nonce":2656857421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":2656857421},{"type":"v1","addr":"172.21.15.173:6829","nonce":2656857421}]},"public_addr":"172.21.15.173:6825/2656857421","cluster_addr":"172.21.15.173:6827/2656857421","heartbeat_back_addr":"172.21.15.173:6831/2656857421","heartbeat_front_addr":"172.21.15.173:6829/2656857421","state":["exists","up"]},{"osd":5,"uuid":"a230c872-71f9-40ab-8999-defd0162d4ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":3676483185},{"type":"v1","addr":"172.21.15.173:6801","nonce":3676483185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":3676483185},{"type":"v1","addr":"172.21.15.173:6803","nonce":3676483185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":3676483185},{"type":"v1","addr":"172.21.15.173:6807","nonce":3676483185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":3676483185},{"type":"v1","addr":"172.21.15.173:6805","nonce":3676483185}]},"public_addr":"172.21.15.173:6801/3676483185","cluster_addr":"172.21.15.173:6803/3676483185","heartbeat_back_addr":"172.21.15.173:6807/3676483185","heartbeat_front_addr":"172.21.15.173:6805/3676483185","state":["exists","up"]},{"osd":6,"uuid":"6193bfd7-8432-4f7f-a117-a3fcee21af99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":3631684087},{"type":"v1","addr":"172.21.15.173:6809","nonce":3631684087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":3631684087},{"type":"v1","addr":"172.21.15.173:6811","nonce":3631684087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":3631684087},{"type":"v1","addr":"172.21.15.173:6815","nonce":3631684087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":3631684087},{"type":"v1","addr":"172.21.15.173:6813","nonce":3631684087}]},"public_addr":"172.21.15.173:6809/3631684087","cluster_addr":"172.21.15.173:6811/3631684087","heartbeat_back_addr":"172.21.15.173:6815/3631684087","heartbeat_front_addr":"172.21.15.173:6813/3631684087","state":["exists","up"]},{"osd":7,"uuid":"b0f1b80a-afc4-4538-83b9-f081d2dcb8cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":3261114948},{"type":"v1","addr":"172.21.15.173:6817","nonce":3261114948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":3261114948},{"type":"v1","addr":"172.21.15.173:6819","nonce":3261114948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":3261114948},{"type":"v1","addr":"172.21.15.173:6823","nonce":3261114948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":3261114948},{"type":"v1","addr":"172.21.15.173:6821","nonce":3261114948}]},"public_addr":"172.21.15.173:6817/3261114948","cluster_addr":"172.21.15.173:6819/3261114948","heartbeat_back_addr":"172.21.15.173:6823/3261114948","heartbeat_front_addr":"172.21.15.173:6821/3261114948","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.188390+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.159932+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.191101+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.181776+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.224365+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.177515+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.153631+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.139972+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T14:56:52.430 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:52.426+0000 7fe006629640 1 -- 172.21.15.164:0/1272801194 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fdfec0672a0 msgr2=0x7fdfec069760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:56:52.430 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:52.426+0000 7fe006629640 1 --2- 172.21.15.164:0/1272801194 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fdfec0672a0 0x7fdfec069760 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fdff0009af0 tx=0x7fdff0008040 comp rx=0 tx=0).stop 2024-03-23T14:56:52.431 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:52.426+0000 7fe006629640 1 -- 172.21.15.164:0/1272801194 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe0001e0690 msgr2=0x7fe000143520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:56:52.431 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:52.426+0000 7fe006629640 1 --2- 172.21.15.164:0/1272801194 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe0001e0690 0x7fe000143520 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fdff4010950 tx=0x7fdff4010e10 comp rx=0 tx=0).stop 2024-03-23T14:56:52.431 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:52.426+0000 7fe006629640 1 -- 172.21.15.164:0/1272801194 shutdown_connections 2024-03-23T14:56:52.431 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:52.426+0000 7fe006629640 1 --2- 172.21.15.164:0/1272801194 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fdfec0672a0 0x7fdfec069760 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:52.431 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:52.426+0000 7fe006629640 1 --2- 172.21.15.164:0/1272801194 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe000143a60 0x7fe000143ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:52.431 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:52.426+0000 7fe006629640 1 --2- 172.21.15.164:0/1272801194 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe0001e0690 0x7fe000143520 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:52.431 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:52.426+0000 7fe006629640 1 --2- 172.21.15.164:0/1272801194 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe000120700 0x7fe0001e0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:52.431 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:52.426+0000 7fe006629640 1 -- 172.21.15.164:0/1272801194 >> 172.21.15.164:0/1272801194 conn(0x7fe0000b8860 msgr2=0x7fe000149920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:56:52.431 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:52.426+0000 7fe006629640 1 -- 172.21.15.164:0/1272801194 shutdown_connections 2024-03-23T14:56:52.431 INFO:teuthology.misc.health.smithi164.stderr:2024-03-23T14:56:52.426+0000 7fe006629640 1 -- 172.21.15.164:0/1272801194 wait complete. 2024-03-23T14:56:52.445 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-03-23T14:56:52.445 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-03-23T14:56:52.446 INFO:tasks.ceph.mds.a:Restarting daemon 2024-03-23T14:56:52.446 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-03-23T14:56:52.448 INFO:tasks.ceph.mds.a:Started 2024-03-23T14:56:52.448 INFO:tasks.ceph.mds.b:Restarting daemon 2024-03-23T14:56:52.448 DEBUG:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-03-23T14:56:52.451 INFO:tasks.ceph.mds.b:Started 2024-03-23T14:56:52.451 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-03-23T14:56:52.451 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T14:56:52.528 INFO:tasks.ceph.mds.a.smithi164.stdout:starting mds.a at 2024-03-23T14:56:52.550 INFO:tasks.ceph.mds.b.smithi173.stdout:starting mds.b at 2024-03-23T14:56:52.565 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.562+0000 7f203c134640 1 Processor -- start 2024-03-23T14:56:52.565 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.562+0000 7f203c134640 1 -- start start 2024-03-23T14:56:52.570 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.562+0000 7f203c134640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f20340b9e80 0x7f20340b5590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:52.570 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.562+0000 7f203c134640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f20340b9190 0x7f203414a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:52.570 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.562+0000 7f203c134640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20340b4280 con 0x7f203414aed0 2024-03-23T14:56:52.570 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.562+0000 7f203c134640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 -- 0x7f20340b48b0 con 0x7f20340b9e80 2024-03-23T14:56:52.570 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.562+0000 7f203c134640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f20340b5f20 con 0x7f20340b9190 2024-03-23T14:56:52.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.562+0000 7f203a6d1640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f203414aed0 0x7f2034140f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:45390/0 (socket says 172.21.15.164:45390) 2024-03-23T14:56:52.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.562+0000 7f203a6d1640 1 -- 172.21.15.164:0/4141420046 learned_addr learned my addr 172.21.15.164:0/4141420046 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-03-23T14:56:52.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.562+0000 7f20396cf640 1 --2- 172.21.15.164:0/4141420046 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f20340b9190 0x7f203414a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:52.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.562+0000 7f2039ed0640 1 --2- 172.21.15.164:0/4141420046 >> v2:172.21.15.173:3300/0 conn(0x7f20340b9e80 0x7f20340b5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:52.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.562+0000 7f2038ece640 1 -- 172.21.15.164:0/4141420046 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 763372883 0 0) 0x7f20340b4280 con 0x7f203414aed0 2024-03-23T14:56:52.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.562+0000 7f2038ece640 1 -- 172.21.15.164:0/4141420046 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2020003530 con 0x7f203414aed0 2024-03-23T14:56:52.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.562+0000 7f2038ece640 1 -- 172.21.15.164:0/4141420046 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2451062663 0 0) 0x7f2020003530 con 0x7f203414aed0 2024-03-23T14:56:52.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.562+0000 7f2038ece640 1 -- 172.21.15.164:0/4141420046 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f20340b9190 msgr2=0x7f203414a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:56:52.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.562+0000 7f2038ece640 1 --2- 172.21.15.164:0/4141420046 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f20340b9190 0x7f203414a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:52.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.562+0000 7f2038ece640 1 -- 172.21.15.164:0/4141420046 >> v2:172.21.15.173:3300/0 conn(0x7f20340b9e80 msgr2=0x7f20340b5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:56:52.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.562+0000 7f2038ece640 1 --2- 172.21.15.164:0/4141420046 >> v2:172.21.15.173:3300/0 conn(0x7f20340b9e80 0x7f20340b5590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:52.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.562+0000 7f2038ece640 1 -- 172.21.15.164:0/4141420046 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20340b4cd0 con 0x7f203414aed0 2024-03-23T14:56:52.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.562+0000 7f2038ece640 1 -- 172.21.15.164:0/4141420046 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2526390548 0 0) 0x7f202c001a00 con 0x7f203414aed0 2024-03-23T14:56:52.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.562+0000 7f2039ed0640 1 --2- 172.21.15.164:0/4141420046 >> v2:172.21.15.173:3300/0 conn(0x7f20340b9e80 0x7f20340b5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T14:56:52.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.562+0000 7f2038ece640 1 -- 172.21.15.164:0/4141420046 >> v1:172.21.15.164:6789/0 conn(0x7f203414aed0 legacy=0x7f2034140f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:56:52.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.562+0000 7f2038ece640 1 --2- 172.21.15.164:0/4141420046 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2020003980 0x7f2020003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:52.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.562+0000 7f2038ece640 1 --2- 172.21.15.164:0/4141420046 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f20200047d0 0x7f2020004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:52.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.562+0000 7f2038ece640 1 --2- 172.21.15.164:0/4141420046 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f20340b9190 0x7f2020005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:52.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.562+0000 7f2038ece640 1 -- 172.21.15.164:0/4141420046 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f20340b48b0 con 0x7f2020003980 2024-03-23T14:56:52.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.562+0000 7f2038ece640 1 -- 172.21.15.164:0/4141420046 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f202000b990 con 0x7f20200047d0 2024-03-23T14:56:52.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.562+0000 7f2038ece640 1 -- 172.21.15.164:0/4141420046 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f202000bb50 con 0x7f20340b9190 2024-03-23T14:56:52.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.562+0000 7f2039ed0640 1 --2- 172.21.15.164:0/4141420046 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2020003980 0x7f2020003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:52.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.562+0000 7f203a6d1640 1 --2- 172.21.15.164:0/4141420046 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f20340b9190 0x7f2020005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:52.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.562+0000 7f20396cf640 1 --2- 172.21.15.164:0/4141420046 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f20200047d0 0x7f2020004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:52.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.562+0000 7f2039ed0640 1 -- 172.21.15.164:0/4141420046 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f20340b9190 msgr2=0x7f2020005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:56:52.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.562+0000 7f2039ed0640 1 --2- 172.21.15.164:0/4141420046 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f20340b9190 0x7f2020005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:52.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.562+0000 7f2039ed0640 1 -- 172.21.15.164:0/4141420046 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f20200047d0 msgr2=0x7f2020004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:56:52.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.562+0000 7f2039ed0640 1 --2- 172.21.15.164:0/4141420046 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f20200047d0 0x7f2020004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:52.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.562+0000 7f2039ed0640 1 -- 172.21.15.164:0/4141420046 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2020003530 con 0x7f2020003980 2024-03-23T14:56:52.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.562+0000 7f203a6d1640 1 --2- 172.21.15.164:0/4141420046 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f20340b9190 0x7f2020005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T14:56:52.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.566+0000 7f2039ed0640 1 --2- 172.21.15.164:0/4141420046 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2020003980 0x7f2020003da0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2030009cb0 tx=0x7f203001cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=440755226ab04e34 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:56:52.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.566+0000 7f2038ece640 1 -- 172.21.15.164:0/4141420046 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20300a5930 con 0x7f2020003980 2024-03-23T14:56:52.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.566+0000 7f2038ece640 1 -- 172.21.15.164:0/4141420046 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20300a5a90 con 0x7f2020003980 2024-03-23T14:56:52.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.566+0000 7f2038ece640 1 -- 172.21.15.164:0/4141420046 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20300a5e40 con 0x7f2020003980 2024-03-23T14:56:52.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.566+0000 7f203c134640 1 -- 172.21.15.164:0/4141420046 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2020003980 msgr2=0x7f2020003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:56:52.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.566+0000 7f203c134640 1 --2- 172.21.15.164:0/4141420046 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2020003980 0x7f2020003da0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2030009cb0 tx=0x7f203001cc90 comp rx=0 tx=0).stop 2024-03-23T14:56:52.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.566+0000 7f203a6d1640 1 -- 172.21.15.164:0/4141420046 reap_dead start 2024-03-23T14:56:52.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.566+0000 7f203c134640 1 -- 172.21.15.164:0/4141420046 shutdown_connections 2024-03-23T14:56:52.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.566+0000 7f203c134640 1 -- 172.21.15.164:0/4141420046 >> 172.21.15.164:0/4141420046 conn(0x7f203402c3c0 msgr2=0x7f203402c7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:56:52.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.566+0000 7f203c134640 1 -- 172.21.15.164:0/4141420046 shutdown_connections 2024-03-23T14:56:52.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.566+0000 7f203c134640 1 -- 172.21.15.164:0/4141420046 wait complete. 2024-03-23T14:56:52.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.566+0000 7f203c134640 1 Processor -- start 2024-03-23T14:56:52.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.566+0000 7f203c134640 1 -- start start 2024-03-23T14:56:52.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.566+0000 7f203c134640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f20341465c0 0x7f2034137920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:52.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.566+0000 7f203c134640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f20340bd270 0x7f2034137e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:52.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.566+0000 7f203c134640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2034138940 0x7f2034135f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:52.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.566+0000 7f203c134640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2034137790 con 0x7f2034138940 2024-03-23T14:56:52.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.566+0000 7f203c134640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f20341469a0 con 0x7f20340bd270 2024-03-23T14:56:52.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.566+0000 7f203c134640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f20340badc0 con 0x7f20341465c0 2024-03-23T14:56:52.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.566+0000 7f203a6d1640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2034138940 0x7f2034135f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:52.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.566+0000 7f203a6d1640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2034138940 0x7f2034135f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:57356/0 (socket says 172.21.15.164:57356) 2024-03-23T14:56:52.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.566+0000 7f203a6d1640 1 -- 172.21.15.164:0/4013044570 learned_addr learned my addr 172.21.15.164:0/4013044570 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:56:52.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.566+0000 7f2039ed0640 1 --2- 172.21.15.164:0/4013044570 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f20341465c0 0x7f2034137920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:52.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.566+0000 7f20396cf640 1 --2- 172.21.15.164:0/4013044570 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f20340bd270 0x7f2034137e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:52.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.570+0000 7f203a6d1640 1 -- 172.21.15.164:0/4013044570 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f20341465c0 msgr2=0x7f2034137920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:56:52.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.570+0000 7f203a6d1640 1 --2- 172.21.15.164:0/4013044570 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f20341465c0 0x7f2034137920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:52.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.570+0000 7f203a6d1640 1 -- 172.21.15.164:0/4013044570 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f20340bd270 msgr2=0x7f2034137e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:56:52.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.570+0000 7f203a6d1640 1 --2- 172.21.15.164:0/4013044570 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f20340bd270 0x7f2034137e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:52.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.570+0000 7f203a6d1640 1 -- 172.21.15.164:0/4013044570 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20341374b0 con 0x7f2034138940 2024-03-23T14:56:52.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.570+0000 7f203a6d1640 1 --2- 172.21.15.164:0/4013044570 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2034138940 0x7f2034135f40 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f202c004bd0 tx=0x7f202c00e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:56:52.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.570+0000 7f202a7fc640 1 -- 172.21.15.164:0/4013044570 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f202c00eec0 con 0x7f2034138940 2024-03-23T14:56:52.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.570+0000 7f202a7fc640 1 -- 172.21.15.164:0/4013044570 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f202c013420 con 0x7f2034138940 2024-03-23T14:56:52.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.570+0000 7f203c134640 1 -- 172.21.15.164:0/4013044570 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f203414a260 con 0x7f2034138940 2024-03-23T14:56:52.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.570+0000 7f202a7fc640 1 -- 172.21.15.164:0/4013044570 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f202c0137b0 con 0x7f2034138940 2024-03-23T14:56:52.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.570+0000 7f203c134640 1 -- 172.21.15.164:0/4013044570 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2034136480 con 0x7f2034138940 2024-03-23T14:56:52.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.570+0000 7f203c134640 1 -- 172.21.15.164:0/4013044570 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f203414a0c0 con 0x7f2034138940 2024-03-23T14:56:52.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.570+0000 7f202a7fc640 1 -- 172.21.15.164:0/4013044570 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7f202c013910 con 0x7f2034138940 2024-03-23T14:56:52.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.570+0000 7f202a7fc640 1 --2- 172.21.15.164:0/4013044570 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f20180672a0 0x7f2018069760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:52.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.570+0000 7f202a7fc640 1 -- 172.21.15.164:0/4013044570 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f202c0853e0 con 0x7f2034138940 2024-03-23T14:56:52.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.570+0000 7f2039ed0640 1 --2- 172.21.15.164:0/4013044570 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f20180672a0 0x7f2018069760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:52.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.570+0000 7f2039ed0640 1 --2- 172.21.15.164:0/4013044570 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f20180672a0 0x7f2018069760 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f20300af000 tx=0x7f20300046d0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:56:52.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.574+0000 7f202a7fc640 1 -- 172.21.15.164:0/4013044570 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7f203414a0c0 con 0x7f2034138940 2024-03-23T14:56:52.739 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.734+0000 7f203c134640 1 -- 172.21.15.164:0/4013044570 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f203414a0c0 con 0x7f2034138940 2024-03-23T14:56:52.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.738+0000 7f202a7fc640 1 -- 172.21.15.164:0/4013044570 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12908 (secure 0 0 0) 0x7f203414a0c0 con 0x7f2034138940 2024-03-23T14:56:52.741 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-23T14:56:52.741 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":16,"fsid":"45c7911b-b69d-4e68-907b-fbfeb780aa3b","created":"2024-03-23T14:56:34.367633+0000","modified":"2024-03-23T14:56:49.497757+0000","last_up_change":"2024-03-23T14:56:46.297403+0000","last_in_change":"2024-03-23T14:56:36.629884+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T14:56:47.186157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"80abdc62-bd0d-4e60-a554-7ed6fd67cf54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":984716690},{"type":"v1","addr":"172.21.15.164:6801","nonce":984716690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":984716690},{"type":"v1","addr":"172.21.15.164:6803","nonce":984716690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":984716690},{"type":"v1","addr":"172.21.15.164:6807","nonce":984716690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":984716690},{"type":"v1","addr":"172.21.15.164:6805","nonce":984716690}]},"public_addr":"172.21.15.164:6801/984716690","cluster_addr":"172.21.15.164:6803/984716690","heartbeat_back_addr":"172.21.15.164:6807/984716690","heartbeat_front_addr":"172.21.15.164:6805/984716690","state":["exists","up"]},{"osd":1,"uuid":"f9001b06-be4d-4f42-a0e6-bcef2afa0d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":1349657239},{"type":"v1","addr":"172.21.15.164:6817","nonce":1349657239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":1349657239},{"type":"v1","addr":"172.21.15.164:6819","nonce":1349657239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":1349657239},{"type":"v1","addr":"172.21.15.164:6823","nonce":1349657239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":1349657239},{"type":"v1","addr":"172.21.15.164:6821","nonce":1349657239}]},"public_addr":"172.21.15.164:6817/1349657239","cluster_addr":"172.21.15.164:6819/1349657239","heartbeat_back_addr":"172.21.15.164:6823/1349657239","heartbeat_front_addr":"172.21.15.164:6821/1349657239","state":["exists","up"]},{"osd":2,"uuid":"fa0aeb9f-8b54-4831-a188-3e361c7980d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":887861252},{"type":"v1","addr":"172.21.15.164:6809","nonce":887861252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":887861252},{"type":"v1","addr":"172.21.15.164:6811","nonce":887861252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":887861252},{"type":"v1","addr":"172.21.15.164:6815","nonce":887861252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":887861252},{"type":"v1","addr":"172.21.15.164:6813","nonce":887861252}]},"public_addr":"172.21.15.164:6809/887861252","cluster_addr":"172.21.15.164:6811/887861252","heartbeat_back_addr":"172.21.15.164:6815/887861252","heartbeat_front_addr":"172.21.15.164:6813/887861252","state":["exists","up"]},{"osd":3,"uuid":"7e18c957-3091-46d5-8e17-8f3354c1726c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":812540271},{"type":"v1","addr":"172.21.15.164:6825","nonce":812540271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":812540271},{"type":"v1","addr":"172.21.15.164:6827","nonce":812540271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":812540271},{"type":"v1","addr":"172.21.15.164:6831","nonce":812540271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":812540271},{"type":"v1","addr":"172.21.15.164:6829","nonce":812540271}]},"public_addr":"172.21.15.164:6825/812540271","cluster_addr":"172.21.15.164:6827/812540271","heartbeat_back_addr":"172.21.15.164:6831/812540271","heartbeat_front_addr":"172.21.15.164:6829/812540271","state":["exists","up"]},{"osd":4,"uuid":"8e83f315-42bf-4471-9ce8-d81948da0f9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":2656857421},{"type":"v1","addr":"172.21.15.173:6825","nonce":2656857421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2656857421},{"type":"v1","addr":"172.21.15.173:6827","nonce":2656857421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2656857421},{"type":"v1","addr":"172.21.15.173:6831","nonce":2656857421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":2656857421},{"type":"v1","addr":"172.21.15.173:6829","nonce":2656857421}]},"public_addr":"172.21.15.173:6825/2656857421","cluster_addr":"172.21.15.173:6827/2656857421","heartbeat_back_addr":"172.21.15.173:6831/2656857421","heartbeat_front_addr":"172.21.15.173:6829/2656857421","state":["exists","up"]},{"osd":5,"uuid":"a230c872-71f9-40ab-8999-defd0162d4ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":3676483185},{"type":"v1","addr":"172.21.15.173:6801","nonce":3676483185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":3676483185},{"type":"v1","addr":"172.21.15.173:6803","nonce":3676483185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":3676483185},{"type":"v1","addr":"172.21.15.173:6807","nonce":3676483185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":3676483185},{"type":"v1","addr":"172.21.15.173:6805","nonce":3676483185}]},"public_addr":"172.21.15.173:6801/3676483185","cluster_addr":"172.21.15.173:6803/3676483185","heartbeat_back_addr":"172.21.15.173:6807/3676483185","heartbeat_front_addr":"172.21.15.173:6805/3676483185","state":["exists","up"]},{"osd":6,"uuid":"6193bfd7-8432-4f7f-a117-a3fcee21af99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":3631684087},{"type":"v1","addr":"172.21.15.173:6809","nonce":3631684087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":3631684087},{"type":"v1","addr":"172.21.15.173:6811","nonce":3631684087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":3631684087},{"type":"v1","addr":"172.21.15.173:6815","nonce":3631684087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":3631684087},{"type":"v1","addr":"172.21.15.173:6813","nonce":3631684087}]},"public_addr":"172.21.15.173:6809/3631684087","cluster_addr":"172.21.15.173:6811/3631684087","heartbeat_back_addr":"172.21.15.173:6815/3631684087","heartbeat_front_addr":"172.21.15.173:6813/3631684087","state":["exists","up"]},{"osd":7,"uuid":"b0f1b80a-afc4-4538-83b9-f081d2dcb8cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":3261114948},{"type":"v1","addr":"172.21.15.173:6817","nonce":3261114948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":3261114948},{"type":"v1","addr":"172.21.15.173:6819","nonce":3261114948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":3261114948},{"type":"v1","addr":"172.21.15.173:6823","nonce":3261114948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":3261114948},{"type":"v1","addr":"172.21.15.173:6821","nonce":3261114948}]},"public_addr":"172.21.15.173:6817/3261114948","cluster_addr":"172.21.15.173:6819/3261114948","heartbeat_back_addr":"172.21.15.173:6823/3261114948","heartbeat_front_addr":"172.21.15.173:6821/3261114948","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.188390+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.159932+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.191101+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.181776+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.224365+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.177515+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.153631+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.139972+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T14:56:52.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.738+0000 7f203c134640 1 -- 172.21.15.164:0/4013044570 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f20180672a0 msgr2=0x7f2018069760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:56:52.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.738+0000 7f203c134640 1 --2- 172.21.15.164:0/4013044570 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f20180672a0 0x7f2018069760 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f20300af000 tx=0x7f20300046d0 comp rx=0 tx=0).stop 2024-03-23T14:56:52.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.738+0000 7f203c134640 1 -- 172.21.15.164:0/4013044570 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2034138940 msgr2=0x7f2034135f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:56:52.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.738+0000 7f203c134640 1 --2- 172.21.15.164:0/4013044570 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2034138940 0x7f2034135f40 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f202c004bd0 tx=0x7f202c00e530 comp rx=0 tx=0).stop 2024-03-23T14:56:52.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.738+0000 7f203c134640 1 -- 172.21.15.164:0/4013044570 shutdown_connections 2024-03-23T14:56:52.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.738+0000 7f203c134640 1 --2- 172.21.15.164:0/4013044570 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2034138940 0x7f2034135f40 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:52.744 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.738+0000 7f203c134640 1 --2- 172.21.15.164:0/4013044570 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f20180672a0 0x7f2018069760 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:52.744 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.738+0000 7f203c134640 1 --2- 172.21.15.164:0/4013044570 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f20340bd270 0x7f2034137e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:52.744 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.738+0000 7f203c134640 1 --2- 172.21.15.164:0/4013044570 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f20341465c0 0x7f2034137920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:52.744 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.738+0000 7f203c134640 1 -- 172.21.15.164:0/4013044570 >> 172.21.15.164:0/4013044570 conn(0x7f203402c3c0 msgr2=0x7f2034150e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:56:52.744 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.738+0000 7f203c134640 1 -- 172.21.15.164:0/4013044570 shutdown_connections 2024-03-23T14:56:52.744 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.738+0000 7f203c134640 1 -- 172.21.15.164:0/4013044570 wait complete. 2024-03-23T14:56:52.764 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T14:56:47.186157+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-23T14:56:52.764 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-23T14:56:52.884 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.882+0000 7f2f7dde8640 1 Processor -- start 2024-03-23T14:56:52.884 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.882+0000 7f2f7dde8640 1 -- start start 2024-03-23T14:56:52.885 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.882+0000 7f2f7dde8640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f2f780b0230 0x7f2f780b34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:52.886 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.882+0000 7f2f7dde8640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f2f7814db70 0x7f2f7814ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:52.886 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.882+0000 7f2f7dde8640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f780b51d0 con 0x7f2f78148cf0 2024-03-23T14:56:52.886 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.882+0000 7f2f7dde8640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f2f780b0610 con 0x7f2f780b0230 2024-03-23T14:56:52.886 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.882+0000 7f2f7dde8640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 -- 0x7f2f780b1500 con 0x7f2f7814db70 2024-03-23T14:56:52.886 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.882+0000 7f2f777fe640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f2f780b0230 0x7f2f780b34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:52.886 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.882+0000 7f2f6ffff640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f2f78148cf0 0x7f2f78149180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:45396/0 (socket says 172.21.15.164:45396) 2024-03-23T14:56:52.886 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.882+0000 7f2f6ffff640 1 -- 172.21.15.164:0/2509699965 learned_addr learned my addr 172.21.15.164:0/2509699965 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-03-23T14:56:52.886 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.882+0000 7f2f777fe640 1 -- 172.21.15.164:0/2509699965 >> v2:172.21.15.173:3300/0 conn(0x7f2f7814db70 msgr2=0x7f2f7814ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:56:52.886 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.882+0000 7f2f777fe640 1 --2- 172.21.15.164:0/2509699965 >> v2:172.21.15.173:3300/0 conn(0x7f2f7814db70 0x7f2f7814ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:52.886 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.882+0000 7f2f777fe640 1 -- 172.21.15.164:0/2509699965 >> v1:172.21.15.164:6789/0 conn(0x7f2f78148cf0 legacy=0x7f2f78149180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:56:52.886 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.882+0000 7f2f777fe640 1 -- 172.21.15.164:0/2509699965 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f7806d250 con 0x7f2f780b0230 2024-03-23T14:56:52.886 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.882+0000 7f2f76ffd640 1 -- 172.21.15.164:0/2509699965 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3999757703 0 0) 0x7f2f780b51d0 con 0x7f2f78148cf0 2024-03-23T14:56:52.887 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.882+0000 7f2f777fe640 1 --2- 172.21.15.164:0/2509699965 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f780b0230 0x7f2f780b34d0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2f781498c0 tx=0x7f2f6801c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8046ff3f695027a server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:56:52.888 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.886+0000 7f2f76ffd640 1 -- 172.21.15.164:0/2509699965 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f680ab600 con 0x7f2f780b0230 2024-03-23T14:56:52.888 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.886+0000 7f2f76ffd640 1 -- 172.21.15.164:0/2509699965 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f680a7050 con 0x7f2f780b0230 2024-03-23T14:56:52.888 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.886+0000 7f2f76ffd640 1 -- 172.21.15.164:0/2509699965 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f680ab980 con 0x7f2f780b0230 2024-03-23T14:56:52.888 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.886+0000 7f2f7dde8640 1 -- 172.21.15.164:0/2509699965 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f780b0230 msgr2=0x7f2f780b34d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:56:52.889 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.886+0000 7f2f7dde8640 1 --2- 172.21.15.164:0/2509699965 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f780b0230 0x7f2f780b34d0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2f781498c0 tx=0x7f2f6801c330 comp rx=0 tx=0).stop 2024-03-23T14:56:52.889 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.886+0000 7f2f7dde8640 1 -- 172.21.15.164:0/2509699965 shutdown_connections 2024-03-23T14:56:52.889 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.886+0000 7f2f7dde8640 1 --2- 172.21.15.164:0/2509699965 >> v2:172.21.15.173:3300/0 conn(0x7f2f7814db70 0x7f2f7814ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:52.889 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.886+0000 7f2f7dde8640 1 --2- 172.21.15.164:0/2509699965 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f780b0230 0x7f2f780b34d0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:52.889 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.886+0000 7f2f7dde8640 1 -- 172.21.15.164:0/2509699965 >> 172.21.15.164:0/2509699965 conn(0x7f2f780b8ca0 msgr2=0x7f2f780b90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:56:52.889 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.886+0000 7f2f7dde8640 1 -- 172.21.15.164:0/2509699965 shutdown_connections 2024-03-23T14:56:52.889 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.886+0000 7f2f7dde8640 1 -- 172.21.15.164:0/2509699965 wait complete. 2024-03-23T14:56:52.889 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.886+0000 7f2f7dde8640 1 Processor -- start 2024-03-23T14:56:52.890 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.886+0000 7f2f7dde8640 1 -- start start 2024-03-23T14:56:52.892 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.886+0000 7f2f7dde8640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2f780b0230 0x7f2f7814fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:52.892 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.886+0000 7f2f7dde8640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f78148cf0 0x7f2f7809b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:52.892 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.886+0000 7f2f7dde8640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2f7814db70 0x7f2f780ada30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:52.892 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.886+0000 7f2f7dde8640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2f7809cf90 con 0x7f2f78148cf0 2024-03-23T14:56:52.892 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.886+0000 7f2f7dde8640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f2f7809d470 con 0x7f2f780b0230 2024-03-23T14:56:52.892 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.886+0000 7f2f7dde8640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f2f78072130 con 0x7f2f7814db70 2024-03-23T14:56:52.892 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.886+0000 7f2f6ffff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f78148cf0 0x7f2f7809b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:52.892 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.886+0000 7f2f6ffff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f78148cf0 0x7f2f7809b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:57378/0 (socket says 172.21.15.164:57378) 2024-03-23T14:56:52.892 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.886+0000 7f2f6ffff640 1 -- 172.21.15.164:0/714271923 learned_addr learned my addr 172.21.15.164:0/714271923 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:56:52.892 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.886+0000 7f2f77fff640 1 --2- 172.21.15.164:0/714271923 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2f7814db70 0x7f2f780ada30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:52.892 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.886+0000 7f2f6ffff640 1 -- 172.21.15.164:0/714271923 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2f7814db70 msgr2=0x7f2f780ada30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:56:52.893 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.886+0000 7f2f6ffff640 1 --2- 172.21.15.164:0/714271923 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2f7814db70 0x7f2f780ada30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:52.893 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.886+0000 7f2f777fe640 1 --2- 172.21.15.164:0/714271923 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2f780b0230 0x7f2f7814fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:52.893 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.886+0000 7f2f6ffff640 1 -- 172.21.15.164:0/714271923 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2f780b0230 msgr2=0x7f2f7814fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:56:52.893 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.886+0000 7f2f6ffff640 1 --2- 172.21.15.164:0/714271923 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2f780b0230 0x7f2f7814fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:52.893 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.886+0000 7f2f6ffff640 1 -- 172.21.15.164:0/714271923 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f68009420 con 0x7f2f78148cf0 2024-03-23T14:56:52.893 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.886+0000 7f2f6ffff640 1 --2- 172.21.15.164:0/714271923 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f78148cf0 0x7f2f7809b810 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f2f600011f0 tx=0x7f2f6000c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:56:52.893 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.886+0000 7f2f74ff9640 1 -- 172.21.15.164:0/714271923 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f600074e0 con 0x7f2f78148cf0 2024-03-23T14:56:52.893 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.886+0000 7f2f74ff9640 1 -- 172.21.15.164:0/714271923 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f60007640 con 0x7f2f78148cf0 2024-03-23T14:56:52.893 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.886+0000 7f2f74ff9640 1 -- 172.21.15.164:0/714271923 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f60004a10 con 0x7f2f78148cf0 2024-03-23T14:56:52.893 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.886+0000 7f2f7dde8640 1 -- 172.21.15.164:0/714271923 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f78072410 con 0x7f2f78148cf0 2024-03-23T14:56:52.893 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.886+0000 7f2f7dde8640 1 -- 172.21.15.164:0/714271923 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f7809d190 con 0x7f2f78148cf0 2024-03-23T14:56:52.893 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.890+0000 7f2f7dde8640 1 -- 172.21.15.164:0/714271923 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f780690a0 con 0x7f2f78148cf0 2024-03-23T14:56:52.894 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.890+0000 7f2f74ff9640 1 -- 172.21.15.164:0/714271923 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7f2f60004b70 con 0x7f2f78148cf0 2024-03-23T14:56:52.894 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.890+0000 7f2f74ff9640 1 --2- 172.21.15.164:0/714271923 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f2f580672a0 0x7f2f58069760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:52.894 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.890+0000 7f2f74ff9640 1 -- 172.21.15.164:0/714271923 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f2f60083dd0 con 0x7f2f78148cf0 2024-03-23T14:56:52.894 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.890+0000 7f2f777fe640 1 --2- 172.21.15.164:0/714271923 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f2f580672a0 0x7f2f58069760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:52.894 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.890+0000 7f2f777fe640 1 --2- 172.21.15.164:0/714271923 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f2f580672a0 0x7f2f58069760 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f2f680ad000 tx=0x7f2f68008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:56:52.897 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:52.894+0000 7f2f74ff9640 1 -- 172.21.15.164:0/714271923 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7f2f60050d60 con 0x7f2f78148cf0 2024-03-23T14:56:53.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:53.038+0000 7f2f7dde8640 1 -- 172.21.15.164:0/714271923 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f2f780b2fa0 con 0x7f2f78148cf0 2024-03-23T14:56:53.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:53.042+0000 7f2f74ff9640 1 -- 172.21.15.164:0/714271923 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) ==== 93+0+10 (secure 0 0 0) 0x7f2f600180a0 con 0x7f2f78148cf0 2024-03-23T14:56:53.044 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 1 2024-03-23T14:56:53.046 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:53.042+0000 7f2f7dde8640 1 -- 172.21.15.164:0/714271923 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f2f580672a0 msgr2=0x7f2f58069760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:56:53.046 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:53.042+0000 7f2f7dde8640 1 --2- 172.21.15.164:0/714271923 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f2f580672a0 0x7f2f58069760 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f2f680ad000 tx=0x7f2f68008040 comp rx=0 tx=0).stop 2024-03-23T14:56:53.046 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:53.042+0000 7f2f7dde8640 1 -- 172.21.15.164:0/714271923 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f78148cf0 msgr2=0x7f2f7809b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:56:53.046 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:53.042+0000 7f2f7dde8640 1 --2- 172.21.15.164:0/714271923 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f78148cf0 0x7f2f7809b810 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f2f600011f0 tx=0x7f2f6000c9b0 comp rx=0 tx=0).stop 2024-03-23T14:56:53.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:53.042+0000 7f2f7dde8640 1 -- 172.21.15.164:0/714271923 shutdown_connections 2024-03-23T14:56:53.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:53.042+0000 7f2f7dde8640 1 --2- 172.21.15.164:0/714271923 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2f7814db70 0x7f2f780ada30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:53.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:53.042+0000 7f2f7dde8640 1 --2- 172.21.15.164:0/714271923 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f78148cf0 0x7f2f7809b810 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:53.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:53.042+0000 7f2f7dde8640 1 --2- 172.21.15.164:0/714271923 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f2f580672a0 0x7f2f58069760 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:53.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:53.042+0000 7f2f7dde8640 1 --2- 172.21.15.164:0/714271923 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2f780b0230 0x7f2f7814fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:53.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:53.042+0000 7f2f7dde8640 1 -- 172.21.15.164:0/714271923 >> 172.21.15.164:0/714271923 conn(0x7f2f780b8ca0 msgr2=0x7f2f78145cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:56:53.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:53.042+0000 7f2f7dde8640 1 -- 172.21.15.164:0/714271923 shutdown_connections 2024-03-23T14:56:53.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:53.042+0000 7f2f7dde8640 1 -- 172.21.15.164:0/714271923 wait complete. 2024-03-23T14:56:53.062 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T14:56:53.184 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:53.178+0000 7f22d5d12640 1 Processor -- start 2024-03-23T14:56:53.184 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:53.182+0000 7f22d5d12640 1 -- start start 2024-03-23T14:56:53.185 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:53.182+0000 7f22d5d12640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f22d009def0 0x7f22d00ae620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:53.186 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:53.182+0000 7f22d5d12640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f22d00b4e90 0x7f22d014e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:53.186 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:53.182+0000 7f22d5d12640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22d009e2d0 con 0x7f22d00b4a00 2024-03-23T14:56:53.186 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:53.182+0000 7f22d5d12640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f22d0146de0 con 0x7f22d009def0 2024-03-23T14:56:53.186 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:53.182+0000 7f22d5d12640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f22d00b52e0 con 0x7f22d00b4e90 2024-03-23T14:56:53.186 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:53.182+0000 7f22cf7fe640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f22d009def0 0x7f22d00ae620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:53.186 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:53.182+0000 7f22cf7fe640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f22d009def0 0x7f22d00ae620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:57382/0 (socket says 172.21.15.164:57382) 2024-03-23T14:56:53.186 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:53.182+0000 7f22cf7fe640 1 -- 172.21.15.164:0/3617704605 learned_addr learned my addr 172.21.15.164:0/3617704605 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:56:53.186 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:53.182+0000 7f22cffff640 1 --2- 172.21.15.164:0/3617704605 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f22d00b4e90 0x7f22d014e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:53.186 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:53.182+0000 7f22cf7fe640 1 -- 172.21.15.164:0/3617704605 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f22d00b4e90 msgr2=0x7f22d014e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:56:53.186 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:53.182+0000 7f22cf7fe640 1 --2- 172.21.15.164:0/3617704605 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f22d00b4e90 0x7f22d014e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:53.186 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:53.182+0000 7f22cf7fe640 1 -- 172.21.15.164:0/3617704605 >> v1:172.21.15.173:6789/0 conn(0x7f22d00b4a00 legacy=0x7f22d0147420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:56:53.186 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:53.182+0000 7f22cf7fe640 1 -- 172.21.15.164:0/3617704605 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22d006cb70 con 0x7f22d009def0 2024-03-23T14:56:53.188 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:53.182+0000 7f22cf7fe640 1 --2- 172.21.15.164:0/3617704605 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f22d009def0 0x7f22d00ae620 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f22c0009870 tx=0x7f22c001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc498bb7afc4ab94 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:56:53.188 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:53.186+0000 7f22ce7fc640 1 -- 172.21.15.164:0/3617704605 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22c00ac620 con 0x7f22d009def0 2024-03-23T14:56:53.188 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:53.186+0000 7f22ce7fc640 1 -- 172.21.15.164:0/3617704605 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22c00ac780 con 0x7f22d009def0 2024-03-23T14:56:53.188 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:53.186+0000 7f22ce7fc640 1 -- 172.21.15.164:0/3617704605 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22c00acb10 con 0x7f22d009def0 2024-03-23T14:56:53.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:53.186+0000 7f22d5d12640 1 -- 172.21.15.164:0/3617704605 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f22d009def0 msgr2=0x7f22d00ae620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:56:53.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:53.186+0000 7f22d5d12640 1 --2- 172.21.15.164:0/3617704605 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f22d009def0 0x7f22d00ae620 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f22c0009870 tx=0x7f22c001c4d0 comp rx=0 tx=0).stop 2024-03-23T14:56:53.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:53.186+0000 7f22d5d12640 1 -- 172.21.15.164:0/3617704605 shutdown_connections 2024-03-23T14:56:53.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:53.186+0000 7f22d5d12640 1 --2- 172.21.15.164:0/3617704605 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f22d00b4e90 0x7f22d014e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:53.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:53.186+0000 7f22d5d12640 1 --2- 172.21.15.164:0/3617704605 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f22d009def0 0x7f22d00ae620 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:53.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:53.186+0000 7f22d5d12640 1 -- 172.21.15.164:0/3617704605 >> 172.21.15.164:0/3617704605 conn(0x7f22d00b8ca0 msgr2=0x7f22d00b90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:56:53.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:53.186+0000 7f22d5d12640 1 -- 172.21.15.164:0/3617704605 shutdown_connections 2024-03-23T14:56:53.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:53.186+0000 7f22d5d12640 1 -- 172.21.15.164:0/3617704605 wait complete. 2024-03-23T14:56:53.190 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:53.186+0000 7f22d5d12640 1 Processor -- start 2024-03-23T14:56:53.190 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:53.186+0000 7f22d5d12640 1 -- start start 2024-03-23T14:56:53.191 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:53.186+0000 7f22d5d12640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f22d009def0 0x7f22d01408b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:53.192 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:53.186+0000 7f22d5d12640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f22d00b4a00 0x7f22d006f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:53.192 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:53.186+0000 7f22d5d12640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f22d00b4e90 0x7f22d006fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:53.192 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:53.186+0000 7f22d5d12640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f22d0071640 con 0x7f22d00b4a00 2024-03-23T14:56:53.192 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:53.186+0000 7f22d5d12640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f22d0070570 con 0x7f22d009def0 2024-03-23T14:56:53.192 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:53.186+0000 7f22d5d12640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f22d01dc530 con 0x7f22d00b4e90 2024-03-23T14:56:53.192 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:53.186+0000 7f22ceffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f22d00b4a00 0x7f22d006f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:53.192 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:53.186+0000 7f22ceffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f22d00b4a00 0x7f22d006f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:57386/0 (socket says 172.21.15.164:57386) 2024-03-23T14:56:53.192 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:53.186+0000 7f22ceffd640 1 -- 172.21.15.164:0/1406294871 learned_addr learned my addr 172.21.15.164:0/1406294871 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:56:53.192 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:53.186+0000 7f22cffff640 1 --2- 172.21.15.164:0/1406294871 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f22d00b4e90 0x7f22d006fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:53.192 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:53.186+0000 7f22ceffd640 1 -- 172.21.15.164:0/1406294871 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f22d00b4e90 msgr2=0x7f22d006fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:56:53.192 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:53.186+0000 7f22ceffd640 1 --2- 172.21.15.164:0/1406294871 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f22d00b4e90 0x7f22d006fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:53.192 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:53.186+0000 7f22cf7fe640 1 --2- 172.21.15.164:0/1406294871 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f22d009def0 0x7f22d01408b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:53.192 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:53.186+0000 7f22ceffd640 1 -- 172.21.15.164:0/1406294871 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f22d009def0 msgr2=0x7f22d01408b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:56:53.193 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:53.186+0000 7f22ceffd640 1 --2- 172.21.15.164:0/1406294871 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f22d009def0 0x7f22d01408b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:53.193 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:53.186+0000 7f22ceffd640 1 -- 172.21.15.164:0/1406294871 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22c00094d0 con 0x7f22d00b4a00 2024-03-23T14:56:53.193 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:53.186+0000 7f22ceffd640 1 --2- 172.21.15.164:0/1406294871 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f22d00b4a00 0x7f22d006f750 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f22c400c420 tx=0x7f22c400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:56:53.193 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:53.186+0000 7f22b7fff640 1 -- 172.21.15.164:0/1406294871 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22c40052d0 con 0x7f22d00b4a00 2024-03-23T14:56:53.193 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:53.186+0000 7f22b7fff640 1 -- 172.21.15.164:0/1406294871 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22c4005430 con 0x7f22d00b4a00 2024-03-23T14:56:53.193 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:53.186+0000 7f22d5d12640 1 -- 172.21.15.164:0/1406294871 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22d00addc0 con 0x7f22d00b4a00 2024-03-23T14:56:53.193 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:53.186+0000 7f22b7fff640 1 -- 172.21.15.164:0/1406294871 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22c4007260 con 0x7f22d00b4a00 2024-03-23T14:56:53.193 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:53.186+0000 7f22d5d12640 1 -- 172.21.15.164:0/1406294871 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22d009d480 con 0x7f22d00b4a00 2024-03-23T14:56:53.194 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:53.190+0000 7f22d5d12640 1 -- 172.21.15.164:0/1406294871 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22d01dc6a0 con 0x7f22d00b4a00 2024-03-23T14:56:53.194 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:53.190+0000 7f22b7fff640 1 -- 172.21.15.164:0/1406294871 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7f22c40073c0 con 0x7f22d00b4a00 2024-03-23T14:56:53.194 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:53.190+0000 7f22b7fff640 1 --2- 172.21.15.164:0/1406294871 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f22bc0672d0 0x7f22bc069790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:53.194 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:53.190+0000 7f22b7fff640 1 -- 172.21.15.164:0/1406294871 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f22c4084020 con 0x7f22d00b4a00 2024-03-23T14:56:53.194 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:53.190+0000 7f22cf7fe640 1 --2- 172.21.15.164:0/1406294871 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f22bc0672d0 0x7f22bc069790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:53.194 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:53.190+0000 7f22cf7fe640 1 --2- 172.21.15.164:0/1406294871 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f22bc0672d0 0x7f22bc069790 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f22c00ae000 tx=0x7f22c0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:56:53.197 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:53.194+0000 7f22b7fff640 1 -- 172.21.15.164:0/1406294871 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7f22d01dc6a0 con 0x7f22d00b4a00 2024-03-23T14:56:53.386 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:53.382+0000 7f22d5d12640 1 -- 172.21.15.164:0/1406294871 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f22d01dc6a0 con 0x7f22d00b4a00 2024-03-23T14:56:53.386 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:53.382+0000 7f22b7fff640 1 -- 172.21.15.164:0/1406294871 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1395 (secure 0 0 0) 0x7f22d01dc6a0 con 0x7f22d00b4a00 2024-03-23T14:56:53.387 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-23T14:56:53.387 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/3352372536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":3352372536},{"type":"v1","addr":"172.21.15.164:6835","nonce":3352372536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-03-23T14:56:53.387 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 2 2024-03-23T14:56:53.389 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:53.386+0000 7f22d5d12640 1 -- 172.21.15.164:0/1406294871 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f22bc0672d0 msgr2=0x7f22bc069790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:56:53.389 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:53.386+0000 7f22d5d12640 1 --2- 172.21.15.164:0/1406294871 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f22bc0672d0 0x7f22bc069790 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f22c00ae000 tx=0x7f22c0008040 comp rx=0 tx=0).stop 2024-03-23T14:56:53.390 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:53.386+0000 7f22d5d12640 1 -- 172.21.15.164:0/1406294871 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f22d00b4a00 msgr2=0x7f22d006f750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:56:53.390 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:53.386+0000 7f22d5d12640 1 --2- 172.21.15.164:0/1406294871 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f22d00b4a00 0x7f22d006f750 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f22c400c420 tx=0x7f22c400c8e0 comp rx=0 tx=0).stop 2024-03-23T14:56:53.390 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:53.386+0000 7f22d5d12640 1 -- 172.21.15.164:0/1406294871 shutdown_connections 2024-03-23T14:56:53.390 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:53.386+0000 7f22d5d12640 1 --2- 172.21.15.164:0/1406294871 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f22d00b4e90 0x7f22d006fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:53.390 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:53.386+0000 7f22d5d12640 1 --2- 172.21.15.164:0/1406294871 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f22d00b4a00 0x7f22d006f750 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:53.390 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:53.386+0000 7f22d5d12640 1 --2- 172.21.15.164:0/1406294871 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f22bc0672d0 0x7f22bc069790 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:53.390 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:53.386+0000 7f22d5d12640 1 --2- 172.21.15.164:0/1406294871 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f22d009def0 0x7f22d01408b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:53.390 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:53.386+0000 7f22d5d12640 1 -- 172.21.15.164:0/1406294871 >> 172.21.15.164:0/1406294871 conn(0x7f22d00b8ca0 msgr2=0x7f22d0144c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:56:53.390 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:53.386+0000 7f22d5d12640 1 -- 172.21.15.164:0/1406294871 shutdown_connections 2024-03-23T14:56:53.390 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:53.386+0000 7f22d5d12640 1 -- 172.21.15.164:0/1406294871 wait complete. 2024-03-23T14:56:55.405 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T14:56:55.536 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.534+0000 7f5abd4bc640 1 Processor -- start 2024-03-23T14:56:55.537 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.534+0000 7f5abd4bc640 1 -- start start 2024-03-23T14:56:55.539 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.534+0000 7f5abd4bc640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5ab80b7630 0x7f5ab80b55a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:55.539 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.534+0000 7f5abd4bc640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ab80b6260 con 0x7f5ab8145260 2024-03-23T14:56:55.539 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.534+0000 7f5abd4bc640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ab80b5bc0 con 0x7f5ab80b7a10 2024-03-23T14:56:55.539 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.534+0000 7f5abd4bc640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f5ab80b64b0 con 0x7f5ab80b7630 2024-03-23T14:56:55.539 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.534+0000 7f5ab77fe640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f5ab8145260 0x7f5ab80b41d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:45400/0 (socket says 172.21.15.164:45400) 2024-03-23T14:56:55.539 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.534+0000 7f5ab77fe640 1 -- 172.21.15.164:0/4175655983 learned_addr learned my addr 172.21.15.164:0/4175655983 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-03-23T14:56:55.539 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.534+0000 7f5ab6ffd640 1 --2- 172.21.15.164:0/4175655983 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5ab80b7630 0x7f5ab80b55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:55.539 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.534+0000 7f5ab67fc640 1 -- 172.21.15.164:0/4175655983 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1177739555 0 0) 0x7f5ab80b6260 con 0x7f5ab8145260 2024-03-23T14:56:55.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.534+0000 7f5ab67fc640 1 -- 172.21.15.164:0/4175655983 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5aa4003530 con 0x7f5ab8145260 2024-03-23T14:56:55.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.534+0000 7f5ab67fc640 1 -- 172.21.15.164:0/4175655983 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2979156894 0 0) 0x7f5aa4003530 con 0x7f5ab8145260 2024-03-23T14:56:55.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.534+0000 7f5ab67fc640 1 -- 172.21.15.164:0/4175655983 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5ab80b7630 msgr2=0x7f5ab80b55a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:56:55.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.534+0000 7f5ab67fc640 1 --2- 172.21.15.164:0/4175655983 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5ab80b7630 0x7f5ab80b55a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:55.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.534+0000 7f5ab67fc640 1 -- 172.21.15.164:0/4175655983 >> v1:172.21.15.173:6789/0 conn(0x7f5ab80b7a10 legacy=0x7f5ab80b69d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:56:55.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.534+0000 7f5ab67fc640 1 -- 172.21.15.164:0/4175655983 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ab80b4f80 con 0x7f5ab8145260 2024-03-23T14:56:55.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.534+0000 7f5ab67fc640 1 -- 172.21.15.164:0/4175655983 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2526390548 0 0) 0x7f5ab0001a00 con 0x7f5ab8145260 2024-03-23T14:56:55.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.534+0000 7f5ab67fc640 1 -- 172.21.15.164:0/4175655983 >> v1:172.21.15.164:6789/0 conn(0x7f5ab8145260 legacy=0x7f5ab80b41d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:56:55.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.534+0000 7f5ab67fc640 1 --2- 172.21.15.164:0/4175655983 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5aa40039b0 0x7f5aa4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:55.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.534+0000 7f5ab67fc640 1 --2- 172.21.15.164:0/4175655983 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5aa40047d0 0x7f5aa4004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:55.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.534+0000 7f5ab67fc640 1 --2- 172.21.15.164:0/4175655983 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5ab80b7630 0x7f5aa4005260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:55.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.534+0000 7f5ab67fc640 1 -- 172.21.15.164:0/4175655983 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab80b64b0 con 0x7f5aa40039b0 2024-03-23T14:56:55.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.534+0000 7f5ab67fc640 1 -- 172.21.15.164:0/4175655983 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f5aa400bb50 con 0x7f5aa40047d0 2024-03-23T14:56:55.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.534+0000 7f5ab67fc640 1 -- 172.21.15.164:0/4175655983 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f5aa400bd10 con 0x7f5ab80b7630 2024-03-23T14:56:55.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.534+0000 7f5ab6ffd640 1 --2- 172.21.15.164:0/4175655983 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5aa40039b0 0x7f5aa4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:55.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.534+0000 7f5ab77fe640 1 --2- 172.21.15.164:0/4175655983 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5ab80b7630 0x7f5aa4005260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:55.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.534+0000 7f5aaffff640 1 --2- 172.21.15.164:0/4175655983 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5aa40047d0 0x7f5aa4004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:55.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.534+0000 7f5ab6ffd640 1 -- 172.21.15.164:0/4175655983 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5ab80b7630 msgr2=0x7f5aa4005260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:56:55.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.534+0000 7f5ab6ffd640 1 --2- 172.21.15.164:0/4175655983 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5ab80b7630 0x7f5aa4005260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:55.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.534+0000 7f5ab6ffd640 1 -- 172.21.15.164:0/4175655983 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5aa40047d0 msgr2=0x7f5aa4004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:56:55.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.534+0000 7f5ab6ffd640 1 --2- 172.21.15.164:0/4175655983 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5aa40047d0 0x7f5aa4004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:55.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.534+0000 7f5ab6ffd640 1 -- 172.21.15.164:0/4175655983 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5aa4003530 con 0x7f5aa40039b0 2024-03-23T14:56:55.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.534+0000 7f5aaffff640 1 --2- 172.21.15.164:0/4175655983 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5aa40047d0 0x7f5aa4004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T14:56:55.542 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.538+0000 7f5ab6ffd640 1 --2- 172.21.15.164:0/4175655983 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5aa40039b0 0x7f5aa4003dd0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5aa8009ae0 tx=0x7f5aa801ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf08d4703d45cb2a server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:56:55.543 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.538+0000 7f5ab67fc640 1 -- 172.21.15.164:0/4175655983 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aa80a5550 con 0x7f5aa40039b0 2024-03-23T14:56:55.543 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.538+0000 7f5ab67fc640 1 -- 172.21.15.164:0/4175655983 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5aa80a56b0 con 0x7f5aa40039b0 2024-03-23T14:56:55.543 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.538+0000 7f5ab67fc640 1 -- 172.21.15.164:0/4175655983 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aa80a5a60 con 0x7f5aa40039b0 2024-03-23T14:56:55.543 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.538+0000 7f5abd4bc640 1 -- 172.21.15.164:0/4175655983 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5aa40039b0 msgr2=0x7f5aa4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:56:55.543 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.538+0000 7f5abd4bc640 1 --2- 172.21.15.164:0/4175655983 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5aa40039b0 0x7f5aa4003dd0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5aa8009ae0 tx=0x7f5aa801ca70 comp rx=0 tx=0).stop 2024-03-23T14:56:55.543 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.538+0000 7f5ab77fe640 1 -- 172.21.15.164:0/4175655983 reap_dead start 2024-03-23T14:56:55.543 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.538+0000 7f5abd4bc640 1 -- 172.21.15.164:0/4175655983 shutdown_connections 2024-03-23T14:56:55.543 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.538+0000 7f5abd4bc640 1 -- 172.21.15.164:0/4175655983 >> 172.21.15.164:0/4175655983 conn(0x7f5ab802bfc0 msgr2=0x7f5ab802c3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:56:55.543 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.538+0000 7f5abd4bc640 1 -- 172.21.15.164:0/4175655983 shutdown_connections 2024-03-23T14:56:55.543 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.538+0000 7f5abd4bc640 1 -- 172.21.15.164:0/4175655983 wait complete. 2024-03-23T14:56:55.543 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.538+0000 7f5abd4bc640 1 Processor -- start 2024-03-23T14:56:55.543 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.538+0000 7f5abd4bc640 1 -- start start 2024-03-23T14:56:55.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.538+0000 7f5abd4bc640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5aa40039b0 0x7f5ab81ddb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:55.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.538+0000 7f5abd4bc640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5ab814cba0 0x7f5ab81491e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:55.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.538+0000 7f5abd4bc640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5ab813d930 0x7f5ab813dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:55.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.538+0000 7f5abd4bc640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab8147d00 con 0x7f5aa40039b0 2024-03-23T14:56:55.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.538+0000 7f5abd4bc640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab814d970 con 0x7f5ab813d930 2024-03-23T14:56:55.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.538+0000 7f5abd4bc640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f5ab814dec0 con 0x7f5ab814cba0 2024-03-23T14:56:55.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.538+0000 7f5ab6ffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5aa40039b0 0x7f5ab81ddb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:55.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.538+0000 7f5ab6ffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5aa40039b0 0x7f5ab81ddb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:57414/0 (socket says 172.21.15.164:57414) 2024-03-23T14:56:55.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.538+0000 7f5ab6ffd640 1 -- 172.21.15.164:0/688586421 learned_addr learned my addr 172.21.15.164:0/688586421 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:56:55.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.538+0000 7f5aaffff640 1 --2- 172.21.15.164:0/688586421 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5ab814cba0 0x7f5ab81491e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:55.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.538+0000 7f5ab77fe640 1 --2- 172.21.15.164:0/688586421 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5ab813d930 0x7f5ab813dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:55.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.538+0000 7f5ab6ffd640 1 -- 172.21.15.164:0/688586421 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5ab814cba0 msgr2=0x7f5ab81491e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:56:55.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.538+0000 7f5ab6ffd640 1 --2- 172.21.15.164:0/688586421 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5ab814cba0 0x7f5ab81491e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:55.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.538+0000 7f5ab6ffd640 1 -- 172.21.15.164:0/688586421 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5ab813d930 msgr2=0x7f5ab813dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:56:55.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.538+0000 7f5ab6ffd640 1 --2- 172.21.15.164:0/688586421 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5ab813d930 0x7f5ab813dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:55.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.538+0000 7f5ab6ffd640 1 -- 172.21.15.164:0/688586421 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ab814be50 con 0x7f5aa40039b0 2024-03-23T14:56:55.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.538+0000 7f5aaffff640 1 --2- 172.21.15.164:0/688586421 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5ab814cba0 0x7f5ab81491e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T14:56:55.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.538+0000 7f5ab6ffd640 1 --2- 172.21.15.164:0/688586421 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5aa40039b0 0x7f5ab81ddb60 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5aa80b0040 tx=0x7f5aa4005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:56:55.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.542+0000 7f5aaf7fe640 1 -- 172.21.15.164:0/688586421 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aa8004220 con 0x7f5aa40039b0 2024-03-23T14:56:55.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.542+0000 7f5aaf7fe640 1 -- 172.21.15.164:0/688586421 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5aa80043a0 con 0x7f5aa40039b0 2024-03-23T14:56:55.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.542+0000 7f5abd4bc640 1 -- 172.21.15.164:0/688586421 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ab814c3a0 con 0x7f5aa40039b0 2024-03-23T14:56:55.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.542+0000 7f5aaf7fe640 1 -- 172.21.15.164:0/688586421 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aa8004510 con 0x7f5aa40039b0 2024-03-23T14:56:55.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.542+0000 7f5abd4bc640 1 -- 172.21.15.164:0/688586421 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ab814dbe0 con 0x7f5aa40039b0 2024-03-23T14:56:55.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.542+0000 7f5abd4bc640 1 -- 172.21.15.164:0/688586421 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ab80690a0 con 0x7f5aa40039b0 2024-03-23T14:56:55.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.542+0000 7f5aaf7fe640 1 -- 172.21.15.164:0/688586421 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7f5aa80b4040 con 0x7f5aa40039b0 2024-03-23T14:56:55.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.542+0000 7f5aaf7fe640 1 --2- 172.21.15.164:0/688586421 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f5a980672a0 0x7f5a98069760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:55.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.542+0000 7f5aaffff640 1 --2- 172.21.15.164:0/688586421 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f5a980672a0 0x7f5a98069760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:55.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.542+0000 7f5aaf7fe640 1 -- 172.21.15.164:0/688586421 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f5ab8072c60 con 0x7f5aa40039b0 2024-03-23T14:56:55.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.542+0000 7f5aaffff640 1 --2- 172.21.15.164:0/688586421 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f5a980672a0 0x7f5a98069760 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5aa0000c10 tx=0x7f5aa0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:56:55.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.546+0000 7f5aaf7fe640 1 -- 172.21.15.164:0/688586421 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7f5aa80ec230 con 0x7f5aa40039b0 2024-03-23T14:56:55.739 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.734+0000 7f5abd4bc640 1 -- 172.21.15.164:0/688586421 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5ab8144720 con 0x7f5aa40039b0 2024-03-23T14:56:55.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.738+0000 7f5aaf7fe640 1 -- 172.21.15.164:0/688586421 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2222 (secure 0 0 0) 0x7f5aa80ef8f0 con 0x7f5aa40039b0 2024-03-23T14:56:55.740 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-23T14:56:55.740 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/3352372536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":3352372536},{"type":"v1","addr":"172.21.15.164:6835","nonce":3352372536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4279,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/3506402771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":3506402771},{"type":"v1","addr":"172.21.15.173:6833","nonce":3506402771}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-03-23T14:56:55.740 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 3 2024-03-23T14:56:55.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.738+0000 7f5abd4bc640 1 -- 172.21.15.164:0/688586421 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f5a980672a0 msgr2=0x7f5a98069760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:56:55.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.738+0000 7f5abd4bc640 1 --2- 172.21.15.164:0/688586421 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f5a980672a0 0x7f5a98069760 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5aa0000c10 tx=0x7f5aa0009040 comp rx=0 tx=0).stop 2024-03-23T14:56:55.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.738+0000 7f5abd4bc640 1 -- 172.21.15.164:0/688586421 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5aa40039b0 msgr2=0x7f5ab81ddb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:56:55.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.738+0000 7f5abd4bc640 1 --2- 172.21.15.164:0/688586421 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5aa40039b0 0x7f5ab81ddb60 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5aa80b0040 tx=0x7f5aa4005800 comp rx=0 tx=0).stop 2024-03-23T14:56:55.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.738+0000 7f5abd4bc640 1 -- 172.21.15.164:0/688586421 shutdown_connections 2024-03-23T14:56:55.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.738+0000 7f5abd4bc640 1 --2- 172.21.15.164:0/688586421 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f5a980672a0 0x7f5a98069760 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:55.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.738+0000 7f5abd4bc640 1 --2- 172.21.15.164:0/688586421 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5ab813d930 0x7f5ab813dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:55.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.738+0000 7f5abd4bc640 1 --2- 172.21.15.164:0/688586421 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5ab814cba0 0x7f5ab81491e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:55.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.738+0000 7f5abd4bc640 1 --2- 172.21.15.164:0/688586421 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5aa40039b0 0x7f5ab81ddb60 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:55.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.738+0000 7f5abd4bc640 1 -- 172.21.15.164:0/688586421 >> 172.21.15.164:0/688586421 conn(0x7f5ab802bfc0 msgr2=0x7f5ab81434c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:56:55.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.738+0000 7f5abd4bc640 1 -- 172.21.15.164:0/688586421 shutdown_connections 2024-03-23T14:56:55.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.738+0000 7f5abd4bc640 1 -- 172.21.15.164:0/688586421 wait complete. 2024-03-23T14:56:55.759 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T14:56:55.872 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.870+0000 7fa342896640 1 Processor -- start 2024-03-23T14:56:55.872 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.870+0000 7fa342896640 1 -- start start 2024-03-23T14:56:55.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.870+0000 7fa342896640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa33c06e8e0 0x7fa33c141380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:55.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.870+0000 7fa342896640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa33c0b9470 con 0x7fa33c02c2a0 2024-03-23T14:56:55.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.870+0000 7fa342896640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa33c02cb70 con 0x7fa33c02c6e0 2024-03-23T14:56:55.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.870+0000 7fa342896640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fa33c072af0 con 0x7fa33c06e8e0 2024-03-23T14:56:55.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.870+0000 7fa333fff640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fa33c02c2a0 0x7fa33c14a880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:45416/0 (socket says 172.21.15.164:45416) 2024-03-23T14:56:55.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.870+0000 7fa333fff640 1 -- 172.21.15.164:0/852997181 learned_addr learned my addr 172.21.15.164:0/852997181 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-03-23T14:56:55.875 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.870+0000 7fa33bfff640 1 --2- 172.21.15.164:0/852997181 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa33c06e8e0 0x7fa33c141380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:55.875 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.870+0000 7fa33b7fe640 1 -- 172.21.15.164:0/852997181 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 349566535 0 0) 0x7fa33c0b9470 con 0x7fa33c02c2a0 2024-03-23T14:56:55.875 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.870+0000 7fa33b7fe640 1 -- 172.21.15.164:0/852997181 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa31c003530 con 0x7fa33c02c2a0 2024-03-23T14:56:55.875 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.870+0000 7fa33b7fe640 1 -- 172.21.15.164:0/852997181 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3914704117 0 0) 0x7fa31c003530 con 0x7fa33c02c2a0 2024-03-23T14:56:55.875 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.870+0000 7fa33b7fe640 1 -- 172.21.15.164:0/852997181 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa33c06e8e0 msgr2=0x7fa33c141380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:56:55.875 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.870+0000 7fa33b7fe640 1 --2- 172.21.15.164:0/852997181 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa33c06e8e0 0x7fa33c141380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:55.876 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.870+0000 7fa33b7fe640 1 -- 172.21.15.164:0/852997181 >> v1:172.21.15.173:6789/0 conn(0x7fa33c02c6e0 legacy=0x7fa33c14af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:56:55.876 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.870+0000 7fa33b7fe640 1 -- 172.21.15.164:0/852997181 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa33c14b6a0 con 0x7fa33c02c2a0 2024-03-23T14:56:55.876 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.870+0000 7fa33b7fe640 1 -- 172.21.15.164:0/852997181 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2526390548 0 0) 0x7fa324001680 con 0x7fa33c02c2a0 2024-03-23T14:56:55.876 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.870+0000 7fa33b7fe640 1 -- 172.21.15.164:0/852997181 >> v1:172.21.15.164:6789/0 conn(0x7fa33c02c2a0 legacy=0x7fa33c14a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:56:55.876 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.870+0000 7fa33b7fe640 1 --2- 172.21.15.164:0/852997181 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa33c06e8e0 0x7fa31c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:55.876 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.870+0000 7fa33b7fe640 1 --2- 172.21.15.164:0/852997181 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa31c003f30 0x7fa31c0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:55.876 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.870+0000 7fa33b7fe640 1 --2- 172.21.15.164:0/852997181 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa31c004d10 0x7fa31c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:55.876 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.870+0000 7fa33b7fe640 1 -- 172.21.15.164:0/852997181 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa33c072af0 con 0x7fa31c003f30 2024-03-23T14:56:55.876 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.870+0000 7fa33b7fe640 1 -- 172.21.15.164:0/852997181 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa31c00bb00 con 0x7fa31c004d10 2024-03-23T14:56:55.876 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.870+0000 7fa33b7fe640 1 -- 172.21.15.164:0/852997181 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fa31c00bcc0 con 0x7fa33c06e8e0 2024-03-23T14:56:55.876 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.870+0000 7fa333fff640 1 --2- 172.21.15.164:0/852997181 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa31c003f30 0x7fa31c0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:55.876 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.870+0000 7fa333fff640 1 -- 172.21.15.164:0/852997181 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa33c06e8e0 msgr2=0x7fa31c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:56:55.876 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.870+0000 7fa340e33640 1 --2- 172.21.15.164:0/852997181 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa31c004d10 0x7fa31c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:55.876 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.870+0000 7fa333fff640 1 --2- 172.21.15.164:0/852997181 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa33c06e8e0 0x7fa31c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:55.876 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.870+0000 7fa333fff640 1 -- 172.21.15.164:0/852997181 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa31c004d10 msgr2=0x7fa31c005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:56:55.876 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.870+0000 7fa333fff640 1 --2- 172.21.15.164:0/852997181 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa31c004d10 0x7fa31c005190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:55.877 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.870+0000 7fa333fff640 1 -- 172.21.15.164:0/852997181 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa31c003530 con 0x7fa31c003f30 2024-03-23T14:56:55.877 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.874+0000 7fa333fff640 1 --2- 172.21.15.164:0/852997181 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa31c003f30 0x7fa31c0047d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa32400c1c0 tx=0x7fa32401cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53bb084ee9523052 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:56:55.878 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.874+0000 7fa33b7fe640 1 -- 172.21.15.164:0/852997181 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3240a5b40 con 0x7fa31c003f30 2024-03-23T14:56:55.878 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.874+0000 7fa33b7fe640 1 -- 172.21.15.164:0/852997181 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3240a5cc0 con 0x7fa31c003f30 2024-03-23T14:56:55.878 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.874+0000 7fa33b7fe640 1 -- 172.21.15.164:0/852997181 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa324005b20 con 0x7fa31c003f30 2024-03-23T14:56:55.878 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.874+0000 7fa342896640 1 -- 172.21.15.164:0/852997181 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa31c003f30 msgr2=0x7fa31c0047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:56:55.878 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.874+0000 7fa342896640 1 --2- 172.21.15.164:0/852997181 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa31c003f30 0x7fa31c0047d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa32400c1c0 tx=0x7fa32401cc40 comp rx=0 tx=0).stop 2024-03-23T14:56:55.878 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.874+0000 7fa340e33640 1 -- 172.21.15.164:0/852997181 reap_dead start 2024-03-23T14:56:55.878 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.874+0000 7fa342896640 1 -- 172.21.15.164:0/852997181 shutdown_connections 2024-03-23T14:56:55.878 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.874+0000 7fa342896640 1 -- 172.21.15.164:0/852997181 >> 172.21.15.164:0/852997181 conn(0x7fa33c0b7970 msgr2=0x7fa33c0b7d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:56:55.878 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.874+0000 7fa342896640 1 -- 172.21.15.164:0/852997181 shutdown_connections 2024-03-23T14:56:55.878 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.874+0000 7fa342896640 1 -- 172.21.15.164:0/852997181 wait complete. 2024-03-23T14:56:55.880 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.874+0000 7fa342896640 1 Processor -- start 2024-03-23T14:56:55.880 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.874+0000 7fa342896640 1 -- start start 2024-03-23T14:56:55.880 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.874+0000 7fa342896640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa31c003f30 0x7fa33c14a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:55.880 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.874+0000 7fa342896640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa33c144890 0x7fa33c1435f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:55.880 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.874+0000 7fa342896640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa33c143b30 0x7fa33c13cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:55.880 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.874+0000 7fa342896640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa33c143fb0 con 0x7fa33c144890 2024-03-23T14:56:55.881 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.874+0000 7fa342896640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa33c142c10 con 0x7fa33c143b30 2024-03-23T14:56:55.881 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.874+0000 7fa342896640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fa33c141d50 con 0x7fa31c003f30 2024-03-23T14:56:55.881 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.874+0000 7fa333fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa33c144890 0x7fa33c1435f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:55.881 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.874+0000 7fa333fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa33c144890 0x7fa33c1435f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:57434/0 (socket says 172.21.15.164:57434) 2024-03-23T14:56:55.881 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.874+0000 7fa333fff640 1 -- 172.21.15.164:0/3251599724 learned_addr learned my addr 172.21.15.164:0/3251599724 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:56:55.881 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.874+0000 7fa340e33640 1 --2- 172.21.15.164:0/3251599724 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa33c143b30 0x7fa33c13cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:55.881 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.874+0000 7fa33bfff640 1 --2- 172.21.15.164:0/3251599724 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa31c003f30 0x7fa33c14a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:55.881 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.874+0000 7fa333fff640 1 -- 172.21.15.164:0/3251599724 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa31c003f30 msgr2=0x7fa33c14a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:56:55.881 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.874+0000 7fa333fff640 1 --2- 172.21.15.164:0/3251599724 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa31c003f30 0x7fa33c14a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:55.881 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.874+0000 7fa333fff640 1 -- 172.21.15.164:0/3251599724 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa33c143b30 msgr2=0x7fa33c13cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:56:55.881 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.874+0000 7fa333fff640 1 --2- 172.21.15.164:0/3251599724 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa33c143b30 0x7fa33c13cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:55.881 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.874+0000 7fa333fff640 1 -- 172.21.15.164:0/3251599724 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa33c142e80 con 0x7fa33c144890 2024-03-23T14:56:55.881 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.874+0000 7fa333fff640 1 --2- 172.21.15.164:0/3251599724 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa33c144890 0x7fa33c1435f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa32401ca70 tx=0x7fa31c005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:56:55.881 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.874+0000 7fa3397fa640 1 -- 172.21.15.164:0/3251599724 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3240b3070 con 0x7fa33c144890 2024-03-23T14:56:55.881 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.874+0000 7fa3397fa640 1 -- 172.21.15.164:0/3251599724 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3240a5b50 con 0x7fa33c144890 2024-03-23T14:56:55.881 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.874+0000 7fa342896640 1 -- 172.21.15.164:0/3251599724 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa33c14b1a0 con 0x7fa33c144890 2024-03-23T14:56:55.881 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.874+0000 7fa3397fa640 1 -- 172.21.15.164:0/3251599724 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa324001640 con 0x7fa33c144890 2024-03-23T14:56:55.882 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.874+0000 7fa342896640 1 -- 172.21.15.164:0/3251599724 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa33c13d090 con 0x7fa33c144890 2024-03-23T14:56:55.882 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.878+0000 7fa342896640 1 -- 172.21.15.164:0/3251599724 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa33c06e460 con 0x7fa33c144890 2024-03-23T14:56:55.882 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.878+0000 7fa3397fa640 1 -- 172.21.15.164:0/3251599724 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7fa324007140 con 0x7fa33c144890 2024-03-23T14:56:55.882 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.878+0000 7fa3397fa640 1 --2- 172.21.15.164:0/3251599724 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fa3200672a0 0x7fa320069760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:55.882 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.878+0000 7fa3397fa640 1 -- 172.21.15.164:0/3251599724 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fa32411ee30 con 0x7fa33c144890 2024-03-23T14:56:55.883 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.878+0000 7fa33bfff640 1 --2- 172.21.15.164:0/3251599724 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fa3200672a0 0x7fa320069760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:55.883 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.878+0000 7fa33bfff640 1 --2- 172.21.15.164:0/3251599724 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fa3200672a0 0x7fa320069760 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa32c009a70 tx=0x7fa32c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:56:55.885 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:55.882+0000 7fa3397fa640 1 -- 172.21.15.164:0/3251599724 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7fa3240ebdc0 con 0x7fa33c144890 2024-03-23T14:56:56.031 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.026+0000 7fa342896640 1 -- 172.21.15.164:0/3251599724 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa33c069080 con 0x7fa33c144890 2024-03-23T14:56:56.033 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.030+0000 7fa3397fa640 1 -- 172.21.15.164:0/3251599724 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12908 (secure 0 0 0) 0x7fa3240ef480 con 0x7fa33c144890 2024-03-23T14:56:56.033 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-23T14:56:56.033 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":16,"fsid":"45c7911b-b69d-4e68-907b-fbfeb780aa3b","created":"2024-03-23T14:56:34.367633+0000","modified":"2024-03-23T14:56:49.497757+0000","last_up_change":"2024-03-23T14:56:46.297403+0000","last_in_change":"2024-03-23T14:56:36.629884+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T14:56:47.186157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"80abdc62-bd0d-4e60-a554-7ed6fd67cf54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":984716690},{"type":"v1","addr":"172.21.15.164:6801","nonce":984716690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":984716690},{"type":"v1","addr":"172.21.15.164:6803","nonce":984716690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":984716690},{"type":"v1","addr":"172.21.15.164:6807","nonce":984716690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":984716690},{"type":"v1","addr":"172.21.15.164:6805","nonce":984716690}]},"public_addr":"172.21.15.164:6801/984716690","cluster_addr":"172.21.15.164:6803/984716690","heartbeat_back_addr":"172.21.15.164:6807/984716690","heartbeat_front_addr":"172.21.15.164:6805/984716690","state":["exists","up"]},{"osd":1,"uuid":"f9001b06-be4d-4f42-a0e6-bcef2afa0d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":1349657239},{"type":"v1","addr":"172.21.15.164:6817","nonce":1349657239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":1349657239},{"type":"v1","addr":"172.21.15.164:6819","nonce":1349657239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":1349657239},{"type":"v1","addr":"172.21.15.164:6823","nonce":1349657239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":1349657239},{"type":"v1","addr":"172.21.15.164:6821","nonce":1349657239}]},"public_addr":"172.21.15.164:6817/1349657239","cluster_addr":"172.21.15.164:6819/1349657239","heartbeat_back_addr":"172.21.15.164:6823/1349657239","heartbeat_front_addr":"172.21.15.164:6821/1349657239","state":["exists","up"]},{"osd":2,"uuid":"fa0aeb9f-8b54-4831-a188-3e361c7980d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":887861252},{"type":"v1","addr":"172.21.15.164:6809","nonce":887861252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":887861252},{"type":"v1","addr":"172.21.15.164:6811","nonce":887861252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":887861252},{"type":"v1","addr":"172.21.15.164:6815","nonce":887861252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":887861252},{"type":"v1","addr":"172.21.15.164:6813","nonce":887861252}]},"public_addr":"172.21.15.164:6809/887861252","cluster_addr":"172.21.15.164:6811/887861252","heartbeat_back_addr":"172.21.15.164:6815/887861252","heartbeat_front_addr":"172.21.15.164:6813/887861252","state":["exists","up"]},{"osd":3,"uuid":"7e18c957-3091-46d5-8e17-8f3354c1726c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":812540271},{"type":"v1","addr":"172.21.15.164:6825","nonce":812540271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":812540271},{"type":"v1","addr":"172.21.15.164:6827","nonce":812540271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":812540271},{"type":"v1","addr":"172.21.15.164:6831","nonce":812540271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":812540271},{"type":"v1","addr":"172.21.15.164:6829","nonce":812540271}]},"public_addr":"172.21.15.164:6825/812540271","cluster_addr":"172.21.15.164:6827/812540271","heartbeat_back_addr":"172.21.15.164:6831/812540271","heartbeat_front_addr":"172.21.15.164:6829/812540271","state":["exists","up"]},{"osd":4,"uuid":"8e83f315-42bf-4471-9ce8-d81948da0f9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":2656857421},{"type":"v1","addr":"172.21.15.173:6825","nonce":2656857421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2656857421},{"type":"v1","addr":"172.21.15.173:6827","nonce":2656857421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2656857421},{"type":"v1","addr":"172.21.15.173:6831","nonce":2656857421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":2656857421},{"type":"v1","addr":"172.21.15.173:6829","nonce":2656857421}]},"public_addr":"172.21.15.173:6825/2656857421","cluster_addr":"172.21.15.173:6827/2656857421","heartbeat_back_addr":"172.21.15.173:6831/2656857421","heartbeat_front_addr":"172.21.15.173:6829/2656857421","state":["exists","up"]},{"osd":5,"uuid":"a230c872-71f9-40ab-8999-defd0162d4ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":3676483185},{"type":"v1","addr":"172.21.15.173:6801","nonce":3676483185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":3676483185},{"type":"v1","addr":"172.21.15.173:6803","nonce":3676483185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":3676483185},{"type":"v1","addr":"172.21.15.173:6807","nonce":3676483185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":3676483185},{"type":"v1","addr":"172.21.15.173:6805","nonce":3676483185}]},"public_addr":"172.21.15.173:6801/3676483185","cluster_addr":"172.21.15.173:6803/3676483185","heartbeat_back_addr":"172.21.15.173:6807/3676483185","heartbeat_front_addr":"172.21.15.173:6805/3676483185","state":["exists","up"]},{"osd":6,"uuid":"6193bfd7-8432-4f7f-a117-a3fcee21af99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":3631684087},{"type":"v1","addr":"172.21.15.173:6809","nonce":3631684087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":3631684087},{"type":"v1","addr":"172.21.15.173:6811","nonce":3631684087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":3631684087},{"type":"v1","addr":"172.21.15.173:6815","nonce":3631684087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":3631684087},{"type":"v1","addr":"172.21.15.173:6813","nonce":3631684087}]},"public_addr":"172.21.15.173:6809/3631684087","cluster_addr":"172.21.15.173:6811/3631684087","heartbeat_back_addr":"172.21.15.173:6815/3631684087","heartbeat_front_addr":"172.21.15.173:6813/3631684087","state":["exists","up"]},{"osd":7,"uuid":"b0f1b80a-afc4-4538-83b9-f081d2dcb8cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":3261114948},{"type":"v1","addr":"172.21.15.173:6817","nonce":3261114948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":3261114948},{"type":"v1","addr":"172.21.15.173:6819","nonce":3261114948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":3261114948},{"type":"v1","addr":"172.21.15.173:6823","nonce":3261114948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":3261114948},{"type":"v1","addr":"172.21.15.173:6821","nonce":3261114948}]},"public_addr":"172.21.15.173:6817/3261114948","cluster_addr":"172.21.15.173:6819/3261114948","heartbeat_back_addr":"172.21.15.173:6823/3261114948","heartbeat_front_addr":"172.21.15.173:6821/3261114948","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.188390+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.159932+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.191101+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.181776+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.224365+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.177515+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.153631+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.139972+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T14:56:56.035 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.030+0000 7fa342896640 1 -- 172.21.15.164:0/3251599724 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fa3200672a0 msgr2=0x7fa320069760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:56:56.035 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.030+0000 7fa342896640 1 --2- 172.21.15.164:0/3251599724 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fa3200672a0 0x7fa320069760 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa32c009a70 tx=0x7fa32c008040 comp rx=0 tx=0).stop 2024-03-23T14:56:56.035 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.030+0000 7fa342896640 1 -- 172.21.15.164:0/3251599724 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa33c144890 msgr2=0x7fa33c1435f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:56:56.035 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.030+0000 7fa342896640 1 --2- 172.21.15.164:0/3251599724 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa33c144890 0x7fa33c1435f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa32401ca70 tx=0x7fa31c005800 comp rx=0 tx=0).stop 2024-03-23T14:56:56.035 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.030+0000 7fa342896640 1 -- 172.21.15.164:0/3251599724 shutdown_connections 2024-03-23T14:56:56.035 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.030+0000 7fa342896640 1 --2- 172.21.15.164:0/3251599724 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fa3200672a0 0x7fa320069760 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:56.035 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.030+0000 7fa342896640 1 --2- 172.21.15.164:0/3251599724 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa33c143b30 0x7fa33c13cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:56.036 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.030+0000 7fa342896640 1 --2- 172.21.15.164:0/3251599724 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa33c144890 0x7fa33c1435f0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:56.036 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.030+0000 7fa342896640 1 --2- 172.21.15.164:0/3251599724 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa31c003f30 0x7fa33c14a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:56.036 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.030+0000 7fa342896640 1 -- 172.21.15.164:0/3251599724 >> 172.21.15.164:0/3251599724 conn(0x7fa33c0b7970 msgr2=0x7fa33c13d8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:56:56.036 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.030+0000 7fa342896640 1 -- 172.21.15.164:0/3251599724 shutdown_connections 2024-03-23T14:56:56.036 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.030+0000 7fa342896640 1 -- 172.21.15.164:0/3251599724 wait complete. 2024-03-23T14:56:56.051 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T14:56:47.186157+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-23T14:56:56.051 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-23T14:56:56.172 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.170+0000 7fb99e4b2640 1 Processor -- start 2024-03-23T14:56:56.172 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.170+0000 7fb99e4b2640 1 -- start start 2024-03-23T14:56:56.173 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.170+0000 7fb99e4b2640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fb9980b56f0 0x7fb99814a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:56.173 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.170+0000 7fb99e4b2640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fb9980b4a70 0x7fb998140f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:56.174 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.170+0000 7fb99e4b2640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb998141470 0x7fb9980b75f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:56.174 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.170+0000 7fb99e4b2640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fb9980b5fa0 con 0x7fb9980b4a70 2024-03-23T14:56:56.174 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.170+0000 7fb99e4b2640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 -- 0x7fb9980b4300 con 0x7fb9980b56f0 2024-03-23T14:56:56.174 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.170+0000 7fb99e4b2640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fb9980b45e0 con 0x7fb998141470 2024-03-23T14:56:56.174 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.170+0000 7fb9977fe640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fb9980b4a70 0x7fb998140f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:56.174 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.170+0000 7fb9977fe640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fb9980b4a70 0x7fb998140f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:57446/0 (socket says 172.21.15.164:57446) 2024-03-23T14:56:56.174 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.170+0000 7fb9977fe640 1 -- 172.21.15.164:0/581664521 learned_addr learned my addr 172.21.15.164:0/581664521 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:56:56.174 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.170+0000 7fb997fff640 1 --2- 172.21.15.164:0/581664521 >> v2:172.21.15.173:3300/0 conn(0x7fb9980b56f0 0x7fb99814a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:56.174 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.170+0000 7fb99ca4f640 1 --2- 172.21.15.164:0/581664521 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb998141470 0x7fb9980b75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:56.174 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.170+0000 7fb9977fe640 1 -- 172.21.15.164:0/581664521 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb998141470 msgr2=0x7fb9980b75f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:56:56.174 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.170+0000 7fb9977fe640 1 --2- 172.21.15.164:0/581664521 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb998141470 0x7fb9980b75f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:56.174 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.170+0000 7fb9977fe640 1 -- 172.21.15.164:0/581664521 >> v2:172.21.15.173:3300/0 conn(0x7fb9980b56f0 msgr2=0x7fb99814a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:56:56.174 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.170+0000 7fb9977fe640 1 --2- 172.21.15.164:0/581664521 >> v2:172.21.15.173:3300/0 conn(0x7fb9980b56f0 0x7fb99814a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:56.175 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.170+0000 7fb9977fe640 1 -- 172.21.15.164:0/581664521 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb99814ade0 con 0x7fb9980b4a70 2024-03-23T14:56:56.175 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.170+0000 7fb9977fe640 1 --2- 172.21.15.164:0/581664521 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb9980b4a70 0x7fb998140f30 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb9980ba730 tx=0x7fb98c01c230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4219abcebe579f97 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:56:56.175 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.170+0000 7fb996ffd640 1 -- 172.21.15.164:0/581664521 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb98c0ab600 con 0x7fb9980b4a70 2024-03-23T14:56:56.176 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.170+0000 7fb996ffd640 1 -- 172.21.15.164:0/581664521 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb98c0a7050 con 0x7fb9980b4a70 2024-03-23T14:56:56.176 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.170+0000 7fb996ffd640 1 -- 172.21.15.164:0/581664521 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb98c0ab980 con 0x7fb9980b4a70 2024-03-23T14:56:56.176 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.174+0000 7fb99e4b2640 1 -- 172.21.15.164:0/581664521 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb9980b4a70 msgr2=0x7fb998140f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:56:56.176 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.174+0000 7fb99e4b2640 1 --2- 172.21.15.164:0/581664521 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb9980b4a70 0x7fb998140f30 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb9980ba730 tx=0x7fb98c01c230 comp rx=0 tx=0).stop 2024-03-23T14:56:56.177 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.174+0000 7fb99e4b2640 1 -- 172.21.15.164:0/581664521 shutdown_connections 2024-03-23T14:56:56.177 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.174+0000 7fb99e4b2640 1 --2- 172.21.15.164:0/581664521 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb998141470 0x7fb9980b75f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:56.177 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.174+0000 7fb99e4b2640 1 --2- 172.21.15.164:0/581664521 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb9980b4a70 0x7fb998140f30 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:56.177 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.174+0000 7fb99e4b2640 1 --2- 172.21.15.164:0/581664521 >> v2:172.21.15.173:3300/0 conn(0x7fb9980b56f0 0x7fb99814a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:56.177 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.174+0000 7fb99e4b2640 1 -- 172.21.15.164:0/581664521 >> 172.21.15.164:0/581664521 conn(0x7fb99802c3c0 msgr2=0x7fb99802c7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:56:56.177 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.174+0000 7fb99e4b2640 1 -- 172.21.15.164:0/581664521 shutdown_connections 2024-03-23T14:56:56.177 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.174+0000 7fb99e4b2640 1 -- 172.21.15.164:0/581664521 wait complete. 2024-03-23T14:56:56.177 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.174+0000 7fb99e4b2640 1 Processor -- start 2024-03-23T14:56:56.177 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.174+0000 7fb99e4b2640 1 -- start start 2024-03-23T14:56:56.180 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.174+0000 7fb99e4b2640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb9980b4a70 0x7fb9981420d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:56.180 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.174+0000 7fb99e4b2640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb9980b56f0 0x7fb998135ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:56.180 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.174+0000 7fb99e4b2640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb998141470 0x7fb99813b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:56.180 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.174+0000 7fb99e4b2640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb998142610 con 0x7fb9980b56f0 2024-03-23T14:56:56.180 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.174+0000 7fb99e4b2640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fb998143c20 con 0x7fb998141470 2024-03-23T14:56:56.180 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.174+0000 7fb99e4b2640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fb9981447b0 con 0x7fb9980b4a70 2024-03-23T14:56:56.180 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.174+0000 7fb9977fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb9980b56f0 0x7fb998135ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:56.180 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.174+0000 7fb9977fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb9980b56f0 0x7fb998135ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:57450/0 (socket says 172.21.15.164:57450) 2024-03-23T14:56:56.180 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.174+0000 7fb9977fe640 1 -- 172.21.15.164:0/619364406 learned_addr learned my addr 172.21.15.164:0/619364406 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:56:56.180 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.174+0000 7fb99ca4f640 1 --2- 172.21.15.164:0/619364406 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb998141470 0x7fb99813b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:56.180 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.174+0000 7fb997fff640 1 --2- 172.21.15.164:0/619364406 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb9980b4a70 0x7fb9981420d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:56.180 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.174+0000 7fb9977fe640 1 -- 172.21.15.164:0/619364406 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb9980b4a70 msgr2=0x7fb9981420d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:56:56.181 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.174+0000 7fb9977fe640 1 --2- 172.21.15.164:0/619364406 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb9980b4a70 0x7fb9981420d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:56.181 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.174+0000 7fb9977fe640 1 -- 172.21.15.164:0/619364406 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb998141470 msgr2=0x7fb99813b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:56:56.181 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.174+0000 7fb9977fe640 1 --2- 172.21.15.164:0/619364406 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb998141470 0x7fb99813b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:56.181 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.174+0000 7fb9977fe640 1 -- 172.21.15.164:0/619364406 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb98c0094d0 con 0x7fb9980b56f0 2024-03-23T14:56:56.181 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.174+0000 7fb99ca4f640 1 --2- 172.21.15.164:0/619364406 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb998141470 0x7fb99813b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T14:56:56.181 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.174+0000 7fb9977fe640 1 --2- 172.21.15.164:0/619364406 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb9980b56f0 0x7fb998135ac0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb98c0097c0 tx=0x7fb98c018fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:56:56.181 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.174+0000 7fb994ff9640 1 -- 172.21.15.164:0/619364406 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb98c0b4840 con 0x7fb9980b56f0 2024-03-23T14:56:56.181 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.174+0000 7fb994ff9640 1 -- 172.21.15.164:0/619364406 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb98c0b49a0 con 0x7fb9980b56f0 2024-03-23T14:56:56.181 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.174+0000 7fb99e4b2640 1 -- 172.21.15.164:0/619364406 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb998143710 con 0x7fb9980b56f0 2024-03-23T14:56:56.181 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.174+0000 7fb994ff9640 1 -- 172.21.15.164:0/619364406 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb98c0b4d10 con 0x7fb9980b56f0 2024-03-23T14:56:56.181 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.174+0000 7fb99e4b2640 1 -- 172.21.15.164:0/619364406 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb998143940 con 0x7fb9980b56f0 2024-03-23T14:56:56.182 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.178+0000 7fb99e4b2640 1 -- 172.21.15.164:0/619364406 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9980690a0 con 0x7fb9980b56f0 2024-03-23T14:56:56.182 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.178+0000 7fb994ff9640 1 -- 172.21.15.164:0/619364406 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7fb98c0b2040 con 0x7fb9980b56f0 2024-03-23T14:56:56.182 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.178+0000 7fb994ff9640 1 --2- 172.21.15.164:0/619364406 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fb9840672a0 0x7fb984069760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:56.182 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.178+0000 7fb994ff9640 1 -- 172.21.15.164:0/619364406 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fb98c11ef80 con 0x7fb9980b56f0 2024-03-23T14:56:56.182 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.178+0000 7fb997fff640 1 --2- 172.21.15.164:0/619364406 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fb9840672a0 0x7fb984069760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:56.183 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.178+0000 7fb997fff640 1 --2- 172.21.15.164:0/619364406 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fb9840672a0 0x7fb984069760 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb988009710 tx=0x7fb988008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:56:56.185 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.182+0000 7fb994ff9640 1 -- 172.21.15.164:0/619364406 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7fb98c0ebf10 con 0x7fb9980b56f0 2024-03-23T14:56:56.344 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.342+0000 7fb99e4b2640 1 -- 172.21.15.164:0/619364406 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb998144a20 con 0x7fb9980b56f0 2024-03-23T14:56:56.345 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.342+0000 7fb994ff9640 1 -- 172.21.15.164:0/619364406 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) ==== 93+0+10 (secure 0 0 0) 0x7fb98c0b0020 con 0x7fb9980b56f0 2024-03-23T14:56:56.345 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 1 2024-03-23T14:56:56.348 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.342+0000 7fb99e4b2640 1 -- 172.21.15.164:0/619364406 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fb9840672a0 msgr2=0x7fb984069760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:56:56.348 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.342+0000 7fb99e4b2640 1 --2- 172.21.15.164:0/619364406 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fb9840672a0 0x7fb984069760 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb988009710 tx=0x7fb988008040 comp rx=0 tx=0).stop 2024-03-23T14:56:56.348 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.342+0000 7fb99e4b2640 1 -- 172.21.15.164:0/619364406 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb9980b56f0 msgr2=0x7fb998135ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:56:56.348 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.342+0000 7fb99e4b2640 1 --2- 172.21.15.164:0/619364406 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb9980b56f0 0x7fb998135ac0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb98c0097c0 tx=0x7fb98c018fd0 comp rx=0 tx=0).stop 2024-03-23T14:56:56.348 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.342+0000 7fb99e4b2640 1 -- 172.21.15.164:0/619364406 shutdown_connections 2024-03-23T14:56:56.348 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.342+0000 7fb99e4b2640 1 --2- 172.21.15.164:0/619364406 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fb9840672a0 0x7fb984069760 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:56.348 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.342+0000 7fb99e4b2640 1 --2- 172.21.15.164:0/619364406 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb998141470 0x7fb99813b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:56.348 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.342+0000 7fb99e4b2640 1 --2- 172.21.15.164:0/619364406 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb9980b56f0 0x7fb998135ac0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:56.349 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.342+0000 7fb99e4b2640 1 --2- 172.21.15.164:0/619364406 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb9980b4a70 0x7fb9981420d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:56.349 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.342+0000 7fb99e4b2640 1 -- 172.21.15.164:0/619364406 >> 172.21.15.164:0/619364406 conn(0x7fb99802c3c0 msgr2=0x7fb9980bc020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:56:56.349 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.342+0000 7fb99e4b2640 1 -- 172.21.15.164:0/619364406 shutdown_connections 2024-03-23T14:56:56.349 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.342+0000 7fb99e4b2640 1 -- 172.21.15.164:0/619364406 wait complete. 2024-03-23T14:56:56.364 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-23T14:56:56.479 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.474+0000 7fa3c7af4640 1 Processor -- start 2024-03-23T14:56:56.480 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.474+0000 7fa3c7af4640 1 -- start start 2024-03-23T14:56:56.481 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.478+0000 7fa3c7af4640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fa3c00adc90 0x7fa3c00ae070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:56.481 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.478+0000 7fa3c7af4640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa3c014ad20 0x7fa3c014b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:56.481 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.478+0000 7fa3c7af4640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3c006c9c0 con 0x7fa3c006bf70 2024-03-23T14:56:56.481 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.478+0000 7fa3c7af4640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fa3c00ae810 con 0x7fa3c00adc90 2024-03-23T14:56:56.481 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.478+0000 7fa3c7af4640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fa3c00b2ce0 con 0x7fa3c014ad20 2024-03-23T14:56:56.481 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.478+0000 7fa3c5890640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fa3c00adc90 0x7fa3c00ae070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:56.481 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.478+0000 7fa3c508f640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fa3c006bf70 0x7fa3c00b4a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:45432/0 (socket says 172.21.15.164:45432) 2024-03-23T14:56:56.481 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.478+0000 7fa3c508f640 1 -- 172.21.15.164:0/1184718012 learned_addr learned my addr 172.21.15.164:0/1184718012 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-03-23T14:56:56.481 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.478+0000 7fa3c6091640 1 --2- 172.21.15.164:0/1184718012 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa3c014ad20 0x7fa3c014b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:56.482 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.478+0000 7fa3c488e640 1 -- 172.21.15.164:0/1184718012 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1185151760 0 0) 0x7fa3c006c9c0 con 0x7fa3c006bf70 2024-03-23T14:56:56.482 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.478+0000 7fa3c488e640 1 -- 172.21.15.164:0/1184718012 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa3a8003530 con 0x7fa3c006bf70 2024-03-23T14:56:56.482 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.478+0000 7fa3c5890640 1 -- 172.21.15.164:0/1184718012 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa3c014ad20 msgr2=0x7fa3c014b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:56:56.482 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.478+0000 7fa3c5890640 1 --2- 172.21.15.164:0/1184718012 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa3c014ad20 0x7fa3c014b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:56.482 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.478+0000 7fa3c5890640 1 -- 172.21.15.164:0/1184718012 >> v1:172.21.15.164:6789/0 conn(0x7fa3c006bf70 legacy=0x7fa3c00b4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:56:56.482 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.478+0000 7fa3c5890640 1 -- 172.21.15.164:0/1184718012 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3c00af460 con 0x7fa3c00adc90 2024-03-23T14:56:56.483 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.478+0000 7fa3c5890640 1 --2- 172.21.15.164:0/1184718012 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa3c00adc90 0x7fa3c00ae070 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa3b8009d90 tx=0x7fa3b801c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c160447828f8a48 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:56:56.483 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.478+0000 7fa3c488e640 1 -- 172.21.15.164:0/1184718012 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3b80a5800 con 0x7fa3c00adc90 2024-03-23T14:56:56.483 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.478+0000 7fa3c488e640 1 -- 172.21.15.164:0/1184718012 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3b80a5960 con 0x7fa3c00adc90 2024-03-23T14:56:56.483 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.478+0000 7fa3c488e640 1 -- 172.21.15.164:0/1184718012 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3b80a5cd0 con 0x7fa3c00adc90 2024-03-23T14:56:56.484 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.478+0000 7fa3c7af4640 1 -- 172.21.15.164:0/1184718012 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa3c00adc90 msgr2=0x7fa3c00ae070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:56:56.484 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.478+0000 7fa3c7af4640 1 --2- 172.21.15.164:0/1184718012 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa3c00adc90 0x7fa3c00ae070 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa3b8009d90 tx=0x7fa3b801c850 comp rx=0 tx=0).stop 2024-03-23T14:56:56.484 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.478+0000 7fa3c7af4640 1 -- 172.21.15.164:0/1184718012 shutdown_connections 2024-03-23T14:56:56.484 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.478+0000 7fa3c7af4640 1 --2- 172.21.15.164:0/1184718012 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa3c014ad20 0x7fa3c014b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:56.484 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.478+0000 7fa3c7af4640 1 --2- 172.21.15.164:0/1184718012 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa3c00adc90 0x7fa3c00ae070 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:56.484 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.478+0000 7fa3c7af4640 1 -- 172.21.15.164:0/1184718012 >> 172.21.15.164:0/1184718012 conn(0x7fa3c00b8ca0 msgr2=0x7fa3c00b90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:56:56.484 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.478+0000 7fa3c7af4640 1 -- 172.21.15.164:0/1184718012 shutdown_connections 2024-03-23T14:56:56.484 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.478+0000 7fa3c7af4640 1 -- 172.21.15.164:0/1184718012 wait complete. 2024-03-23T14:56:56.485 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.482+0000 7fa3c7af4640 1 Processor -- start 2024-03-23T14:56:56.485 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.482+0000 7fa3c7af4640 1 -- start start 2024-03-23T14:56:56.487 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.482+0000 7fa3c7af4640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa3c00adc90 0x7fa3c009b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:56.487 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.482+0000 7fa3c7af4640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa3c014ad20 0x7fa3c01e3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:56.487 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.482+0000 7fa3c7af4640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa3c0145b10 0x7fa3c006f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:56.488 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.482+0000 7fa3c7af4640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c0145fc0 con 0x7fa3c0145b10 2024-03-23T14:56:56.488 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.482+0000 7fa3c7af4640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c009bfc0 con 0x7fa3c00adc90 2024-03-23T14:56:56.488 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.482+0000 7fa3c7af4640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fa3c01e2f20 con 0x7fa3c014ad20 2024-03-23T14:56:56.488 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.482+0000 7fa3c6091640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa3c0145b10 0x7fa3c006f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:56.488 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.482+0000 7fa3c6091640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa3c0145b10 0x7fa3c006f1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:57470/0 (socket says 172.21.15.164:57470) 2024-03-23T14:56:56.488 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.482+0000 7fa3c6091640 1 -- 172.21.15.164:0/1934245559 learned_addr learned my addr 172.21.15.164:0/1934245559 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:56:56.488 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.482+0000 7fa3c508f640 1 --2- 172.21.15.164:0/1934245559 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa3c014ad20 0x7fa3c01e3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:56.488 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.482+0000 7fa3c5890640 1 --2- 172.21.15.164:0/1934245559 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa3c00adc90 0x7fa3c009b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:56.488 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.482+0000 7fa3c6091640 1 -- 172.21.15.164:0/1934245559 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa3c014ad20 msgr2=0x7fa3c01e3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:56:56.488 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.482+0000 7fa3c6091640 1 --2- 172.21.15.164:0/1934245559 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa3c014ad20 0x7fa3c01e3990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:56.488 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.482+0000 7fa3c6091640 1 -- 172.21.15.164:0/1934245559 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa3c00adc90 msgr2=0x7fa3c009b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:56:56.489 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.482+0000 7fa3c6091640 1 --2- 172.21.15.164:0/1934245559 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa3c00adc90 0x7fa3c009b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:56.489 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.482+0000 7fa3c6091640 1 -- 172.21.15.164:0/1934245559 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3b8009a70 con 0x7fa3c0145b10 2024-03-23T14:56:56.489 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.482+0000 7fa3c5890640 1 --2- 172.21.15.164:0/1934245559 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa3c00adc90 0x7fa3c009b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T14:56:56.489 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.482+0000 7fa3c6091640 1 --2- 172.21.15.164:0/1934245559 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa3c0145b10 0x7fa3c006f1b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa3b000d560 tx=0x7fa3b000da20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:56:56.489 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.482+0000 7fa3b67fc640 1 -- 172.21.15.164:0/1934245559 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3b00125f0 con 0x7fa3c0145b10 2024-03-23T14:56:56.489 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.482+0000 7fa3b67fc640 1 -- 172.21.15.164:0/1934245559 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3b0012770 con 0x7fa3c0145b10 2024-03-23T14:56:56.489 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.482+0000 7fa3c7af4640 1 -- 172.21.15.164:0/1934245559 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3c009c4d0 con 0x7fa3c0145b10 2024-03-23T14:56:56.489 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.482+0000 7fa3b67fc640 1 -- 172.21.15.164:0/1934245559 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3b0012b20 con 0x7fa3c0145b10 2024-03-23T14:56:56.489 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.482+0000 7fa3c7af4640 1 -- 172.21.15.164:0/1934245559 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3c01e3470 con 0x7fa3c0145b10 2024-03-23T14:56:56.489 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.482+0000 7fa3c7af4640 1 -- 172.21.15.164:0/1934245559 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3c00b4d70 con 0x7fa3c0145b10 2024-03-23T14:56:56.490 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.486+0000 7fa3b67fc640 1 -- 172.21.15.164:0/1934245559 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7fa3b0012c80 con 0x7fa3c0145b10 2024-03-23T14:56:56.490 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.486+0000 7fa3b67fc640 1 --2- 172.21.15.164:0/1934245559 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fa3ac0672a0 0x7fa3ac069760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:56.490 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.486+0000 7fa3b67fc640 1 -- 172.21.15.164:0/1934245559 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fa3b00850c0 con 0x7fa3c0145b10 2024-03-23T14:56:56.490 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.486+0000 7fa3c5890640 1 --2- 172.21.15.164:0/1934245559 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fa3ac0672a0 0x7fa3ac069760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:56.490 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.486+0000 7fa3c5890640 1 --2- 172.21.15.164:0/1934245559 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fa3ac0672a0 0x7fa3ac069760 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa3b80af000 tx=0x7fa3b80a7070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:56:56.493 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.490+0000 7fa3b67fc640 1 -- 172.21.15.164:0/1934245559 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7fa3b0052050 con 0x7fa3c0145b10 2024-03-23T14:56:56.643 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.638+0000 7fa3c7af4640 1 -- 172.21.15.164:0/1934245559 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fa3c00690a0 con 0x7fa3c0145b10 2024-03-23T14:56:56.643 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.638+0000 7fa3b67fc640 1 -- 172.21.15.164:0/1934245559 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) ==== 84+0+65 (secure 0 0 0) 0x7fa3b0055710 con 0x7fa3c0145b10 2024-03-23T14:56:56.643 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-23T14:56:56.643 INFO:teuthology.orchestra.run.smithi164.stdout:[ 2024-03-23T14:56:56.643 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-03-23T14:56:56.643 INFO:teuthology.orchestra.run.smithi164.stdout: "poolnum": 1, 2024-03-23T14:56:56.643 INFO:teuthology.orchestra.run.smithi164.stdout: "poolname": ".mgr" 2024-03-23T14:56:56.643 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-03-23T14:56:56.643 INFO:teuthology.orchestra.run.smithi164.stdout:] 2024-03-23T14:56:56.646 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.642+0000 7fa3c7af4640 1 -- 172.21.15.164:0/1934245559 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fa3ac0672a0 msgr2=0x7fa3ac069760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:56:56.646 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.642+0000 7fa3c7af4640 1 --2- 172.21.15.164:0/1934245559 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fa3ac0672a0 0x7fa3ac069760 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa3b80af000 tx=0x7fa3b80a7070 comp rx=0 tx=0).stop 2024-03-23T14:56:56.646 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.642+0000 7fa3c7af4640 1 -- 172.21.15.164:0/1934245559 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa3c0145b10 msgr2=0x7fa3c006f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:56:56.646 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.642+0000 7fa3c7af4640 1 --2- 172.21.15.164:0/1934245559 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa3c0145b10 0x7fa3c006f1b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa3b000d560 tx=0x7fa3b000da20 comp rx=0 tx=0).stop 2024-03-23T14:56:56.646 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.642+0000 7fa3c7af4640 1 -- 172.21.15.164:0/1934245559 shutdown_connections 2024-03-23T14:56:56.646 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.642+0000 7fa3c7af4640 1 --2- 172.21.15.164:0/1934245559 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa3c0145b10 0x7fa3c006f1b0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:56.646 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.642+0000 7fa3c7af4640 1 --2- 172.21.15.164:0/1934245559 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa3c014ad20 0x7fa3c01e3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:56.646 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.642+0000 7fa3c7af4640 1 --2- 172.21.15.164:0/1934245559 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fa3ac0672a0 0x7fa3ac069760 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:56.646 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.642+0000 7fa3c7af4640 1 --2- 172.21.15.164:0/1934245559 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa3c00adc90 0x7fa3c009b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:56.646 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.642+0000 7fa3c7af4640 1 -- 172.21.15.164:0/1934245559 >> 172.21.15.164:0/1934245559 conn(0x7fa3c00b8ca0 msgr2=0x7fa3c00b5cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:56:56.646 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.642+0000 7fa3c7af4640 1 -- 172.21.15.164:0/1934245559 shutdown_connections 2024-03-23T14:56:56.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.642+0000 7fa3c7af4640 1 -- 172.21.15.164:0/1934245559 wait complete. 2024-03-23T14:56:56.661 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-23T14:56:56.661 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-23T14:56:56.787 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.782+0000 7f37ef24b640 1 Processor -- start 2024-03-23T14:56:56.787 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.782+0000 7f37ef24b640 1 -- start start 2024-03-23T14:56:56.787 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.782+0000 7f37ef24b640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f37e80b6360 0x7f37e8145a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:56.787 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.782+0000 7f37ef24b640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37e80b5530 con 0x7f37e80b6740 2024-03-23T14:56:56.787 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.782+0000 7f37ef24b640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37e80b2a70 con 0x7f37e8145f60 2024-03-23T14:56:56.787 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.782+0000 7f37ef24b640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f37e80b5a60 con 0x7f37e80b6360 2024-03-23T14:56:56.787 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.782+0000 7f37ecfe7640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f37e80b6360 0x7f37e8145a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:56.787 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.782+0000 7f37e7fff640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f37e80b6740 0x7f37e80b41d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:45436/0 (socket says 172.21.15.164:45436) 2024-03-23T14:56:56.787 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.782+0000 7f37e7fff640 1 -- 172.21.15.164:0/1441359102 learned_addr learned my addr 172.21.15.164:0/1441359102 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-03-23T14:56:56.787 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.782+0000 7f37ecfe7640 1 -- 172.21.15.164:0/1441359102 >> v1:172.21.15.173:6789/0 conn(0x7f37e8145f60 legacy=0x7f37e814fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:56:56.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.782+0000 7f37ecfe7640 1 -- 172.21.15.164:0/1441359102 >> v1:172.21.15.164:6789/0 conn(0x7f37e80b6740 legacy=0x7f37e80b41d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:56:56.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.782+0000 7f37ecfe7640 1 -- 172.21.15.164:0/1441359102 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37e80b2800 con 0x7f37e80b6360 2024-03-23T14:56:56.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.786+0000 7f37ecfe7640 1 --2- 172.21.15.164:0/1441359102 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f37e80b6360 0x7f37e8145a20 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f37e0009d90 tx=0x7f37e001c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e0bc0d273133de2 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:56:56.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.786+0000 7f37e77fe640 1 -- 172.21.15.164:0/1441359102 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e00a5760 con 0x7f37e80b6360 2024-03-23T14:56:56.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.786+0000 7f37e77fe640 1 -- 172.21.15.164:0/1441359102 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37e00a58c0 con 0x7f37e80b6360 2024-03-23T14:56:56.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.786+0000 7f37e77fe640 1 -- 172.21.15.164:0/1441359102 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e00a5c50 con 0x7f37e80b6360 2024-03-23T14:56:56.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.786+0000 7f37ef24b640 1 -- 172.21.15.164:0/1441359102 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f37e80b6360 msgr2=0x7f37e8145a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:56:56.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.786+0000 7f37ef24b640 1 --2- 172.21.15.164:0/1441359102 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f37e80b6360 0x7f37e8145a20 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f37e0009d90 tx=0x7f37e001c8a0 comp rx=0 tx=0).stop 2024-03-23T14:56:56.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.786+0000 7f37ef24b640 1 -- 172.21.15.164:0/1441359102 shutdown_connections 2024-03-23T14:56:56.790 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.786+0000 7f37ef24b640 1 --2- 172.21.15.164:0/1441359102 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f37e80b6360 0x7f37e8145a20 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:56.790 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.786+0000 7f37ef24b640 1 -- 172.21.15.164:0/1441359102 >> 172.21.15.164:0/1441359102 conn(0x7f37e80b7630 msgr2=0x7f37e80b7a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:56:56.790 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.786+0000 7f37ef24b640 1 -- 172.21.15.164:0/1441359102 shutdown_connections 2024-03-23T14:56:56.790 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.786+0000 7f37ef24b640 1 -- 172.21.15.164:0/1441359102 wait complete. 2024-03-23T14:56:56.792 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.786+0000 7f37ef24b640 1 Processor -- start 2024-03-23T14:56:56.792 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.786+0000 7f37ef24b640 1 -- start start 2024-03-23T14:56:56.792 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.786+0000 7f37ef24b640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f37e80b6360 0x7f37e813ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:56.793 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.786+0000 7f37ef24b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f37e80b6740 0x7f37e813b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:56.793 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.786+0000 7f37ef24b640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f37e8145f60 0x7f37e81de120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:56.793 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.786+0000 7f37ef24b640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f37e80abad0 con 0x7f37e80b6740 2024-03-23T14:56:56.793 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.786+0000 7f37ef24b640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f37e813b810 con 0x7f37e8145f60 2024-03-23T14:56:56.793 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.786+0000 7f37ef24b640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f37e80ab130 con 0x7f37e80b6360 2024-03-23T14:56:56.793 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.786+0000 7f37e7fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f37e80b6740 0x7f37e813b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:56.793 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.786+0000 7f37e7fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f37e80b6740 0x7f37e813b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:57486/0 (socket says 172.21.15.164:57486) 2024-03-23T14:56:56.793 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.786+0000 7f37e7fff640 1 -- 172.21.15.164:0/856068156 learned_addr learned my addr 172.21.15.164:0/856068156 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:56:56.793 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.786+0000 7f37ecfe7640 1 --2- 172.21.15.164:0/856068156 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f37e80b6360 0x7f37e813ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:56.793 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.786+0000 7f37ed7e8640 1 --2- 172.21.15.164:0/856068156 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f37e8145f60 0x7f37e81de120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:56.793 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.786+0000 7f37e7fff640 1 -- 172.21.15.164:0/856068156 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f37e80b6360 msgr2=0x7f37e813ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:56:56.793 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.786+0000 7f37e7fff640 1 --2- 172.21.15.164:0/856068156 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f37e80b6360 0x7f37e813ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:56.794 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.786+0000 7f37e7fff640 1 -- 172.21.15.164:0/856068156 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f37e8145f60 msgr2=0x7f37e81de120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:56:56.794 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.786+0000 7f37e7fff640 1 --2- 172.21.15.164:0/856068156 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f37e8145f60 0x7f37e81de120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:56.794 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.786+0000 7f37e7fff640 1 -- 172.21.15.164:0/856068156 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37e0009a70 con 0x7f37e80b6740 2024-03-23T14:56:56.794 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.786+0000 7f37e7fff640 1 --2- 172.21.15.164:0/856068156 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f37e80b6740 0x7f37e813b2d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f37dc00c460 tx=0x7f37dc00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:56:56.794 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.786+0000 7f37e57fa640 1 -- 172.21.15.164:0/856068156 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37dc005cf0 con 0x7f37e80b6740 2024-03-23T14:56:56.794 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.790+0000 7f37e57fa640 1 -- 172.21.15.164:0/856068156 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37dc005e50 con 0x7f37e80b6740 2024-03-23T14:56:56.794 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.790+0000 7f37ef24b640 1 -- 172.21.15.164:0/856068156 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37e80aa010 con 0x7f37e80b6740 2024-03-23T14:56:56.794 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.790+0000 7f37e57fa640 1 -- 172.21.15.164:0/856068156 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37dc0072f0 con 0x7f37e80b6740 2024-03-23T14:56:56.794 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.790+0000 7f37ef24b640 1 -- 172.21.15.164:0/856068156 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37e80aa830 con 0x7f37e80b6740 2024-03-23T14:56:56.794 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.790+0000 7f37ef24b640 1 -- 172.21.15.164:0/856068156 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37e80690a0 con 0x7f37e80b6740 2024-03-23T14:56:56.795 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.790+0000 7f37e57fa640 1 -- 172.21.15.164:0/856068156 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7f37dc007450 con 0x7f37e80b6740 2024-03-23T14:56:56.795 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.790+0000 7f37e57fa640 1 --2- 172.21.15.164:0/856068156 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f37d40672a0 0x7f37d4069760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:56.795 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.790+0000 7f37e57fa640 1 -- 172.21.15.164:0/856068156 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f37dc084890 con 0x7f37e80b6740 2024-03-23T14:56:56.796 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.790+0000 7f37ecfe7640 1 --2- 172.21.15.164:0/856068156 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f37d40672a0 0x7f37d4069760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:56.796 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.790+0000 7f37ecfe7640 1 --2- 172.21.15.164:0/856068156 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f37d40672a0 0x7f37d4069760 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f37e00af000 tx=0x7f37e00a7070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:56:56.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.794+0000 7f37e57fa640 1 -- 172.21.15.164:0/856068156 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7f37dc051900 con 0x7f37e80b6740 2024-03-23T14:56:56.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:56.950+0000 7f37ef24b640 1 -- 172.21.15.164:0/856068156 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f37e80b24b0 con 0x7f37e80b6740 2024-03-23T14:56:57.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:57.578+0000 7f37e57fa640 1 -- 172.21.15.164:0/856068156 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v17) ==== 138+0+0 (secure 0 0 0) 0x7f37dc017020 con 0x7f37e80b6740 2024-03-23T14:56:57.584 INFO:teuthology.orchestra.run.smithi164.stderr:pool 'cephfs_metadata' created 2024-03-23T14:56:57.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:57.582+0000 7f37ef24b640 1 -- 172.21.15.164:0/856068156 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f37d40672a0 msgr2=0x7f37d4069760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:56:57.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:57.582+0000 7f37ef24b640 1 --2- 172.21.15.164:0/856068156 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f37d40672a0 0x7f37d4069760 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f37e00af000 tx=0x7f37e00a7070 comp rx=0 tx=0).stop 2024-03-23T14:56:57.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:57.582+0000 7f37ef24b640 1 -- 172.21.15.164:0/856068156 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f37e80b6740 msgr2=0x7f37e813b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:56:57.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:57.582+0000 7f37ef24b640 1 --2- 172.21.15.164:0/856068156 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f37e80b6740 0x7f37e813b2d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f37dc00c460 tx=0x7f37dc00c920 comp rx=0 tx=0).stop 2024-03-23T14:56:57.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:57.582+0000 7f37ef24b640 1 -- 172.21.15.164:0/856068156 shutdown_connections 2024-03-23T14:56:57.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:57.582+0000 7f37ef24b640 1 --2- 172.21.15.164:0/856068156 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f37d40672a0 0x7f37d4069760 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:57.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:57.582+0000 7f37ef24b640 1 --2- 172.21.15.164:0/856068156 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f37e8145f60 0x7f37e81de120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:57.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:57.582+0000 7f37ef24b640 1 --2- 172.21.15.164:0/856068156 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f37e80b6740 0x7f37e813b2d0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:57.588 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:57.582+0000 7f37ef24b640 1 --2- 172.21.15.164:0/856068156 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f37e80b6360 0x7f37e813ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:57.588 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:57.582+0000 7f37ef24b640 1 -- 172.21.15.164:0/856068156 >> 172.21.15.164:0/856068156 conn(0x7f37e80b7630 msgr2=0x7f37e80b2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:56:57.588 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:57.582+0000 7f37ef24b640 1 -- 172.21.15.164:0/856068156 shutdown_connections 2024-03-23T14:56:57.588 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:57.582+0000 7f37ef24b640 1 -- 172.21.15.164:0/856068156 wait complete. 2024-03-23T14:56:57.612 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-23T14:56:57.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:57.738+0000 7f740b068640 1 Processor -- start 2024-03-23T14:56:57.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:57.738+0000 7f740b068640 1 -- start start 2024-03-23T14:56:57.744 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:57.738+0000 7f740b068640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f740406e8e0 0x7f7404141380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:57.744 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:57.742+0000 7f740b068640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f740402c6e0 0x7f740414af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:57.744 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:57.742+0000 7f740b068640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74040b9470 con 0x7f740402c2a0 2024-03-23T14:56:57.744 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:57.742+0000 7f740b068640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f740402cb70 con 0x7f740402c6e0 2024-03-23T14:56:57.745 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:57.742+0000 7f740b068640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f7404072af0 con 0x7f740406e8e0 2024-03-23T14:56:57.745 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:57.742+0000 7f7409605640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f740402c6e0 0x7f740414af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:57.745 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:57.742+0000 7f7409605640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f740402c6e0 0x7f740414af90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:57488/0 (socket says 172.21.15.164:57488) 2024-03-23T14:56:57.745 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:57.742+0000 7f7409605640 1 -- 172.21.15.164:0/1162030418 learned_addr learned my addr 172.21.15.164:0/1162030418 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:56:57.745 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:57.742+0000 7f7408e04640 1 --2- 172.21.15.164:0/1162030418 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f740406e8e0 0x7f7404141380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:57.745 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:57.742+0000 7f7409605640 1 -- 172.21.15.164:0/1162030418 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f740406e8e0 msgr2=0x7f7404141380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:56:57.745 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:57.742+0000 7f7409605640 1 --2- 172.21.15.164:0/1162030418 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f740406e8e0 0x7f7404141380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:57.745 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:57.742+0000 7f7409605640 1 -- 172.21.15.164:0/1162030418 >> v1:172.21.15.173:6789/0 conn(0x7f740402c2a0 legacy=0x7f740414a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:56:57.745 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:57.742+0000 7f7409605640 1 -- 172.21.15.164:0/1162030418 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f740406ecc0 con 0x7f740402c6e0 2024-03-23T14:56:57.747 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:57.742+0000 7f7409605640 1 --2- 172.21.15.164:0/1162030418 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f740402c6e0 0x7f740414af90 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f73fc009e50 tx=0x7f73fc01f210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d508b17a07582646 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:56:57.747 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:57.742+0000 7f74037fe640 1 -- 172.21.15.164:0/1162030418 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73fc0af400 con 0x7f740402c6e0 2024-03-23T14:56:57.747 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:57.742+0000 7f74037fe640 1 -- 172.21.15.164:0/1162030418 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73fc0ab050 con 0x7f740402c6e0 2024-03-23T14:56:57.747 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:57.742+0000 7f74037fe640 1 -- 172.21.15.164:0/1162030418 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73fc0af780 con 0x7f740402c6e0 2024-03-23T14:56:57.747 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:57.742+0000 7f740b068640 1 -- 172.21.15.164:0/1162030418 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f740402c6e0 msgr2=0x7f740414af90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:56:57.748 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:57.742+0000 7f740b068640 1 --2- 172.21.15.164:0/1162030418 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f740402c6e0 0x7f740414af90 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f73fc009e50 tx=0x7f73fc01f210 comp rx=0 tx=0).stop 2024-03-23T14:56:57.748 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:57.742+0000 7f740b068640 1 -- 172.21.15.164:0/1162030418 shutdown_connections 2024-03-23T14:56:57.748 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:57.742+0000 7f740b068640 1 --2- 172.21.15.164:0/1162030418 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f740402c6e0 0x7f740414af90 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:57.748 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:57.742+0000 7f740b068640 1 --2- 172.21.15.164:0/1162030418 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f740406e8e0 0x7f7404141380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:57.748 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:57.742+0000 7f740b068640 1 -- 172.21.15.164:0/1162030418 >> 172.21.15.164:0/1162030418 conn(0x7f74040b7970 msgr2=0x7f74040b7d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:56:57.748 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:57.742+0000 7f740b068640 1 -- 172.21.15.164:0/1162030418 shutdown_connections 2024-03-23T14:56:57.748 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:57.742+0000 7f740b068640 1 -- 172.21.15.164:0/1162030418 wait complete. 2024-03-23T14:56:57.748 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:57.746+0000 7f740b068640 1 Processor -- start 2024-03-23T14:56:57.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:57.746+0000 7f740b068640 1 -- start start 2024-03-23T14:56:57.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:57.746+0000 7f740b068640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f740402c2a0 0x7f74041d2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:57.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:57.746+0000 7f740b068640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f740402c6e0 0x7f74040af070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:57.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:57.746+0000 7f740b068640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f740406e8e0 0x7f740413c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:57.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:57.746+0000 7f740b068640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f74040b4330 con 0x7f740402c6e0 2024-03-23T14:56:57.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:57.746+0000 7f740b068640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f7404141d50 con 0x7f740406e8e0 2024-03-23T14:56:57.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:57.746+0000 7f740b068640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f7404142d00 con 0x7f740402c2a0 2024-03-23T14:56:57.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:57.746+0000 7f7403fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f740402c6e0 0x7f74040af070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:57.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:57.746+0000 7f7403fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f740402c6e0 0x7f74040af070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:57490/0 (socket says 172.21.15.164:57490) 2024-03-23T14:56:57.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:57.746+0000 7f7403fff640 1 -- 172.21.15.164:0/3278945785 learned_addr learned my addr 172.21.15.164:0/3278945785 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:56:57.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:57.746+0000 7f7408e04640 1 --2- 172.21.15.164:0/3278945785 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f740402c2a0 0x7f74041d2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:57.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:57.746+0000 7f7409605640 1 --2- 172.21.15.164:0/3278945785 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f740406e8e0 0x7f740413c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:57.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:57.746+0000 7f7403fff640 1 -- 172.21.15.164:0/3278945785 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f740402c2a0 msgr2=0x7f74041d2460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:56:57.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:57.746+0000 7f7403fff640 1 --2- 172.21.15.164:0/3278945785 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f740402c2a0 0x7f74041d2460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:57.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:57.746+0000 7f7403fff640 1 -- 172.21.15.164:0/3278945785 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f740406e8e0 msgr2=0x7f740413c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:56:57.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:57.746+0000 7f7403fff640 1 --2- 172.21.15.164:0/3278945785 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f740406e8e0 0x7f740413c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:57.752 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:57.746+0000 7f7403fff640 1 -- 172.21.15.164:0/3278945785 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73fc009b30 con 0x7f740402c6e0 2024-03-23T14:56:57.752 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:57.746+0000 7f7409605640 1 --2- 172.21.15.164:0/3278945785 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f740406e8e0 0x7f740413c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T14:56:57.752 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:57.746+0000 7f7403fff640 1 --2- 172.21.15.164:0/3278945785 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f740402c6e0 0x7f74040af070 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f73f000c420 tx=0x7f73f000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:56:57.752 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:57.746+0000 7f74017fa640 1 -- 172.21.15.164:0/3278945785 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73f00052d0 con 0x7f740402c6e0 2024-03-23T14:56:57.752 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:57.746+0000 7f74017fa640 1 -- 172.21.15.164:0/3278945785 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73f0005430 con 0x7f740402c6e0 2024-03-23T14:56:57.752 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:57.746+0000 7f74017fa640 1 -- 172.21.15.164:0/3278945785 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73f0007260 con 0x7f740402c6e0 2024-03-23T14:56:57.752 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:57.746+0000 7f740b068640 1 -- 172.21.15.164:0/3278945785 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74040b4050 con 0x7f740402c6e0 2024-03-23T14:56:57.752 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:57.746+0000 7f740b068640 1 -- 172.21.15.164:0/3278945785 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74040af5b0 con 0x7f740402c6e0 2024-03-23T14:56:57.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:57.746+0000 7f740b068640 1 -- 172.21.15.164:0/3278945785 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74040690a0 con 0x7f740402c6e0 2024-03-23T14:56:57.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:57.750+0000 7f74017fa640 1 -- 172.21.15.164:0/3278945785 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7f73f00073c0 con 0x7f740402c6e0 2024-03-23T14:56:57.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:57.750+0000 7f74017fa640 1 --2- 172.21.15.164:0/3278945785 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f73e80672a0 0x7f73e8069760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:57.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:57.750+0000 7f7408e04640 1 --2- 172.21.15.164:0/3278945785 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f73e80672a0 0x7f73e8069760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:57.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:57.750+0000 7f74017fa640 1 -- 172.21.15.164:0/3278945785 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 5227+0+0 (secure 0 0 0) 0x7f73f00841b0 con 0x7f740402c6e0 2024-03-23T14:56:57.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:57.750+0000 7f7408e04640 1 --2- 172.21.15.164:0/3278945785 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f73e80672a0 0x7f73e8069760 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f73f800b930 tx=0x7f73f8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:56:57.757 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:57.754+0000 7f74017fa640 1 -- 172.21.15.164:0/3278945785 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7f73f0050fd0 con 0x7f740402c6e0 2024-03-23T14:56:57.915 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:57.910+0000 7f740b068640 1 -- 172.21.15.164:0/3278945785 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f740406e460 con 0x7f740402c6e0 2024-03-23T14:56:58.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:58.582+0000 7f74017fa640 1 -- 172.21.15.164:0/3278945785 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v18) ==== 170+0+0 (secure 0 0 0) 0x7f73f0054690 con 0x7f740402c6e0 2024-03-23T14:56:58.588 INFO:teuthology.orchestra.run.smithi164.stderr:pool 'cephfs_data' created 2024-03-23T14:56:58.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:58.586+0000 7f740b068640 1 -- 172.21.15.164:0/3278945785 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f73e80672a0 msgr2=0x7f73e8069760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:56:58.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:58.586+0000 7f740b068640 1 --2- 172.21.15.164:0/3278945785 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f73e80672a0 0x7f73e8069760 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f73f800b930 tx=0x7f73f8008040 comp rx=0 tx=0).stop 2024-03-23T14:56:58.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:58.586+0000 7f740b068640 1 -- 172.21.15.164:0/3278945785 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f740402c6e0 msgr2=0x7f74040af070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:56:58.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:58.586+0000 7f740b068640 1 --2- 172.21.15.164:0/3278945785 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f740402c6e0 0x7f74040af070 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f73f000c420 tx=0x7f73f000c8e0 comp rx=0 tx=0).stop 2024-03-23T14:56:58.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:58.586+0000 7f740b068640 1 -- 172.21.15.164:0/3278945785 shutdown_connections 2024-03-23T14:56:58.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:58.586+0000 7f740b068640 1 --2- 172.21.15.164:0/3278945785 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f73e80672a0 0x7f73e8069760 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:58.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:58.586+0000 7f740b068640 1 --2- 172.21.15.164:0/3278945785 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f740406e8e0 0x7f740413c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:58.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:58.586+0000 7f740b068640 1 --2- 172.21.15.164:0/3278945785 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f740402c6e0 0x7f74040af070 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:58.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:58.586+0000 7f740b068640 1 --2- 172.21.15.164:0/3278945785 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f740402c2a0 0x7f74041d2460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:58.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:58.586+0000 7f740b068640 1 -- 172.21.15.164:0/3278945785 >> 172.21.15.164:0/3278945785 conn(0x7f74040b7970 msgr2=0x7f74040b85e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:56:58.592 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:58.590+0000 7f740b068640 1 -- 172.21.15.164:0/3278945785 shutdown_connections 2024-03-23T14:56:58.592 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:58.590+0000 7f740b068640 1 -- 172.21.15.164:0/3278945785 wait complete. 2024-03-23T14:56:58.628 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-23T14:56:58.708 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-23T14:56:58.710 DEBUG:teuthology.orchestra.run.smithi173:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-23T14:56:58.744 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:58.742+0000 7f3e36845640 1 Processor -- start 2024-03-23T14:56:58.744 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:58.742+0000 7f3e36845640 1 -- start start 2024-03-23T14:56:58.745 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:58.742+0000 7f3e36845640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f3e30072610 0x7f3e3013f170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:58.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:58.742+0000 7f3e36845640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f3e3013f6b0 0x7f3e30148760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:58.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:58.742+0000 7f3e36845640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e3006b510 con 0x7f3e30148ca0 2024-03-23T14:56:58.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:58.742+0000 7f3e36845640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f3e300729f0 con 0x7f3e30072610 2024-03-23T14:56:58.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:58.742+0000 7f3e36845640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 -- 0x7f3e300b6390 con 0x7f3e3013f6b0 2024-03-23T14:56:58.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:58.742+0000 7f3e2ffff640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f3e30072610 0x7f3e3013f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:58.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:58.742+0000 7f3e2ffff640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f3e30072610 0x7f3e3013f170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:57500/0 (socket says 172.21.15.164:57500) 2024-03-23T14:56:58.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:58.742+0000 7f3e2ffff640 1 -- 172.21.15.164:0/3372229907 learned_addr learned my addr 172.21.15.164:0/3372229907 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:56:58.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:58.742+0000 7f3e2f7fe640 1 --2- 172.21.15.164:0/3372229907 >> v2:172.21.15.173:3300/0 conn(0x7f3e3013f6b0 0x7f3e30148760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:58.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:58.742+0000 7f3e2ffff640 1 -- 172.21.15.164:0/3372229907 >> v2:172.21.15.173:3300/0 conn(0x7f3e3013f6b0 msgr2=0x7f3e30148760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:56:58.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:58.742+0000 7f3e2ffff640 1 --2- 172.21.15.164:0/3372229907 >> v2:172.21.15.173:3300/0 conn(0x7f3e3013f6b0 0x7f3e30148760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:58.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:58.742+0000 7f3e2ffff640 1 -- 172.21.15.164:0/3372229907 >> v1:172.21.15.173:6789/0 conn(0x7f3e30148ca0 legacy=0x7f3e300b44b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:56:58.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:58.742+0000 7f3e2ffff640 1 -- 172.21.15.164:0/3372229907 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e3006bc50 con 0x7f3e30072610 2024-03-23T14:56:58.748 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:58.746+0000 7f3e2ffff640 1 --2- 172.21.15.164:0/3372229907 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3e30072610 0x7f3e3013f170 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f3e3006d280 tx=0x7f3e2001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dcaf99b3a6882ba server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:56:58.748 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:58.746+0000 7f3e2effd640 1 -- 172.21.15.164:0/3372229907 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e200ab400 con 0x7f3e30072610 2024-03-23T14:56:58.748 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:58.746+0000 7f3e2effd640 1 -- 172.21.15.164:0/3372229907 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e200a7050 con 0x7f3e30072610 2024-03-23T14:56:58.748 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:58.746+0000 7f3e2effd640 1 -- 172.21.15.164:0/3372229907 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e200ab780 con 0x7f3e30072610 2024-03-23T14:56:58.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:58.746+0000 7f3e36845640 1 -- 172.21.15.164:0/3372229907 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3e30072610 msgr2=0x7f3e3013f170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:56:58.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:58.746+0000 7f3e36845640 1 --2- 172.21.15.164:0/3372229907 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3e30072610 0x7f3e3013f170 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f3e3006d280 tx=0x7f3e2001c450 comp rx=0 tx=0).stop 2024-03-23T14:56:58.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:58.746+0000 7f3e36845640 1 -- 172.21.15.164:0/3372229907 shutdown_connections 2024-03-23T14:56:58.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:58.746+0000 7f3e36845640 1 --2- 172.21.15.164:0/3372229907 >> v2:172.21.15.173:3300/0 conn(0x7f3e3013f6b0 0x7f3e30148760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:58.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:58.746+0000 7f3e36845640 1 --2- 172.21.15.164:0/3372229907 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3e30072610 0x7f3e3013f170 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:58.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:58.746+0000 7f3e36845640 1 -- 172.21.15.164:0/3372229907 >> 172.21.15.164:0/3372229907 conn(0x7f3e300b9ab0 msgr2=0x7f3e300b9ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:56:58.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:58.746+0000 7f3e36845640 1 -- 172.21.15.164:0/3372229907 shutdown_connections 2024-03-23T14:56:58.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:58.746+0000 7f3e36845640 1 -- 172.21.15.164:0/3372229907 wait complete. 2024-03-23T14:56:58.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:58.746+0000 7f3e36845640 1 Processor -- start 2024-03-23T14:56:58.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:58.746+0000 7f3e36845640 1 -- start start 2024-03-23T14:56:58.752 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:58.746+0000 7f3e36845640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3e30072610 0x7f3e30134160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:58.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:58.746+0000 7f3e36845640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3e3013f6b0 0x7f3e301e0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:58.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:58.746+0000 7f3e36845640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3e30148ca0 0x7f3e3013b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:58.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:58.746+0000 7f3e36845640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f3e3013b720 con 0x7f3e3013f6b0 2024-03-23T14:56:58.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:58.746+0000 7f3e36845640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f3e30147430 con 0x7f3e30072610 2024-03-23T14:56:58.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:58.746+0000 7f3e36845640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f3e301468e0 con 0x7f3e30148ca0 2024-03-23T14:56:58.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:58.746+0000 7f3e2f7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3e3013f6b0 0x7f3e301e0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:58.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:58.746+0000 7f3e2f7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3e3013f6b0 0x7f3e301e0ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:57502/0 (socket says 172.21.15.164:57502) 2024-03-23T14:56:58.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:58.746+0000 7f3e2f7fe640 1 -- 172.21.15.164:0/3458837496 learned_addr learned my addr 172.21.15.164:0/3458837496 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:56:58.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:58.746+0000 7f3e34de2640 1 --2- 172.21.15.164:0/3458837496 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3e30148ca0 0x7f3e3013b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:58.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:58.746+0000 7f3e2ffff640 1 --2- 172.21.15.164:0/3458837496 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3e30072610 0x7f3e30134160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:58.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:58.746+0000 7f3e2f7fe640 1 -- 172.21.15.164:0/3458837496 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3e30148ca0 msgr2=0x7f3e3013b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:56:58.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:58.746+0000 7f3e2f7fe640 1 --2- 172.21.15.164:0/3458837496 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3e30148ca0 0x7f3e3013b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:58.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:58.746+0000 7f3e2f7fe640 1 -- 172.21.15.164:0/3458837496 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3e30072610 msgr2=0x7f3e30134160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:56:58.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:58.746+0000 7f3e2f7fe640 1 --2- 172.21.15.164:0/3458837496 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3e30072610 0x7f3e30134160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:58.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:58.746+0000 7f3e2f7fe640 1 -- 172.21.15.164:0/3458837496 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e200094d0 con 0x7f3e3013f6b0 2024-03-23T14:56:58.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:58.750+0000 7f3e2f7fe640 1 --2- 172.21.15.164:0/3458837496 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3e3013f6b0 0x7f3e301e0ea0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3e2400c7f0 tx=0x7f3e2400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:56:58.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:58.750+0000 7f3e2cff9640 1 -- 172.21.15.164:0/3458837496 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e24004220 con 0x7f3e3013f6b0 2024-03-23T14:56:58.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:58.750+0000 7f3e2cff9640 1 -- 172.21.15.164:0/3458837496 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e240043a0 con 0x7f3e3013f6b0 2024-03-23T14:56:58.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:58.750+0000 7f3e36845640 1 -- 172.21.15.164:0/3458837496 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e30139890 con 0x7f3e3013f6b0 2024-03-23T14:56:58.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:58.750+0000 7f3e2cff9640 1 -- 172.21.15.164:0/3458837496 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e24004510 con 0x7f3e3013f6b0 2024-03-23T14:56:58.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:58.750+0000 7f3e36845640 1 -- 172.21.15.164:0/3458837496 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e30146560 con 0x7f3e3013f6b0 2024-03-23T14:56:58.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:58.750+0000 7f3e36845640 1 -- 172.21.15.164:0/3458837496 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e300686f0 con 0x7f3e3013f6b0 2024-03-23T14:56:58.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:58.750+0000 7f3e2cff9640 1 -- 172.21.15.164:0/3458837496 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7f3e24003a70 con 0x7f3e3013f6b0 2024-03-23T14:56:58.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:58.750+0000 7f3e2cff9640 1 --2- 172.21.15.164:0/3458837496 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f3e1c0672a0 0x7f3e1c069760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:58.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:58.750+0000 7f3e2cff9640 1 -- 172.21.15.164:0/3458837496 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 5609+0+0 (secure 0 0 0) 0x7f3e240851c0 con 0x7f3e3013f6b0 2024-03-23T14:56:58.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:58.750+0000 7f3e2ffff640 1 --2- 172.21.15.164:0/3458837496 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f3e1c0672a0 0x7f3e1c069760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:58.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:58.750+0000 7f3e2ffff640 1 --2- 172.21.15.164:0/3458837496 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f3e1c0672a0 0x7f3e1c069760 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3e200ad000 tx=0x7f3e20008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:56:58.758 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:58.754+0000 7f3e2cff9640 1 -- 172.21.15.164:0/3458837496 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7f3e24051ff0 con 0x7f3e3013f6b0 2024-03-23T14:56:58.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:58.946+0000 7f3e36845640 1 -- 172.21.15.164:0/3458837496 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f3e301475c0 con 0x7f3e3013f6b0 2024-03-23T14:56:59.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:59.606+0000 7f3e2cff9640 1 -- 172.21.15.164:0/3458837496 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7f3e240556b0 con 0x7f3e3013f6b0 2024-03-23T14:56:59.611 INFO:teuthology.orchestra.run.smithi164.stderr:new fs with metadata pool 2 and data pool 3 2024-03-23T14:56:59.614 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:59.610+0000 7f3e36845640 1 -- 172.21.15.164:0/3458837496 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f3e1c0672a0 msgr2=0x7f3e1c069760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:56:59.614 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:59.610+0000 7f3e36845640 1 --2- 172.21.15.164:0/3458837496 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f3e1c0672a0 0x7f3e1c069760 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3e200ad000 tx=0x7f3e20008040 comp rx=0 tx=0).stop 2024-03-23T14:56:59.614 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:59.610+0000 7f3e36845640 1 -- 172.21.15.164:0/3458837496 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3e3013f6b0 msgr2=0x7f3e301e0ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:56:59.614 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:59.610+0000 7f3e36845640 1 --2- 172.21.15.164:0/3458837496 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3e3013f6b0 0x7f3e301e0ea0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3e2400c7f0 tx=0x7f3e2400ccb0 comp rx=0 tx=0).stop 2024-03-23T14:56:59.614 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:59.610+0000 7f3e36845640 1 -- 172.21.15.164:0/3458837496 shutdown_connections 2024-03-23T14:56:59.615 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:59.610+0000 7f3e36845640 1 --2- 172.21.15.164:0/3458837496 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3e30148ca0 0x7f3e3013b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:59.615 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:59.610+0000 7f3e36845640 1 --2- 172.21.15.164:0/3458837496 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3e3013f6b0 0x7f3e301e0ea0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:59.615 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:59.610+0000 7f3e36845640 1 --2- 172.21.15.164:0/3458837496 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f3e1c0672a0 0x7f3e1c069760 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:59.615 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:59.610+0000 7f3e36845640 1 --2- 172.21.15.164:0/3458837496 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3e30072610 0x7f3e30134160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:59.615 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:59.610+0000 7f3e36845640 1 -- 172.21.15.164:0/3458837496 >> 172.21.15.164:0/3458837496 conn(0x7f3e300b9ab0 msgr2=0x7f3e300b6d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:56:59.615 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:59.610+0000 7f3e36845640 1 -- 172.21.15.164:0/3458837496 shutdown_connections 2024-03-23T14:56:59.615 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:59.610+0000 7f3e36845640 1 -- 172.21.15.164:0/3458837496 wait complete. 2024-03-23T14:56:59.649 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T14:56:59.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:59.766+0000 7f4cff8f9640 1 Processor -- start 2024-03-23T14:56:59.772 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:59.766+0000 7f4cff8f9640 1 -- start start 2024-03-23T14:56:59.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:59.770+0000 7f4cff8f9640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4cf8141640 0x7f4cf8152c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:59.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:59.770+0000 7f4cff8f9640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4cf80b4300 con 0x7f4cf80b56f0 2024-03-23T14:56:59.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:59.770+0000 7f4cff8f9640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4cf80b45e0 con 0x7f4cf80b4a70 2024-03-23T14:56:59.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:59.770+0000 7f4cff8f9640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f4cf80b5fa0 con 0x7f4cf8141640 2024-03-23T14:56:59.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:59.770+0000 7f4cfd695640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f4cf80b56f0 0x7f4cf814a8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:45448/0 (socket says 172.21.15.164:45448) 2024-03-23T14:56:59.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:59.770+0000 7f4cfd695640 1 -- 172.21.15.164:0/4057793441 learned_addr learned my addr 172.21.15.164:0/4057793441 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-03-23T14:56:59.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:59.770+0000 7f4cfde96640 1 --2- 172.21.15.164:0/4057793441 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4cf8141640 0x7f4cf8152c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:59.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:59.770+0000 7f4ceffff640 1 -- 172.21.15.164:0/4057793441 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 973430293 0 0) 0x7f4cf80b4300 con 0x7f4cf80b56f0 2024-03-23T14:56:59.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:59.770+0000 7f4ceffff640 1 -- 172.21.15.164:0/4057793441 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4ce4003530 con 0x7f4cf80b56f0 2024-03-23T14:56:59.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:59.770+0000 7f4ceffff640 1 -- 172.21.15.164:0/4057793441 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3511456096 0 0) 0x7f4ce4003530 con 0x7f4cf80b56f0 2024-03-23T14:56:59.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:59.770+0000 7f4ceffff640 1 -- 172.21.15.164:0/4057793441 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4cf8141640 msgr2=0x7f4cf8152c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:56:59.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:59.770+0000 7f4ceffff640 1 --2- 172.21.15.164:0/4057793441 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4cf8141640 0x7f4cf8152c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:59.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:59.770+0000 7f4ceffff640 1 -- 172.21.15.164:0/4057793441 >> v1:172.21.15.173:6789/0 conn(0x7f4cf80b4a70 legacy=0x7f4cf8140f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:56:59.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:59.770+0000 7f4ceffff640 1 -- 172.21.15.164:0/4057793441 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cf80bd7e0 con 0x7f4cf80b56f0 2024-03-23T14:56:59.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:59.770+0000 7f4ceffff640 1 -- 172.21.15.164:0/4057793441 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2526390548 0 0) 0x7f4ce8001680 con 0x7f4cf80b56f0 2024-03-23T14:56:59.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:59.770+0000 7f4ceffff640 1 -- 172.21.15.164:0/4057793441 >> v1:172.21.15.164:6789/0 conn(0x7f4cf80b56f0 legacy=0x7f4cf814a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:56:59.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:59.770+0000 7f4ceffff640 1 --2- 172.21.15.164:0/4057793441 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4ce40039b0 0x7f4ce4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:59.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:59.770+0000 7f4ceffff640 1 --2- 172.21.15.164:0/4057793441 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4cf8141640 0x7f4ce40047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:59.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:59.770+0000 7f4ceffff640 1 --2- 172.21.15.164:0/4057793441 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4ce4004d10 0x7f4ce4005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:59.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:59.770+0000 7f4ceffff640 1 -- 172.21.15.164:0/4057793441 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce400ba40 con 0x7f4ce4004d10 2024-03-23T14:56:59.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:59.770+0000 7f4ceffff640 1 -- 172.21.15.164:0/4057793441 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce400bb80 con 0x7f4ce40039b0 2024-03-23T14:56:59.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:59.770+0000 7f4ceffff640 1 -- 172.21.15.164:0/4057793441 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f4ce400bd40 con 0x7f4cf8141640 2024-03-23T14:56:59.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:59.770+0000 7f4cfde96640 1 --2- 172.21.15.164:0/4057793441 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4ce4004d10 0x7f4ce4005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:59.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:59.770+0000 7f4cfd695640 1 --2- 172.21.15.164:0/4057793441 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4ce40039b0 0x7f4ce4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:59.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:59.770+0000 7f4cfce94640 1 --2- 172.21.15.164:0/4057793441 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4cf8141640 0x7f4ce40047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:59.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:59.770+0000 7f4cfde96640 1 -- 172.21.15.164:0/4057793441 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4cf8141640 msgr2=0x7f4ce40047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:56:59.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:59.770+0000 7f4cfde96640 1 --2- 172.21.15.164:0/4057793441 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4cf8141640 0x7f4ce40047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:59.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:59.770+0000 7f4cfde96640 1 -- 172.21.15.164:0/4057793441 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4ce40039b0 msgr2=0x7f4ce4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:56:59.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:59.770+0000 7f4cfde96640 1 --2- 172.21.15.164:0/4057793441 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4ce40039b0 0x7f4ce4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:59.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:59.770+0000 7f4cfde96640 1 -- 172.21.15.164:0/4057793441 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ce4003530 con 0x7f4ce4004d10 2024-03-23T14:56:59.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:59.770+0000 7f4cfde96640 1 --2- 172.21.15.164:0/4057793441 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4ce4004d10 0x7f4ce4005190 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4cf0009f90 tx=0x7f4cf001cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a280c664dec17a6d server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:56:59.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:59.774+0000 7f4ceffff640 1 -- 172.21.15.164:0/4057793441 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cf00a5c50 con 0x7f4ce4004d10 2024-03-23T14:56:59.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:59.774+0000 7f4ceffff640 1 -- 172.21.15.164:0/4057793441 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4cf000f040 con 0x7f4ce4004d10 2024-03-23T14:56:59.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:59.774+0000 7f4ceffff640 1 -- 172.21.15.164:0/4057793441 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cf00ad440 con 0x7f4ce4004d10 2024-03-23T14:56:59.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:59.774+0000 7f4cff8f9640 1 -- 172.21.15.164:0/4057793441 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4ce4004d10 msgr2=0x7f4ce4005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:56:59.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:59.774+0000 7f4cff8f9640 1 --2- 172.21.15.164:0/4057793441 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4ce4004d10 0x7f4ce4005190 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4cf0009f90 tx=0x7f4cf001cc40 comp rx=0 tx=0).stop 2024-03-23T14:56:59.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:59.774+0000 7f4cfde96640 1 -- 172.21.15.164:0/4057793441 reap_dead start 2024-03-23T14:56:59.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:59.774+0000 7f4cff8f9640 1 -- 172.21.15.164:0/4057793441 shutdown_connections 2024-03-23T14:56:59.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:59.774+0000 7f4cff8f9640 1 -- 172.21.15.164:0/4057793441 >> 172.21.15.164:0/4057793441 conn(0x7f4cf802c3c0 msgr2=0x7f4cf802c7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:56:59.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:59.774+0000 7f4cff8f9640 1 -- 172.21.15.164:0/4057793441 shutdown_connections 2024-03-23T14:56:59.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:59.774+0000 7f4cff8f9640 1 -- 172.21.15.164:0/4057793441 wait complete. 2024-03-23T14:56:59.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:59.774+0000 7f4cff8f9640 1 Processor -- start 2024-03-23T14:56:59.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:59.774+0000 7f4cff8f9640 1 -- start start 2024-03-23T14:56:59.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:59.774+0000 7f4cff8f9640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4cf814b380 0x7f4cf81e08f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:59.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:59.774+0000 7f4cff8f9640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4cf8148c60 0x7f4cf81490e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:59.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:59.774+0000 7f4cff8f9640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4cf8149620 0x7f4cf81dc8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:59.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:59.774+0000 7f4cff8f9640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f4cf81dfee0 con 0x7f4cf814b380 2024-03-23T14:56:59.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:59.774+0000 7f4cff8f9640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f4cf81e0e30 con 0x7f4cf8149620 2024-03-23T14:56:59.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:59.774+0000 7f4cff8f9640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f4cf8149aa0 con 0x7f4cf8148c60 2024-03-23T14:56:59.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:59.774+0000 7f4cfd695640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4cf814b380 0x7f4cf81e08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:59.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:59.774+0000 7f4cfd695640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4cf814b380 0x7f4cf81e08f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:57508/0 (socket says 172.21.15.164:57508) 2024-03-23T14:56:59.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:59.774+0000 7f4cfd695640 1 -- 172.21.15.164:0/3911468423 learned_addr learned my addr 172.21.15.164:0/3911468423 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:56:59.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:59.774+0000 7f4cfce94640 1 --2- 172.21.15.164:0/3911468423 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4cf8148c60 0x7f4cf81490e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:59.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:59.774+0000 7f4cfde96640 1 --2- 172.21.15.164:0/3911468423 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4cf8149620 0x7f4cf81dc8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:59.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:59.774+0000 7f4cfd695640 1 -- 172.21.15.164:0/3911468423 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4cf8148c60 msgr2=0x7f4cf81490e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:56:59.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:59.774+0000 7f4cfd695640 1 --2- 172.21.15.164:0/3911468423 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4cf8148c60 0x7f4cf81490e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:59.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:59.774+0000 7f4cfd695640 1 -- 172.21.15.164:0/3911468423 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4cf8149620 msgr2=0x7f4cf81dc8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:56:59.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:59.774+0000 7f4cfd695640 1 --2- 172.21.15.164:0/3911468423 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4cf8149620 0x7f4cf81dc8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:59.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:59.774+0000 7f4cfd695640 1 -- 172.21.15.164:0/3911468423 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cf814b760 con 0x7f4cf814b380 2024-03-23T14:56:59.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:59.774+0000 7f4cfd695640 1 --2- 172.21.15.164:0/3911468423 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4cf814b380 0x7f4cf81e08f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4ce800eb50 tx=0x7f4ce800c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:56:59.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:59.774+0000 7f4cedffb640 1 -- 172.21.15.164:0/3911468423 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce8005e20 con 0x7f4cf814b380 2024-03-23T14:56:59.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:59.774+0000 7f4cedffb640 1 -- 172.21.15.164:0/3911468423 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ce800ee50 con 0x7f4cf814b380 2024-03-23T14:56:59.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:59.774+0000 7f4cff8f9640 1 -- 172.21.15.164:0/3911468423 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cf81e00f0 con 0x7f4cf814b380 2024-03-23T14:56:59.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:59.774+0000 7f4cedffb640 1 -- 172.21.15.164:0/3911468423 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce800ce00 con 0x7f4cf814b380 2024-03-23T14:56:59.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:59.774+0000 7f4cff8f9640 1 -- 172.21.15.164:0/3911468423 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cf814ac00 con 0x7f4cf814b380 2024-03-23T14:56:59.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:59.778+0000 7f4cff8f9640 1 -- 172.21.15.164:0/3911468423 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4cf80690a0 con 0x7f4cf814b380 2024-03-23T14:56:59.783 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:59.778+0000 7f4cedffb640 1 -- 172.21.15.164:0/3911468423 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7f4ce801d460 con 0x7f4cf814b380 2024-03-23T14:56:59.783 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:59.778+0000 7f4cedffb640 1 --2- 172.21.15.164:0/3911468423 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f4cdc0672d0 0x7f4cdc069790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:56:59.783 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:59.778+0000 7f4cfce94640 1 --2- 172.21.15.164:0/3911468423 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f4cdc0672d0 0x7f4cdc069790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:56:59.783 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:59.778+0000 7f4cedffb640 1 -- 172.21.15.164:0/3911468423 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5709+0+0 (secure 0 0 0) 0x7f4ce8084fd0 con 0x7f4cf814b380 2024-03-23T14:56:59.783 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:59.778+0000 7f4cfce94640 1 --2- 172.21.15.164:0/3911468423 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f4cdc0672d0 0x7f4cdc069790 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4cf8146ba0 tx=0x7f4cf4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:56:59.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:59.782+0000 7f4cedffb640 1 -- 172.21.15.164:0/3911468423 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7f4ce8055800 con 0x7f4cf814b380 2024-03-23T14:56:59.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:59.950+0000 7f4cff8f9640 1 -- 172.21.15.164:0/3911468423 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4cf8148250 con 0x7f4cf814b380 2024-03-23T14:56:59.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:59.950+0000 7f4cedffb640 1 -- 172.21.15.164:0/3911468423 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) ==== 74+0+16642 (secure 0 0 0) 0x7f4ce8055390 con 0x7f4cf814b380 2024-03-23T14:56:59.956 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-23T14:56:59.956 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":19,"fsid":"45c7911b-b69d-4e68-907b-fbfeb780aa3b","created":"2024-03-23T14:56:34.367633+0000","modified":"2024-03-23T14:56:58.952999+0000","last_up_change":"2024-03-23T14:56:46.297403+0000","last_in_change":"2024-03-23T14:56:36.629884+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T14:56:47.186157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-23T14:56:56.994057+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-23T14:56:57.953862+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"80abdc62-bd0d-4e60-a554-7ed6fd67cf54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":984716690},{"type":"v1","addr":"172.21.15.164:6801","nonce":984716690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":984716690},{"type":"v1","addr":"172.21.15.164:6803","nonce":984716690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":984716690},{"type":"v1","addr":"172.21.15.164:6807","nonce":984716690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":984716690},{"type":"v1","addr":"172.21.15.164:6805","nonce":984716690}]},"public_addr":"172.21.15.164:6801/984716690","cluster_addr":"172.21.15.164:6803/984716690","heartbeat_back_addr":"172.21.15.164:6807/984716690","heartbeat_front_addr":"172.21.15.164:6805/984716690","state":["exists","up"]},{"osd":1,"uuid":"f9001b06-be4d-4f42-a0e6-bcef2afa0d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":1349657239},{"type":"v1","addr":"172.21.15.164:6817","nonce":1349657239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":1349657239},{"type":"v1","addr":"172.21.15.164:6819","nonce":1349657239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":1349657239},{"type":"v1","addr":"172.21.15.164:6823","nonce":1349657239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":1349657239},{"type":"v1","addr":"172.21.15.164:6821","nonce":1349657239}]},"public_addr":"172.21.15.164:6817/1349657239","cluster_addr":"172.21.15.164:6819/1349657239","heartbeat_back_addr":"172.21.15.164:6823/1349657239","heartbeat_front_addr":"172.21.15.164:6821/1349657239","state":["exists","up"]},{"osd":2,"uuid":"fa0aeb9f-8b54-4831-a188-3e361c7980d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":887861252},{"type":"v1","addr":"172.21.15.164:6809","nonce":887861252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":887861252},{"type":"v1","addr":"172.21.15.164:6811","nonce":887861252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":887861252},{"type":"v1","addr":"172.21.15.164:6815","nonce":887861252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":887861252},{"type":"v1","addr":"172.21.15.164:6813","nonce":887861252}]},"public_addr":"172.21.15.164:6809/887861252","cluster_addr":"172.21.15.164:6811/887861252","heartbeat_back_addr":"172.21.15.164:6815/887861252","heartbeat_front_addr":"172.21.15.164:6813/887861252","state":["exists","up"]},{"osd":3,"uuid":"7e18c957-3091-46d5-8e17-8f3354c1726c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":812540271},{"type":"v1","addr":"172.21.15.164:6825","nonce":812540271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":812540271},{"type":"v1","addr":"172.21.15.164:6827","nonce":812540271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":812540271},{"type":"v1","addr":"172.21.15.164:6831","nonce":812540271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":812540271},{"type":"v1","addr":"172.21.15.164:6829","nonce":812540271}]},"public_addr":"172.21.15.164:6825/812540271","cluster_addr":"172.21.15.164:6827/812540271","heartbeat_back_addr":"172.21.15.164:6831/812540271","heartbeat_front_addr":"172.21.15.164:6829/812540271","state":["exists","up"]},{"osd":4,"uuid":"8e83f315-42bf-4471-9ce8-d81948da0f9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":2656857421},{"type":"v1","addr":"172.21.15.173:6825","nonce":2656857421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2656857421},{"type":"v1","addr":"172.21.15.173:6827","nonce":2656857421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2656857421},{"type":"v1","addr":"172.21.15.173:6831","nonce":2656857421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":2656857421},{"type":"v1","addr":"172.21.15.173:6829","nonce":2656857421}]},"public_addr":"172.21.15.173:6825/2656857421","cluster_addr":"172.21.15.173:6827/2656857421","heartbeat_back_addr":"172.21.15.173:6831/2656857421","heartbeat_front_addr":"172.21.15.173:6829/2656857421","state":["exists","up"]},{"osd":5,"uuid":"a230c872-71f9-40ab-8999-defd0162d4ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":3676483185},{"type":"v1","addr":"172.21.15.173:6801","nonce":3676483185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":3676483185},{"type":"v1","addr":"172.21.15.173:6803","nonce":3676483185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":3676483185},{"type":"v1","addr":"172.21.15.173:6807","nonce":3676483185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":3676483185},{"type":"v1","addr":"172.21.15.173:6805","nonce":3676483185}]},"public_addr":"172.21.15.173:6801/3676483185","cluster_addr":"172.21.15.173:6803/3676483185","heartbeat_back_addr":"172.21.15.173:6807/3676483185","heartbeat_front_addr":"172.21.15.173:6805/3676483185","state":["exists","up"]},{"osd":6,"uuid":"6193bfd7-8432-4f7f-a117-a3fcee21af99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":3631684087},{"type":"v1","addr":"172.21.15.173:6809","nonce":3631684087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":3631684087},{"type":"v1","addr":"172.21.15.173:6811","nonce":3631684087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":3631684087},{"type":"v1","addr":"172.21.15.173:6815","nonce":3631684087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":3631684087},{"type":"v1","addr":"172.21.15.173:6813","nonce":3631684087}]},"public_addr":"172.21.15.173:6809/3631684087","cluster_addr":"172.21.15.173:6811/3631684087","heartbeat_back_addr":"172.21.15.173:6815/3631684087","heartbeat_front_addr":"172.21.15.173:6813/3631684087","state":["exists","up"]},{"osd":7,"uuid":"b0f1b80a-afc4-4538-83b9-f081d2dcb8cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":3261114948},{"type":"v1","addr":"172.21.15.173:6817","nonce":3261114948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":3261114948},{"type":"v1","addr":"172.21.15.173:6819","nonce":3261114948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":3261114948},{"type":"v1","addr":"172.21.15.173:6823","nonce":3261114948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":3261114948},{"type":"v1","addr":"172.21.15.173:6821","nonce":3261114948}]},"public_addr":"172.21.15.173:6817/3261114948","cluster_addr":"172.21.15.173:6819/3261114948","heartbeat_back_addr":"172.21.15.173:6823/3261114948","heartbeat_front_addr":"172.21.15.173:6821/3261114948","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.188390+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.159932+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.191101+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.181776+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.224365+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.177515+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.153631+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.139972+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T14:56:59.958 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:59.954+0000 7f4cff8f9640 1 -- 172.21.15.164:0/3911468423 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f4cdc0672d0 msgr2=0x7f4cdc069790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:56:59.958 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:59.954+0000 7f4cff8f9640 1 --2- 172.21.15.164:0/3911468423 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f4cdc0672d0 0x7f4cdc069790 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4cf8146ba0 tx=0x7f4cf4009040 comp rx=0 tx=0).stop 2024-03-23T14:56:59.958 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:59.954+0000 7f4cff8f9640 1 -- 172.21.15.164:0/3911468423 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4cf814b380 msgr2=0x7f4cf81e08f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:56:59.958 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:59.954+0000 7f4cff8f9640 1 --2- 172.21.15.164:0/3911468423 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4cf814b380 0x7f4cf81e08f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4ce800eb50 tx=0x7f4ce800c4b0 comp rx=0 tx=0).stop 2024-03-23T14:56:59.958 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:59.954+0000 7f4cff8f9640 1 -- 172.21.15.164:0/3911468423 shutdown_connections 2024-03-23T14:56:59.958 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:59.954+0000 7f4cff8f9640 1 --2- 172.21.15.164:0/3911468423 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f4cdc0672d0 0x7f4cdc069790 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:59.958 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:59.954+0000 7f4cff8f9640 1 --2- 172.21.15.164:0/3911468423 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4cf8149620 0x7f4cf81dc8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:59.958 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:59.954+0000 7f4cff8f9640 1 --2- 172.21.15.164:0/3911468423 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4cf8148c60 0x7f4cf81490e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:59.958 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:59.954+0000 7f4cff8f9640 1 --2- 172.21.15.164:0/3911468423 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4cf814b380 0x7f4cf81e08f0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:56:59.958 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:59.954+0000 7f4cff8f9640 1 -- 172.21.15.164:0/3911468423 >> 172.21.15.164:0/3911468423 conn(0x7f4cf802c3c0 msgr2=0x7f4cf8140060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:56:59.958 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:59.954+0000 7f4cff8f9640 1 -- 172.21.15.164:0/3911468423 shutdown_connections 2024-03-23T14:56:59.959 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:56:59.954+0000 7f4cff8f9640 1 -- 172.21.15.164:0/3911468423 wait complete. 2024-03-23T14:56:59.974 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T14:57:00.092 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.090+0000 7fb36a79e640 1 Processor -- start 2024-03-23T14:57:00.092 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.090+0000 7fb36a79e640 1 -- start start 2024-03-23T14:57:00.093 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.090+0000 7fb36a79e640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fb3640add50 0x7fb364147770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:00.093 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.090+0000 7fb36a79e640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fb364147cb0 0x7fb364148090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:00.093 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.090+0000 7fb36a79e640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb36406d650 con 0x7fb36409def0 2024-03-23T14:57:00.093 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.090+0000 7fb36a79e640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fb3641485d0 con 0x7fb3640add50 2024-03-23T14:57:00.094 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.090+0000 7fb36a79e640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 -- 0x7fb3640af4a0 con 0x7fb364147cb0 2024-03-23T14:57:00.094 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.090+0000 7fb3637fe640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fb3640add50 0x7fb364147770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:00.094 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.090+0000 7fb3637fe640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fb3640add50 0x7fb364147770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:57514/0 (socket says 172.21.15.164:57514) 2024-03-23T14:57:00.094 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.090+0000 7fb3637fe640 1 -- 172.21.15.164:0/750857169 learned_addr learned my addr 172.21.15.164:0/750857169 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:00.094 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.090+0000 7fb368d3b640 1 --2- 172.21.15.164:0/750857169 >> v2:172.21.15.173:3300/0 conn(0x7fb364147cb0 0x7fb364148090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:00.094 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.090+0000 7fb3637fe640 1 -- 172.21.15.164:0/750857169 >> v2:172.21.15.173:3300/0 conn(0x7fb364147cb0 msgr2=0x7fb364148090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:00.094 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.090+0000 7fb3637fe640 1 --2- 172.21.15.164:0/750857169 >> v2:172.21.15.173:3300/0 conn(0x7fb364147cb0 0x7fb364148090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:00.094 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.090+0000 7fb3637fe640 1 -- 172.21.15.164:0/750857169 >> v1:172.21.15.173:6789/0 conn(0x7fb36409def0 legacy=0x7fb3640b45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:00.094 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.090+0000 7fb3637fe640 1 -- 172.21.15.164:0/750857169 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb36409e2d0 con 0x7fb3640add50 2024-03-23T14:57:00.095 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.090+0000 7fb3637fe640 1 --2- 172.21.15.164:0/750857169 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb3640add50 0x7fb364147770 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb3640b54c0 tx=0x7fb35401ead0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=854e22f004793da8 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:00.096 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.090+0000 7fb362ffd640 1 -- 172.21.15.164:0/750857169 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3540a7aa0 con 0x7fb3640add50 2024-03-23T14:57:00.096 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.090+0000 7fb362ffd640 1 -- 172.21.15.164:0/750857169 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3540a7c00 con 0x7fb3640add50 2024-03-23T14:57:00.096 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.090+0000 7fb362ffd640 1 -- 172.21.15.164:0/750857169 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3540af440 con 0x7fb3640add50 2024-03-23T14:57:00.097 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.094+0000 7fb36a79e640 1 -- 172.21.15.164:0/750857169 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb3640add50 msgr2=0x7fb364147770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:00.097 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.094+0000 7fb36a79e640 1 --2- 172.21.15.164:0/750857169 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb3640add50 0x7fb364147770 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb3640b54c0 tx=0x7fb35401ead0 comp rx=0 tx=0).stop 2024-03-23T14:57:00.097 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.094+0000 7fb36a79e640 1 -- 172.21.15.164:0/750857169 shutdown_connections 2024-03-23T14:57:00.097 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.094+0000 7fb36a79e640 1 --2- 172.21.15.164:0/750857169 >> v2:172.21.15.173:3300/0 conn(0x7fb364147cb0 0x7fb364148090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:00.097 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.094+0000 7fb36a79e640 1 --2- 172.21.15.164:0/750857169 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb3640add50 0x7fb364147770 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:00.097 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.094+0000 7fb36a79e640 1 -- 172.21.15.164:0/750857169 >> 172.21.15.164:0/750857169 conn(0x7fb3640b8ca0 msgr2=0x7fb3640b90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:00.097 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.094+0000 7fb36a79e640 1 -- 172.21.15.164:0/750857169 shutdown_connections 2024-03-23T14:57:00.097 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.094+0000 7fb36a79e640 1 -- 172.21.15.164:0/750857169 wait complete. 2024-03-23T14:57:00.099 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.094+0000 7fb36a79e640 1 Processor -- start 2024-03-23T14:57:00.099 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.094+0000 7fb36a79e640 1 -- start start 2024-03-23T14:57:00.099 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.094+0000 7fb36a79e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb36409def0 0x7fb364071be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:00.099 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.094+0000 7fb36a79e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb3640add50 0x7fb36406fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:00.100 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.094+0000 7fb36a79e640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb364147cb0 0x7fb36406ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:00.100 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.094+0000 7fb36a79e640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb36406f570 con 0x7fb3640add50 2024-03-23T14:57:00.100 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.094+0000 7fb36a79e640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fb364140870 con 0x7fb36409def0 2024-03-23T14:57:00.100 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.094+0000 7fb36a79e640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fb36413f850 con 0x7fb364147cb0 2024-03-23T14:57:00.100 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.094+0000 7fb3637fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb3640add50 0x7fb36406fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:00.100 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.094+0000 7fb3637fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb3640add50 0x7fb36406fa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:57524/0 (socket says 172.21.15.164:57524) 2024-03-23T14:57:00.100 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.094+0000 7fb3637fe640 1 -- 172.21.15.164:0/3055115132 learned_addr learned my addr 172.21.15.164:0/3055115132 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:00.100 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.094+0000 7fb363fff640 1 --2- 172.21.15.164:0/3055115132 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb36409def0 0x7fb364071be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:00.100 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.094+0000 7fb368d3b640 1 --2- 172.21.15.164:0/3055115132 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb364147cb0 0x7fb36406ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:00.100 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.094+0000 7fb3637fe640 1 -- 172.21.15.164:0/3055115132 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb364147cb0 msgr2=0x7fb36406ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:00.100 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.094+0000 7fb3637fe640 1 --2- 172.21.15.164:0/3055115132 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb364147cb0 0x7fb36406ff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:00.100 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.094+0000 7fb3637fe640 1 -- 172.21.15.164:0/3055115132 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb36409def0 msgr2=0x7fb364071be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:00.100 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.094+0000 7fb3637fe640 1 --2- 172.21.15.164:0/3055115132 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb36409def0 0x7fb364071be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:00.100 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.094+0000 7fb3637fe640 1 -- 172.21.15.164:0/3055115132 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb35400b770 con 0x7fb3640add50 2024-03-23T14:57:00.101 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.094+0000 7fb3637fe640 1 --2- 172.21.15.164:0/3055115132 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb3640add50 0x7fb36406fa00 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb354000c10 tx=0x7fb35401b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:00.101 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.094+0000 7fb360ff9640 1 -- 172.21.15.164:0/3055115132 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb354006ee0 con 0x7fb3640add50 2024-03-23T14:57:00.101 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.094+0000 7fb360ff9640 1 -- 172.21.15.164:0/3055115132 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3540ba640 con 0x7fb3640add50 2024-03-23T14:57:00.101 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.094+0000 7fb36a79e640 1 -- 172.21.15.164:0/3055115132 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb364140b00 con 0x7fb3640add50 2024-03-23T14:57:00.101 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.094+0000 7fb360ff9640 1 -- 172.21.15.164:0/3055115132 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3540ba9d0 con 0x7fb3640add50 2024-03-23T14:57:00.101 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.094+0000 7fb36a79e640 1 -- 172.21.15.164:0/3055115132 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb364070730 con 0x7fb3640add50 2024-03-23T14:57:00.102 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.098+0000 7fb36a79e640 1 -- 172.21.15.164:0/3055115132 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3640690a0 con 0x7fb3640add50 2024-03-23T14:57:00.102 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.098+0000 7fb360ff9640 1 -- 172.21.15.164:0/3055115132 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7fb3540b6040 con 0x7fb3640add50 2024-03-23T14:57:00.102 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.098+0000 7fb360ff9640 1 --2- 172.21.15.164:0/3055115132 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fb3500672a0 0x7fb350069760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:00.102 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.098+0000 7fb360ff9640 1 -- 172.21.15.164:0/3055115132 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5709+0+0 (secure 0 0 0) 0x7fb3541223f0 con 0x7fb3640add50 2024-03-23T14:57:00.102 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.098+0000 7fb363fff640 1 --2- 172.21.15.164:0/3055115132 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fb3500672a0 0x7fb350069760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:00.102 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.098+0000 7fb363fff640 1 --2- 172.21.15.164:0/3055115132 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fb3500672a0 0x7fb350069760 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb3580015b0 tx=0x7fb358009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:00.105 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.102+0000 7fb360ff9640 1 -- 172.21.15.164:0/3055115132 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7fb3540ef030 con 0x7fb3640add50 2024-03-23T14:57:00.264 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.262+0000 7fb36a79e640 1 -- 172.21.15.164:0/3055115132 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb3640722e0 con 0x7fb3640add50 2024-03-23T14:57:00.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.262+0000 7fb360ff9640 1 -- 172.21.15.164:0/3055115132 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) ==== 74+0+16642 (secure 0 0 0) 0x7fb3540f26f0 con 0x7fb3640add50 2024-03-23T14:57:00.266 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-23T14:57:00.266 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":19,"fsid":"45c7911b-b69d-4e68-907b-fbfeb780aa3b","created":"2024-03-23T14:56:34.367633+0000","modified":"2024-03-23T14:56:58.952999+0000","last_up_change":"2024-03-23T14:56:46.297403+0000","last_in_change":"2024-03-23T14:56:36.629884+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T14:56:47.186157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-23T14:56:56.994057+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-23T14:56:57.953862+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"80abdc62-bd0d-4e60-a554-7ed6fd67cf54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":984716690},{"type":"v1","addr":"172.21.15.164:6801","nonce":984716690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":984716690},{"type":"v1","addr":"172.21.15.164:6803","nonce":984716690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":984716690},{"type":"v1","addr":"172.21.15.164:6807","nonce":984716690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":984716690},{"type":"v1","addr":"172.21.15.164:6805","nonce":984716690}]},"public_addr":"172.21.15.164:6801/984716690","cluster_addr":"172.21.15.164:6803/984716690","heartbeat_back_addr":"172.21.15.164:6807/984716690","heartbeat_front_addr":"172.21.15.164:6805/984716690","state":["exists","up"]},{"osd":1,"uuid":"f9001b06-be4d-4f42-a0e6-bcef2afa0d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":1349657239},{"type":"v1","addr":"172.21.15.164:6817","nonce":1349657239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":1349657239},{"type":"v1","addr":"172.21.15.164:6819","nonce":1349657239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":1349657239},{"type":"v1","addr":"172.21.15.164:6823","nonce":1349657239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":1349657239},{"type":"v1","addr":"172.21.15.164:6821","nonce":1349657239}]},"public_addr":"172.21.15.164:6817/1349657239","cluster_addr":"172.21.15.164:6819/1349657239","heartbeat_back_addr":"172.21.15.164:6823/1349657239","heartbeat_front_addr":"172.21.15.164:6821/1349657239","state":["exists","up"]},{"osd":2,"uuid":"fa0aeb9f-8b54-4831-a188-3e361c7980d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":887861252},{"type":"v1","addr":"172.21.15.164:6809","nonce":887861252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":887861252},{"type":"v1","addr":"172.21.15.164:6811","nonce":887861252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":887861252},{"type":"v1","addr":"172.21.15.164:6815","nonce":887861252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":887861252},{"type":"v1","addr":"172.21.15.164:6813","nonce":887861252}]},"public_addr":"172.21.15.164:6809/887861252","cluster_addr":"172.21.15.164:6811/887861252","heartbeat_back_addr":"172.21.15.164:6815/887861252","heartbeat_front_addr":"172.21.15.164:6813/887861252","state":["exists","up"]},{"osd":3,"uuid":"7e18c957-3091-46d5-8e17-8f3354c1726c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":812540271},{"type":"v1","addr":"172.21.15.164:6825","nonce":812540271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":812540271},{"type":"v1","addr":"172.21.15.164:6827","nonce":812540271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":812540271},{"type":"v1","addr":"172.21.15.164:6831","nonce":812540271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":812540271},{"type":"v1","addr":"172.21.15.164:6829","nonce":812540271}]},"public_addr":"172.21.15.164:6825/812540271","cluster_addr":"172.21.15.164:6827/812540271","heartbeat_back_addr":"172.21.15.164:6831/812540271","heartbeat_front_addr":"172.21.15.164:6829/812540271","state":["exists","up"]},{"osd":4,"uuid":"8e83f315-42bf-4471-9ce8-d81948da0f9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":2656857421},{"type":"v1","addr":"172.21.15.173:6825","nonce":2656857421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2656857421},{"type":"v1","addr":"172.21.15.173:6827","nonce":2656857421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2656857421},{"type":"v1","addr":"172.21.15.173:6831","nonce":2656857421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":2656857421},{"type":"v1","addr":"172.21.15.173:6829","nonce":2656857421}]},"public_addr":"172.21.15.173:6825/2656857421","cluster_addr":"172.21.15.173:6827/2656857421","heartbeat_back_addr":"172.21.15.173:6831/2656857421","heartbeat_front_addr":"172.21.15.173:6829/2656857421","state":["exists","up"]},{"osd":5,"uuid":"a230c872-71f9-40ab-8999-defd0162d4ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":3676483185},{"type":"v1","addr":"172.21.15.173:6801","nonce":3676483185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":3676483185},{"type":"v1","addr":"172.21.15.173:6803","nonce":3676483185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":3676483185},{"type":"v1","addr":"172.21.15.173:6807","nonce":3676483185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":3676483185},{"type":"v1","addr":"172.21.15.173:6805","nonce":3676483185}]},"public_addr":"172.21.15.173:6801/3676483185","cluster_addr":"172.21.15.173:6803/3676483185","heartbeat_back_addr":"172.21.15.173:6807/3676483185","heartbeat_front_addr":"172.21.15.173:6805/3676483185","state":["exists","up"]},{"osd":6,"uuid":"6193bfd7-8432-4f7f-a117-a3fcee21af99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":3631684087},{"type":"v1","addr":"172.21.15.173:6809","nonce":3631684087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":3631684087},{"type":"v1","addr":"172.21.15.173:6811","nonce":3631684087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":3631684087},{"type":"v1","addr":"172.21.15.173:6815","nonce":3631684087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":3631684087},{"type":"v1","addr":"172.21.15.173:6813","nonce":3631684087}]},"public_addr":"172.21.15.173:6809/3631684087","cluster_addr":"172.21.15.173:6811/3631684087","heartbeat_back_addr":"172.21.15.173:6815/3631684087","heartbeat_front_addr":"172.21.15.173:6813/3631684087","state":["exists","up"]},{"osd":7,"uuid":"b0f1b80a-afc4-4538-83b9-f081d2dcb8cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":3261114948},{"type":"v1","addr":"172.21.15.173:6817","nonce":3261114948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":3261114948},{"type":"v1","addr":"172.21.15.173:6819","nonce":3261114948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":3261114948},{"type":"v1","addr":"172.21.15.173:6823","nonce":3261114948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":3261114948},{"type":"v1","addr":"172.21.15.173:6821","nonce":3261114948}]},"public_addr":"172.21.15.173:6817/3261114948","cluster_addr":"172.21.15.173:6819/3261114948","heartbeat_back_addr":"172.21.15.173:6823/3261114948","heartbeat_front_addr":"172.21.15.173:6821/3261114948","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.188390+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.159932+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.191101+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.181776+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.224365+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.177515+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.153631+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.139972+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T14:57:00.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.262+0000 7fb36a79e640 1 -- 172.21.15.164:0/3055115132 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fb3500672a0 msgr2=0x7fb350069760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:00.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.262+0000 7fb36a79e640 1 --2- 172.21.15.164:0/3055115132 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fb3500672a0 0x7fb350069760 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb3580015b0 tx=0x7fb358009040 comp rx=0 tx=0).stop 2024-03-23T14:57:00.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.262+0000 7fb36a79e640 1 -- 172.21.15.164:0/3055115132 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb3640add50 msgr2=0x7fb36406fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:00.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.262+0000 7fb36a79e640 1 --2- 172.21.15.164:0/3055115132 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb3640add50 0x7fb36406fa00 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb354000c10 tx=0x7fb35401b870 comp rx=0 tx=0).stop 2024-03-23T14:57:00.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.262+0000 7fb36a79e640 1 -- 172.21.15.164:0/3055115132 shutdown_connections 2024-03-23T14:57:00.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.262+0000 7fb36a79e640 1 --2- 172.21.15.164:0/3055115132 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb364147cb0 0x7fb36406ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:00.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.262+0000 7fb36a79e640 1 --2- 172.21.15.164:0/3055115132 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb3640add50 0x7fb36406fa00 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:00.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.262+0000 7fb36a79e640 1 --2- 172.21.15.164:0/3055115132 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fb3500672a0 0x7fb350069760 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:00.269 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.262+0000 7fb36a79e640 1 --2- 172.21.15.164:0/3055115132 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb36409def0 0x7fb364071be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:00.269 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.266+0000 7fb36a79e640 1 -- 172.21.15.164:0/3055115132 >> 172.21.15.164:0/3055115132 conn(0x7fb3640b8ca0 msgr2=0x7fb3641454a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:00.269 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.266+0000 7fb36a79e640 1 -- 172.21.15.164:0/3055115132 shutdown_connections 2024-03-23T14:57:00.269 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.266+0000 7fb36a79e640 1 -- 172.21.15.164:0/3055115132 wait complete. 2024-03-23T14:57:00.285 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-23T14:57:00.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.402+0000 7f4af2232640 1 Processor -- start 2024-03-23T14:57:00.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.402+0000 7f4af2232640 1 -- start start 2024-03-23T14:57:00.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.406+0000 7f4af2232640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f4aec13f010 0x7f4aec13f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:00.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.406+0000 7f4af2232640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4aec148860 0x7f4aec148cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:00.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.406+0000 7f4af2232640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4aec06b7b0 con 0x7f4aec0b58a0 2024-03-23T14:57:00.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.406+0000 7f4af2232640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f4aec13f9b0 con 0x7f4aec13f010 2024-03-23T14:57:00.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.406+0000 7f4af2232640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f4aec149210 con 0x7f4aec148860 2024-03-23T14:57:00.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.406+0000 7f4aeb7fe640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f4aec0b58a0 0x7f4aec0b44e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:45450/0 (socket says 172.21.15.164:45450) 2024-03-23T14:57:00.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.406+0000 7f4aeb7fe640 1 -- 172.21.15.164:0/2369076986 learned_addr learned my addr 172.21.15.164:0/2369076986 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-03-23T14:57:00.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.406+0000 7f4aeaffd640 1 --2- 172.21.15.164:0/2369076986 >> v2:172.21.15.164:3300/0 conn(0x7f4aec13f010 0x7f4aec13f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:00.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.406+0000 7f4aebfff640 1 --2- 172.21.15.164:0/2369076986 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4aec148860 0x7f4aec148cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:00.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.406+0000 7f4aeaffd640 1 -- 172.21.15.164:0/2369076986 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4aec148860 msgr2=0x7f4aec148cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:00.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.406+0000 7f4aeaffd640 1 --2- 172.21.15.164:0/2369076986 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4aec148860 0x7f4aec148cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:00.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.406+0000 7f4aeaffd640 1 -- 172.21.15.164:0/2369076986 >> v1:172.21.15.164:6789/0 conn(0x7f4aec0b58a0 legacy=0x7f4aec0b44e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:00.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.406+0000 7f4aeaffd640 1 -- 172.21.15.164:0/2369076986 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4aec06b1a0 con 0x7f4aec13f010 2024-03-23T14:57:00.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.406+0000 7f4aeaffd640 1 --2- 172.21.15.164:0/2369076986 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4aec13f010 0x7f4aec13f470 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4ae00097c0 tx=0x7f4ae001c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbca127a4fe126ca server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:00.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.406+0000 7f4aea7fc640 1 -- 172.21.15.164:0/2369076986 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ae00ab600 con 0x7f4aec13f010 2024-03-23T14:57:00.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.406+0000 7f4aea7fc640 1 -- 172.21.15.164:0/2369076986 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ae00a7050 con 0x7f4aec13f010 2024-03-23T14:57:00.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.406+0000 7f4aea7fc640 1 -- 172.21.15.164:0/2369076986 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ae00ab980 con 0x7f4aec13f010 2024-03-23T14:57:00.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.406+0000 7f4af2232640 1 -- 172.21.15.164:0/2369076986 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4aec13f010 msgr2=0x7f4aec13f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:00.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.406+0000 7f4af2232640 1 --2- 172.21.15.164:0/2369076986 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4aec13f010 0x7f4aec13f470 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4ae00097c0 tx=0x7f4ae001c3e0 comp rx=0 tx=0).stop 2024-03-23T14:57:00.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.406+0000 7f4af2232640 1 -- 172.21.15.164:0/2369076986 shutdown_connections 2024-03-23T14:57:00.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.406+0000 7f4af2232640 1 --2- 172.21.15.164:0/2369076986 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4aec148860 0x7f4aec148cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:00.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.406+0000 7f4af2232640 1 --2- 172.21.15.164:0/2369076986 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4aec13f010 0x7f4aec13f470 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:00.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.406+0000 7f4af2232640 1 -- 172.21.15.164:0/2369076986 >> 172.21.15.164:0/2369076986 conn(0x7f4aec0b9ae0 msgr2=0x7f4aec0b9ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:00.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.406+0000 7f4af2232640 1 -- 172.21.15.164:0/2369076986 shutdown_connections 2024-03-23T14:57:00.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.406+0000 7f4af2232640 1 -- 172.21.15.164:0/2369076986 wait complete. 2024-03-23T14:57:00.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.410+0000 7f4af2232640 1 Processor -- start 2024-03-23T14:57:00.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.410+0000 7f4af2232640 1 -- start start 2024-03-23T14:57:00.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.410+0000 7f4af2232640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4aec0b58a0 0x7f4aec138c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:00.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.410+0000 7f4af2232640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4aec13f010 0x7f4aec1391c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:00.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.410+0000 7f4af2232640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4aec148860 0x7f4aec1e0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:00.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.410+0000 7f4af2232640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f4aec0b8470 con 0x7f4aec0b58a0 2024-03-23T14:57:00.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.410+0000 7f4af2232640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f4aec0b92f0 con 0x7f4aec13f010 2024-03-23T14:57:00.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.410+0000 7f4af2232640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f4aec0b1dc0 con 0x7f4aec148860 2024-03-23T14:57:00.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.410+0000 7f4aeb7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4aec0b58a0 0x7f4aec138c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:00.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.410+0000 7f4aeb7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4aec0b58a0 0x7f4aec138c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:57546/0 (socket says 172.21.15.164:57546) 2024-03-23T14:57:00.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.410+0000 7f4aeb7fe640 1 -- 172.21.15.164:0/1670175968 learned_addr learned my addr 172.21.15.164:0/1670175968 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:00.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.410+0000 7f4aebfff640 1 --2- 172.21.15.164:0/1670175968 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4aec148860 0x7f4aec1e0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:00.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.410+0000 7f4aeaffd640 1 --2- 172.21.15.164:0/1670175968 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4aec13f010 0x7f4aec1391c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:00.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.410+0000 7f4aeb7fe640 1 -- 172.21.15.164:0/1670175968 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4aec148860 msgr2=0x7f4aec1e0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:00.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.410+0000 7f4aeb7fe640 1 --2- 172.21.15.164:0/1670175968 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4aec148860 0x7f4aec1e0dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:00.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.410+0000 7f4aeb7fe640 1 -- 172.21.15.164:0/1670175968 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4aec13f010 msgr2=0x7f4aec1391c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:00.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.410+0000 7f4aeb7fe640 1 --2- 172.21.15.164:0/1670175968 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4aec13f010 0x7f4aec1391c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:00.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.410+0000 7f4aeb7fe640 1 -- 172.21.15.164:0/1670175968 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ae0009420 con 0x7f4aec0b58a0 2024-03-23T14:57:00.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.410+0000 7f4aebfff640 1 --2- 172.21.15.164:0/1670175968 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4aec148860 0x7f4aec1e0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T14:57:00.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.410+0000 7f4aeb7fe640 1 --2- 172.21.15.164:0/1670175968 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4aec0b58a0 0x7f4aec138c80 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4adc00c480 tx=0x7f4adc00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:00.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.410+0000 7f4ad3fff640 1 -- 172.21.15.164:0/1670175968 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4adc005cf0 con 0x7f4aec0b58a0 2024-03-23T14:57:00.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.410+0000 7f4ad3fff640 1 -- 172.21.15.164:0/1670175968 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4adc005e50 con 0x7f4aec0b58a0 2024-03-23T14:57:00.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.410+0000 7f4af2232640 1 -- 172.21.15.164:0/1670175968 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4aec0b2030 con 0x7f4aec0b58a0 2024-03-23T14:57:00.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.410+0000 7f4ad3fff640 1 -- 172.21.15.164:0/1670175968 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4adc007290 con 0x7f4aec0b58a0 2024-03-23T14:57:00.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.410+0000 7f4af2232640 1 -- 172.21.15.164:0/1670175968 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4aec0b86c0 con 0x7f4aec0b58a0 2024-03-23T14:57:00.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.410+0000 7f4af2232640 1 -- 172.21.15.164:0/1670175968 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4aec02d160 con 0x7f4aec0b58a0 2024-03-23T14:57:00.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.410+0000 7f4ad3fff640 1 -- 172.21.15.164:0/1670175968 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7f4adc0073f0 con 0x7f4aec0b58a0 2024-03-23T14:57:00.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.414+0000 7f4ad3fff640 1 --2- 172.21.15.164:0/1670175968 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f4ad80672a0 0x7f4ad8069760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:00.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.414+0000 7f4ad3fff640 1 -- 172.21.15.164:0/1670175968 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5709+0+0 (secure 0 0 0) 0x7f4adc084520 con 0x7f4aec0b58a0 2024-03-23T14:57:00.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.414+0000 7f4aeaffd640 1 --2- 172.21.15.164:0/1670175968 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f4ad80672a0 0x7f4ad8069760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:00.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.414+0000 7f4aeaffd640 1 --2- 172.21.15.164:0/1670175968 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f4ad80672a0 0x7f4ad8069760 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f4aec06cee0 tx=0x7f4ae0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:00.420 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.418+0000 7f4ad3fff640 1 -- 172.21.15.164:0/1670175968 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7f4adc051160 con 0x7f4aec0b58a0 2024-03-23T14:57:00.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.598+0000 7f4af2232640 1 -- 172.21.15.164:0/1670175968 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f4aec069080 con 0x7f4aec0b58a0 2024-03-23T14:57:00.654 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.650+0000 7f4ad3fff640 1 -- 172.21.15.164:0/1670175968 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v5) ==== 118+0+0 (secure 0 0 0) 0x7f4adc054820 con 0x7f4aec0b58a0 2024-03-23T14:57:00.657 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.654+0000 7f4af2232640 1 -- 172.21.15.164:0/1670175968 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f4ad80672a0 msgr2=0x7f4ad8069760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:00.657 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.654+0000 7f4af2232640 1 --2- 172.21.15.164:0/1670175968 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f4ad80672a0 0x7f4ad8069760 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f4aec06cee0 tx=0x7f4ae0008040 comp rx=0 tx=0).stop 2024-03-23T14:57:00.658 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.654+0000 7f4af2232640 1 -- 172.21.15.164:0/1670175968 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4aec0b58a0 msgr2=0x7f4aec138c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:00.658 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.654+0000 7f4af2232640 1 --2- 172.21.15.164:0/1670175968 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4aec0b58a0 0x7f4aec138c80 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4adc00c480 tx=0x7f4adc00c940 comp rx=0 tx=0).stop 2024-03-23T14:57:00.658 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.654+0000 7f4af2232640 1 -- 172.21.15.164:0/1670175968 shutdown_connections 2024-03-23T14:57:00.658 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.654+0000 7f4af2232640 1 --2- 172.21.15.164:0/1670175968 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4aec148860 0x7f4aec1e0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:00.658 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.654+0000 7f4af2232640 1 --2- 172.21.15.164:0/1670175968 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f4ad80672a0 0x7f4ad8069760 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:00.658 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.654+0000 7f4af2232640 1 --2- 172.21.15.164:0/1670175968 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4aec13f010 0x7f4aec1391c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:00.658 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.654+0000 7f4af2232640 1 --2- 172.21.15.164:0/1670175968 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4aec0b58a0 0x7f4aec138c80 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:00.658 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.654+0000 7f4af2232640 1 -- 172.21.15.164:0/1670175968 >> 172.21.15.164:0/1670175968 conn(0x7f4aec0b9ae0 msgr2=0x7f4aec06cfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:00.658 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.654+0000 7f4af2232640 1 -- 172.21.15.164:0/1670175968 shutdown_connections 2024-03-23T14:57:00.658 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.654+0000 7f4af2232640 1 -- 172.21.15.164:0/1670175968 wait complete. 2024-03-23T14:57:00.673 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-03-23T14:57:00.673 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-03-23T14:57:00.792 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.790+0000 7fcd7aef5640 1 Processor -- start 2024-03-23T14:57:00.792 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.790+0000 7fcd7aef5640 1 -- start start 2024-03-23T14:57:00.793 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.790+0000 7fcd7aef5640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fcd740b0230 0x7fcd740b34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:00.794 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.790+0000 7fcd7aef5640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd740b51d0 con 0x7fcd74148cf0 2024-03-23T14:57:00.794 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.790+0000 7fcd7aef5640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd740b0610 con 0x7fcd7414db70 2024-03-23T14:57:00.794 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.790+0000 7fcd7aef5640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fcd740b1500 con 0x7fcd740b0230 2024-03-23T14:57:00.794 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.790+0000 7fcd78c91640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fcd740b0230 0x7fcd740b34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:00.794 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.790+0000 7fcd73fff640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fcd74148cf0 0x7fcd74149180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:45452/0 (socket says 172.21.15.164:45452) 2024-03-23T14:57:00.794 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.790+0000 7fcd73fff640 1 -- 172.21.15.164:0/360627261 learned_addr learned my addr 172.21.15.164:0/360627261 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-03-23T14:57:00.794 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.790+0000 7fcd737fe640 1 -- 172.21.15.164:0/360627261 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1301898000 0 0) 0x7fcd740b51d0 con 0x7fcd74148cf0 2024-03-23T14:57:00.794 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.790+0000 7fcd737fe640 1 -- 172.21.15.164:0/360627261 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcd64003530 con 0x7fcd74148cf0 2024-03-23T14:57:00.794 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.790+0000 7fcd78c91640 1 -- 172.21.15.164:0/360627261 >> v1:172.21.15.173:6789/0 conn(0x7fcd7414db70 legacy=0x7fcd7414ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:00.794 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.790+0000 7fcd78c91640 1 -- 172.21.15.164:0/360627261 >> v1:172.21.15.164:6789/0 conn(0x7fcd74148cf0 legacy=0x7fcd74149180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:00.794 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.790+0000 7fcd78c91640 1 -- 172.21.15.164:0/360627261 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd7406d250 con 0x7fcd740b0230 2024-03-23T14:57:00.796 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.790+0000 7fcd78c91640 1 --2- 172.21.15.164:0/360627261 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcd740b0230 0x7fcd740b34d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fcd680097c0 tx=0x7fcd6801c350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a3cf84a08ae626d server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:00.796 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.790+0000 7fcd737fe640 1 -- 172.21.15.164:0/360627261 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd68004230 con 0x7fcd740b0230 2024-03-23T14:57:00.796 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.794+0000 7fcd737fe640 1 -- 172.21.15.164:0/360627261 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd680a7050 con 0x7fcd740b0230 2024-03-23T14:57:00.796 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.794+0000 7fcd737fe640 1 -- 172.21.15.164:0/360627261 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd68003620 con 0x7fcd740b0230 2024-03-23T14:57:00.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.794+0000 7fcd7aef5640 1 -- 172.21.15.164:0/360627261 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcd740b0230 msgr2=0x7fcd740b34d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:00.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.794+0000 7fcd7aef5640 1 --2- 172.21.15.164:0/360627261 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcd740b0230 0x7fcd740b34d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fcd680097c0 tx=0x7fcd6801c350 comp rx=0 tx=0).stop 2024-03-23T14:57:00.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.794+0000 7fcd7aef5640 1 -- 172.21.15.164:0/360627261 shutdown_connections 2024-03-23T14:57:00.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.794+0000 7fcd7aef5640 1 --2- 172.21.15.164:0/360627261 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcd740b0230 0x7fcd740b34d0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:00.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.794+0000 7fcd7aef5640 1 -- 172.21.15.164:0/360627261 >> 172.21.15.164:0/360627261 conn(0x7fcd740b8ca0 msgr2=0x7fcd740b90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:00.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.794+0000 7fcd7aef5640 1 -- 172.21.15.164:0/360627261 shutdown_connections 2024-03-23T14:57:00.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.794+0000 7fcd7aef5640 1 -- 172.21.15.164:0/360627261 wait complete. 2024-03-23T14:57:00.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.794+0000 7fcd7aef5640 1 Processor -- start 2024-03-23T14:57:00.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.794+0000 7fcd7aef5640 1 -- start start 2024-03-23T14:57:00.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.794+0000 7fcd7aef5640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcd740b0230 0x7fcd7409c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:00.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.794+0000 7fcd7aef5640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcd74148cf0 0x7fcd7414f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:00.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.794+0000 7fcd7aef5640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcd7414db70 0x7fcd7414fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:00.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.794+0000 7fcd7aef5640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fcd7414f2b0 con 0x7fcd740b0230 2024-03-23T14:57:00.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.794+0000 7fcd7aef5640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fcd7414e1a0 con 0x7fcd74148cf0 2024-03-23T14:57:00.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.794+0000 7fcd7aef5640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fcd7414e480 con 0x7fcd7414db70 2024-03-23T14:57:00.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.794+0000 7fcd78c91640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcd740b0230 0x7fcd7409c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:00.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.794+0000 7fcd78c91640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcd740b0230 0x7fcd7409c6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:57568/0 (socket says 172.21.15.164:57568) 2024-03-23T14:57:00.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.794+0000 7fcd78c91640 1 -- 172.21.15.164:0/4165623156 learned_addr learned my addr 172.21.15.164:0/4165623156 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:00.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.794+0000 7fcd79492640 1 --2- 172.21.15.164:0/4165623156 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcd7414db70 0x7fcd7414fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:00.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.794+0000 7fcd73fff640 1 --2- 172.21.15.164:0/4165623156 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcd74148cf0 0x7fcd7414f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:00.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.794+0000 7fcd78c91640 1 -- 172.21.15.164:0/4165623156 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcd7414db70 msgr2=0x7fcd7414fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:00.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.794+0000 7fcd78c91640 1 --2- 172.21.15.164:0/4165623156 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcd7414db70 0x7fcd7414fc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:00.801 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.794+0000 7fcd78c91640 1 -- 172.21.15.164:0/4165623156 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcd74148cf0 msgr2=0x7fcd7414f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:00.801 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.794+0000 7fcd78c91640 1 --2- 172.21.15.164:0/4165623156 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcd74148cf0 0x7fcd7414f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:00.801 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.794+0000 7fcd78c91640 1 -- 172.21.15.164:0/4165623156 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd68009420 con 0x7fcd740b0230 2024-03-23T14:57:00.801 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.794+0000 7fcd78c91640 1 --2- 172.21.15.164:0/4165623156 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcd740b0230 0x7fcd7409c6b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fcd68018f60 tx=0x7fcd680190f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:00.801 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.794+0000 7fcd717fa640 1 -- 172.21.15.164:0/4165623156 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd68003bf0 con 0x7fcd740b0230 2024-03-23T14:57:00.801 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.794+0000 7fcd717fa640 1 -- 172.21.15.164:0/4165623156 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd68003d70 con 0x7fcd740b0230 2024-03-23T14:57:00.801 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.794+0000 7fcd7aef5640 1 -- 172.21.15.164:0/4165623156 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd7414e910 con 0x7fcd740b0230 2024-03-23T14:57:00.801 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.794+0000 7fcd717fa640 1 -- 172.21.15.164:0/4165623156 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd680ab680 con 0x7fcd740b0230 2024-03-23T14:57:00.801 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.794+0000 7fcd7aef5640 1 -- 172.21.15.164:0/4165623156 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd7414eed0 con 0x7fcd740b0230 2024-03-23T14:57:00.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.798+0000 7fcd7aef5640 1 -- 172.21.15.164:0/4165623156 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd7414f520 con 0x7fcd740b0230 2024-03-23T14:57:00.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.798+0000 7fcd717fa640 1 -- 172.21.15.164:0/4165623156 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7fcd680b2040 con 0x7fcd740b0230 2024-03-23T14:57:00.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.798+0000 7fcd717fa640 1 --2- 172.21.15.164:0/4165623156 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fcd580672a0 0x7fcd58069760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:00.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.798+0000 7fcd717fa640 1 -- 172.21.15.164:0/4165623156 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fcd6811f020 con 0x7fcd740b0230 2024-03-23T14:57:00.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.798+0000 7fcd73fff640 1 --2- 172.21.15.164:0/4165623156 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fcd580672a0 0x7fcd58069760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:00.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.798+0000 7fcd73fff640 1 --2- 172.21.15.164:0/4165623156 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fcd580672a0 0x7fcd58069760 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fcd74144c00 tx=0x7fcd60009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:00.805 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.802+0000 7fcd717fa640 1 -- 172.21.15.164:0/4165623156 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7fcd680ebc60 con 0x7fcd740b0230 2024-03-23T14:57:00.998 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:00.994+0000 7fcd7aef5640 1 -- 172.21.15.164:0/4165623156 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fcd740690a0 con 0x7fcd740b0230 2024-03-23T14:57:01.657 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:01.654+0000 7fcd717fa640 1 -- 172.21.15.164:0/4165623156 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v6) ==== 122+0+0 (secure 0 0 0) 0x7fcd680ef320 con 0x7fcd740b0230 2024-03-23T14:57:01.661 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:01.658+0000 7fcd7aef5640 1 -- 172.21.15.164:0/4165623156 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fcd580672a0 msgr2=0x7fcd58069760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:01.661 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:01.658+0000 7fcd7aef5640 1 --2- 172.21.15.164:0/4165623156 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fcd580672a0 0x7fcd58069760 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fcd74144c00 tx=0x7fcd60009040 comp rx=0 tx=0).stop 2024-03-23T14:57:01.661 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:01.658+0000 7fcd7aef5640 1 -- 172.21.15.164:0/4165623156 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcd740b0230 msgr2=0x7fcd7409c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:01.661 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:01.658+0000 7fcd7aef5640 1 --2- 172.21.15.164:0/4165623156 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcd740b0230 0x7fcd7409c6b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fcd68018f60 tx=0x7fcd680190f0 comp rx=0 tx=0).stop 2024-03-23T14:57:01.661 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:01.658+0000 7fcd7aef5640 1 -- 172.21.15.164:0/4165623156 shutdown_connections 2024-03-23T14:57:01.661 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:01.658+0000 7fcd7aef5640 1 --2- 172.21.15.164:0/4165623156 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcd7414db70 0x7fcd7414fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:01.661 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:01.658+0000 7fcd7aef5640 1 --2- 172.21.15.164:0/4165623156 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fcd580672a0 0x7fcd58069760 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:01.661 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:01.658+0000 7fcd7aef5640 1 --2- 172.21.15.164:0/4165623156 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcd74148cf0 0x7fcd7414f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:01.662 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:01.658+0000 7fcd7aef5640 1 --2- 172.21.15.164:0/4165623156 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcd740b0230 0x7fcd7409c6b0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:01.662 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:01.658+0000 7fcd7aef5640 1 -- 172.21.15.164:0/4165623156 >> 172.21.15.164:0/4165623156 conn(0x7fcd740b8ca0 msgr2=0x7fcd74147cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:01.662 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:01.658+0000 7fcd7aef5640 1 -- 172.21.15.164:0/4165623156 shutdown_connections 2024-03-23T14:57:01.662 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:01.658+0000 7fcd7aef5640 1 -- 172.21.15.164:0/4165623156 wait complete. 2024-03-23T14:57:01.682 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-03-23T14:57:01.682 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T14:57:01.804 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:01.802+0000 7fbc67a94640 1 Processor -- start 2024-03-23T14:57:01.804 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:01.802+0000 7fbc67a94640 1 -- start start 2024-03-23T14:57:01.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:01.802+0000 7fbc67a94640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fbc600725b0 0x7fbc60148890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:01.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:01.802+0000 7fbc67a94640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fbc600b3990 0x7fbc6014fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:01.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:01.802+0000 7fbc67a94640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc600b0ca0 con 0x7fbc6006e730 2024-03-23T14:57:01.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:01.802+0000 7fbc67a94640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fbc60072990 con 0x7fbc600725b0 2024-03-23T14:57:01.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:01.802+0000 7fbc67a94640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 -- 0x7fbc600b95f0 con 0x7fbc600b3990 2024-03-23T14:57:01.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:01.802+0000 7fbc65830640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fbc600725b0 0x7fbc60148890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:01.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:01.802+0000 7fbc65830640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fbc600725b0 0x7fbc60148890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:57570/0 (socket says 172.21.15.164:57570) 2024-03-23T14:57:01.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:01.802+0000 7fbc65830640 1 -- 172.21.15.164:0/3211600033 learned_addr learned my addr 172.21.15.164:0/3211600033 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:01.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:01.802+0000 7fbc66031640 1 --2- 172.21.15.164:0/3211600033 >> v2:172.21.15.173:3300/0 conn(0x7fbc600b3990 0x7fbc6014fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:01.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:01.802+0000 7fbc65830640 1 -- 172.21.15.164:0/3211600033 >> v2:172.21.15.173:3300/0 conn(0x7fbc600b3990 msgr2=0x7fbc6014fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:01.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:01.802+0000 7fbc65830640 1 --2- 172.21.15.164:0/3211600033 >> v2:172.21.15.173:3300/0 conn(0x7fbc600b3990 0x7fbc6014fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:01.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:01.802+0000 7fbc65830640 1 -- 172.21.15.164:0/3211600033 >> v1:172.21.15.173:6789/0 conn(0x7fbc6006e730 legacy=0x7fbc600b3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:01.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:01.802+0000 7fbc65830640 1 -- 172.21.15.164:0/3211600033 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc60148dd0 con 0x7fbc600725b0 2024-03-23T14:57:01.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:01.806+0000 7fbc65830640 1 --2- 172.21.15.164:0/3211600033 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbc600725b0 0x7fbc60148890 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fbc6006fab0 tx=0x7fbc5c01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2e7d325581304a3 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:01.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:01.806+0000 7fbc6482e640 1 -- 172.21.15.164:0/3211600033 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc5c0ac600 con 0x7fbc600725b0 2024-03-23T14:57:01.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:01.806+0000 7fbc6482e640 1 -- 172.21.15.164:0/3211600033 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc5c0a8050 con 0x7fbc600725b0 2024-03-23T14:57:01.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:01.806+0000 7fbc6482e640 1 -- 172.21.15.164:0/3211600033 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc5c0ac980 con 0x7fbc600725b0 2024-03-23T14:57:01.810 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:01.806+0000 7fbc67a94640 1 -- 172.21.15.164:0/3211600033 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbc600725b0 msgr2=0x7fbc60148890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:01.810 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:01.806+0000 7fbc67a94640 1 --2- 172.21.15.164:0/3211600033 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbc600725b0 0x7fbc60148890 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fbc6006fab0 tx=0x7fbc5c01c490 comp rx=0 tx=0).stop 2024-03-23T14:57:01.810 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:01.806+0000 7fbc67a94640 1 -- 172.21.15.164:0/3211600033 shutdown_connections 2024-03-23T14:57:01.810 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:01.806+0000 7fbc67a94640 1 --2- 172.21.15.164:0/3211600033 >> v2:172.21.15.173:3300/0 conn(0x7fbc600b3990 0x7fbc6014fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:01.810 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:01.806+0000 7fbc67a94640 1 --2- 172.21.15.164:0/3211600033 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbc600725b0 0x7fbc60148890 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:01.810 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:01.806+0000 7fbc67a94640 1 -- 172.21.15.164:0/3211600033 >> 172.21.15.164:0/3211600033 conn(0x7fbc600b7970 msgr2=0x7fbc600b7d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:01.810 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:01.806+0000 7fbc67a94640 1 -- 172.21.15.164:0/3211600033 shutdown_connections 2024-03-23T14:57:01.810 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:01.806+0000 7fbc67a94640 1 -- 172.21.15.164:0/3211600033 wait complete. 2024-03-23T14:57:01.810 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:01.806+0000 7fbc67a94640 1 Processor -- start 2024-03-23T14:57:01.811 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:01.806+0000 7fbc67a94640 1 -- start start 2024-03-23T14:57:01.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:01.806+0000 7fbc67a94640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbc6006e730 0x7fbc60148090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:01.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:01.806+0000 7fbc67a94640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbc600725b0 0x7fbc60147520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:01.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:01.806+0000 7fbc67a94640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbc600b3990 0x7fbc60136930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:01.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:01.806+0000 7fbc67a94640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fbc60146e60 con 0x7fbc600b3990 2024-03-23T14:57:01.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:01.806+0000 7fbc67a94640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fbc601485d0 con 0x7fbc6006e730 2024-03-23T14:57:01.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:01.806+0000 7fbc67a94640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fbc60147a60 con 0x7fbc600725b0 2024-03-23T14:57:01.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:01.806+0000 7fbc66031640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbc600b3990 0x7fbc60136930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:01.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:01.806+0000 7fbc66031640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbc600b3990 0x7fbc60136930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:57582/0 (socket says 172.21.15.164:57582) 2024-03-23T14:57:01.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:01.806+0000 7fbc66031640 1 -- 172.21.15.164:0/922834680 learned_addr learned my addr 172.21.15.164:0/922834680 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:01.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:01.806+0000 7fbc65830640 1 --2- 172.21.15.164:0/922834680 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbc6006e730 0x7fbc60148090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:01.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:01.806+0000 7fbc6502f640 1 --2- 172.21.15.164:0/922834680 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbc600725b0 0x7fbc60147520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:01.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:01.806+0000 7fbc66031640 1 -- 172.21.15.164:0/922834680 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbc600725b0 msgr2=0x7fbc60147520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:01.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:01.806+0000 7fbc66031640 1 --2- 172.21.15.164:0/922834680 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbc600725b0 0x7fbc60147520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:01.814 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:01.806+0000 7fbc66031640 1 -- 172.21.15.164:0/922834680 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbc6006e730 msgr2=0x7fbc60148090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:01.814 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:01.806+0000 7fbc66031640 1 --2- 172.21.15.164:0/922834680 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbc6006e730 0x7fbc60148090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:01.814 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:01.806+0000 7fbc66031640 1 -- 172.21.15.164:0/922834680 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc5c0094d0 con 0x7fbc600b3990 2024-03-23T14:57:01.814 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:01.810+0000 7fbc6502f640 1 --2- 172.21.15.164:0/922834680 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbc600725b0 0x7fbc60147520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T14:57:01.814 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:01.810+0000 7fbc66031640 1 --2- 172.21.15.164:0/922834680 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbc600b3990 0x7fbc60136930 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fbc58007e70 tx=0x7fbc5800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:01.814 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:01.810+0000 7fbc567fc640 1 -- 172.21.15.164:0/922834680 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc58014620 con 0x7fbc600b3990 2024-03-23T14:57:01.814 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:01.810+0000 7fbc567fc640 1 -- 172.21.15.164:0/922834680 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc580147a0 con 0x7fbc600b3990 2024-03-23T14:57:01.814 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:01.810+0000 7fbc67a94640 1 -- 172.21.15.164:0/922834680 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc60146b30 con 0x7fbc600b3990 2024-03-23T14:57:01.814 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:01.810+0000 7fbc567fc640 1 -- 172.21.15.164:0/922834680 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc58014b50 con 0x7fbc600b3990 2024-03-23T14:57:01.814 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:01.810+0000 7fbc67a94640 1 -- 172.21.15.164:0/922834680 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc600b08d0 con 0x7fbc600b3990 2024-03-23T14:57:01.815 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:01.810+0000 7fbc67a94640 1 -- 172.21.15.164:0/922834680 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc60147010 con 0x7fbc600b3990 2024-03-23T14:57:01.815 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:01.810+0000 7fbc567fc640 1 -- 172.21.15.164:0/922834680 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7fbc58014cb0 con 0x7fbc600b3990 2024-03-23T14:57:01.815 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:01.810+0000 7fbc567fc640 1 --2- 172.21.15.164:0/922834680 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fbc440672a0 0x7fbc44069760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:01.815 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:01.810+0000 7fbc567fc640 1 -- 172.21.15.164:0/922834680 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fbc580859d0 con 0x7fbc600b3990 2024-03-23T14:57:01.815 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:01.810+0000 7fbc65830640 1 --2- 172.21.15.164:0/922834680 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fbc440672a0 0x7fbc44069760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:01.815 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:01.810+0000 7fbc65830640 1 --2- 172.21.15.164:0/922834680 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fbc440672a0 0x7fbc44069760 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fbc5c0ae000 tx=0x7fbc5c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:01.818 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:01.814+0000 7fbc567fc640 1 -- 172.21.15.164:0/922834680 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7fbc60147010 con 0x7fbc600b3990 2024-03-23T14:57:02.010 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.006+0000 7fbc67a94640 1 -- 172.21.15.164:0/922834680 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbc60147010 con 0x7fbc600b3990 2024-03-23T14:57:02.011 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.006+0000 7fbc567fc640 1 -- 172.21.15.164:0/922834680 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+3533 (secure 0 0 0) 0x7fbc60147010 con 0x7fbc600b3990 2024-03-23T14:57:02.011 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-23T14:57:02.012 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/3352372536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":3352372536},{"type":"v1","addr":"172.21.15.164:6835","nonce":3352372536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-23T14:56:58.953326+0000","modified":"2024-03-23T14:57:00.999623+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4279":{"gid":4279,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.173:6833/3506402771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":3506402771},{"type":"v1","addr":"172.21.15.173:6833","nonce":3506402771}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4279,"qdb_cluster":[4279]},"id":1}]} 2024-03-23T14:57:02.012 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 6 2024-03-23T14:57:02.014 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.010+0000 7fbc67a94640 1 -- 172.21.15.164:0/922834680 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fbc440672a0 msgr2=0x7fbc44069760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:02.014 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.010+0000 7fbc67a94640 1 --2- 172.21.15.164:0/922834680 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fbc440672a0 0x7fbc44069760 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fbc5c0ae000 tx=0x7fbc5c008040 comp rx=0 tx=0).stop 2024-03-23T14:57:02.014 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.010+0000 7fbc67a94640 1 -- 172.21.15.164:0/922834680 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbc600b3990 msgr2=0x7fbc60136930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:02.014 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.010+0000 7fbc67a94640 1 --2- 172.21.15.164:0/922834680 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbc600b3990 0x7fbc60136930 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fbc58007e70 tx=0x7fbc5800e4b0 comp rx=0 tx=0).stop 2024-03-23T14:57:02.014 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.010+0000 7fbc67a94640 1 -- 172.21.15.164:0/922834680 shutdown_connections 2024-03-23T14:57:02.014 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.010+0000 7fbc67a94640 1 --2- 172.21.15.164:0/922834680 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbc600b3990 0x7fbc60136930 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:02.014 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.010+0000 7fbc67a94640 1 --2- 172.21.15.164:0/922834680 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbc600725b0 0x7fbc60147520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:02.014 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.010+0000 7fbc67a94640 1 --2- 172.21.15.164:0/922834680 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fbc440672a0 0x7fbc44069760 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:02.014 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.010+0000 7fbc67a94640 1 --2- 172.21.15.164:0/922834680 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbc6006e730 0x7fbc60148090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:02.014 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.010+0000 7fbc67a94640 1 -- 172.21.15.164:0/922834680 >> 172.21.15.164:0/922834680 conn(0x7fbc600b7970 msgr2=0x7fbc600b4900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:02.014 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.010+0000 7fbc67a94640 1 -- 172.21.15.164:0/922834680 shutdown_connections 2024-03-23T14:57:02.014 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.010+0000 7fbc67a94640 1 -- 172.21.15.164:0/922834680 wait complete. 2024-03-23T14:57:02.031 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T14:57:02.147 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.142+0000 7f5dfc955640 1 Processor -- start 2024-03-23T14:57:02.147 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.142+0000 7f5dfc955640 1 -- start start 2024-03-23T14:57:02.149 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.146+0000 7f5dfc955640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f5df809def0 0x7f5df80ae620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:02.149 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.146+0000 7f5dfc955640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f5df80b4a00 0x7f5df80b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:02.149 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.146+0000 7f5dfc955640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5df809e2d0 con 0x7f5df8147420 2024-03-23T14:57:02.149 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.146+0000 7f5dfc955640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f5df8146de0 con 0x7f5df809def0 2024-03-23T14:57:02.149 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.146+0000 7f5dfc955640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 -- 0x7f5df806cb70 con 0x7f5df80b4a00 2024-03-23T14:57:02.149 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.146+0000 7f5df659c640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f5df809def0 0x7f5df80ae620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:02.149 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.146+0000 7f5df6d9d640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f5df8147420 0x7f5df80b34d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:45454/0 (socket says 172.21.15.164:45454) 2024-03-23T14:57:02.149 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.146+0000 7f5df6d9d640 1 -- 172.21.15.164:0/3028940042 learned_addr learned my addr 172.21.15.164:0/3028940042 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-03-23T14:57:02.149 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.146+0000 7f5df5d9b640 1 --2- 172.21.15.164:0/3028940042 >> v2:172.21.15.173:3300/0 conn(0x7f5df80b4a00 0x7f5df80b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:02.149 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.146+0000 7f5df659c640 1 -- 172.21.15.164:0/3028940042 >> v2:172.21.15.173:3300/0 conn(0x7f5df80b4a00 msgr2=0x7f5df80b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:02.149 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.146+0000 7f5df659c640 1 --2- 172.21.15.164:0/3028940042 >> v2:172.21.15.173:3300/0 conn(0x7f5df80b4a00 0x7f5df80b4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:02.149 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.146+0000 7f5df659c640 1 -- 172.21.15.164:0/3028940042 >> v1:172.21.15.164:6789/0 conn(0x7f5df8147420 legacy=0x7f5df80b34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:02.149 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.146+0000 7f5df659c640 1 -- 172.21.15.164:0/3028940042 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5df806ce50 con 0x7f5df809def0 2024-03-23T14:57:02.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.146+0000 7f5df659c640 1 --2- 172.21.15.164:0/3028940042 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5df809def0 0x7f5df80ae620 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5df80b4770 tx=0x7f5dec01c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9923b06d87ef0b server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:02.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.146+0000 7f5df559a640 1 -- 172.21.15.164:0/3028940042 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dec0ab600 con 0x7f5df809def0 2024-03-23T14:57:02.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.146+0000 7f5df559a640 1 -- 172.21.15.164:0/3028940042 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5dec0a7050 con 0x7f5df809def0 2024-03-23T14:57:02.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.146+0000 7f5df559a640 1 -- 172.21.15.164:0/3028940042 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dec0ab980 con 0x7f5df809def0 2024-03-23T14:57:02.152 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.146+0000 7f5dfc955640 1 -- 172.21.15.164:0/3028940042 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5df809def0 msgr2=0x7f5df80ae620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:02.152 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.146+0000 7f5dfc955640 1 --2- 172.21.15.164:0/3028940042 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5df809def0 0x7f5df80ae620 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5df80b4770 tx=0x7f5dec01c3f0 comp rx=0 tx=0).stop 2024-03-23T14:57:02.152 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.146+0000 7f5dfc955640 1 -- 172.21.15.164:0/3028940042 shutdown_connections 2024-03-23T14:57:02.152 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.146+0000 7f5dfc955640 1 --2- 172.21.15.164:0/3028940042 >> v2:172.21.15.173:3300/0 conn(0x7f5df80b4a00 0x7f5df80b4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:02.152 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.146+0000 7f5dfc955640 1 --2- 172.21.15.164:0/3028940042 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5df809def0 0x7f5df80ae620 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:02.152 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.146+0000 7f5dfc955640 1 -- 172.21.15.164:0/3028940042 >> 172.21.15.164:0/3028940042 conn(0x7f5df80b8ca0 msgr2=0x7f5df80b90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:02.152 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.146+0000 7f5dfc955640 1 -- 172.21.15.164:0/3028940042 shutdown_connections 2024-03-23T14:57:02.152 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.146+0000 7f5dfc955640 1 -- 172.21.15.164:0/3028940042 wait complete. 2024-03-23T14:57:02.153 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.150+0000 7f5dfc955640 1 Processor -- start 2024-03-23T14:57:02.153 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.150+0000 7f5dfc955640 1 -- start start 2024-03-23T14:57:02.154 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.150+0000 7f5dfc955640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5df809def0 0x7f5df814ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:02.155 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.150+0000 7f5dfc955640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5df80b4a00 0x7f5df81e32e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:02.155 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.150+0000 7f5dfc955640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5df8147420 0x7f5df81e3820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:02.155 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.150+0000 7f5dfc955640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5df80b2240 con 0x7f5df80b4a00 2024-03-23T14:57:02.155 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.150+0000 7f5dfc955640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f5df809d5a0 con 0x7f5df8147420 2024-03-23T14:57:02.155 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.150+0000 7f5dfc955640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f5df814d230 con 0x7f5df809def0 2024-03-23T14:57:02.155 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.150+0000 7f5df5d9b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5df80b4a00 0x7f5df81e32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:02.155 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.150+0000 7f5df5d9b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5df80b4a00 0x7f5df81e32e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:57608/0 (socket says 172.21.15.164:57608) 2024-03-23T14:57:02.155 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.150+0000 7f5df5d9b640 1 -- 172.21.15.164:0/3793257068 learned_addr learned my addr 172.21.15.164:0/3793257068 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:02.155 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.150+0000 7f5df659c640 1 --2- 172.21.15.164:0/3793257068 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5df809def0 0x7f5df814ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:02.155 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.150+0000 7f5df6d9d640 1 --2- 172.21.15.164:0/3793257068 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5df8147420 0x7f5df81e3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:02.155 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.150+0000 7f5df5d9b640 1 -- 172.21.15.164:0/3793257068 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5df809def0 msgr2=0x7f5df814ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:02.155 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.150+0000 7f5df5d9b640 1 --2- 172.21.15.164:0/3793257068 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5df809def0 0x7f5df814ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:02.155 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.150+0000 7f5df5d9b640 1 -- 172.21.15.164:0/3793257068 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5df8147420 msgr2=0x7f5df81e3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:02.155 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.150+0000 7f5df5d9b640 1 --2- 172.21.15.164:0/3793257068 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5df8147420 0x7f5df81e3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:02.155 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.150+0000 7f5df5d9b640 1 -- 172.21.15.164:0/3793257068 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dec009420 con 0x7f5df80b4a00 2024-03-23T14:57:02.156 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.150+0000 7f5df6d9d640 1 --2- 172.21.15.164:0/3793257068 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5df8147420 0x7f5df81e3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T14:57:02.156 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.150+0000 7f5df5d9b640 1 --2- 172.21.15.164:0/3793257068 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5df80b4a00 0x7f5df81e32e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f5de800d800 tx=0x7f5de800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:02.156 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.150+0000 7f5de6ffd640 1 -- 172.21.15.164:0/3793257068 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de800b6c0 con 0x7f5df80b4a00 2024-03-23T14:57:02.156 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.150+0000 7f5de6ffd640 1 -- 172.21.15.164:0/3793257068 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5de800b820 con 0x7f5df80b4a00 2024-03-23T14:57:02.156 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.150+0000 7f5dfc955640 1 -- 172.21.15.164:0/3793257068 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5df80addc0 con 0x7f5df80b4a00 2024-03-23T14:57:02.156 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.150+0000 7f5de6ffd640 1 -- 172.21.15.164:0/3793257068 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de800bb90 con 0x7f5df80b4a00 2024-03-23T14:57:02.156 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.150+0000 7f5dfc955640 1 -- 172.21.15.164:0/3793257068 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5df814cf40 con 0x7f5df80b4a00 2024-03-23T14:57:02.157 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.150+0000 7f5dfc955640 1 -- 172.21.15.164:0/3793257068 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5df80690a0 con 0x7f5df80b4a00 2024-03-23T14:57:02.157 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.154+0000 7f5de6ffd640 1 -- 172.21.15.164:0/3793257068 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7f5de800bcf0 con 0x7f5df80b4a00 2024-03-23T14:57:02.157 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.154+0000 7f5de6ffd640 1 --2- 172.21.15.164:0/3793257068 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f5dd80672a0 0x7f5dd8069760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:02.157 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.154+0000 7f5de6ffd640 1 -- 172.21.15.164:0/3793257068 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f5de8085570 con 0x7f5df80b4a00 2024-03-23T14:57:02.157 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.154+0000 7f5df659c640 1 --2- 172.21.15.164:0/3793257068 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f5dd80672a0 0x7f5dd8069760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:02.157 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.154+0000 7f5df659c640 1 --2- 172.21.15.164:0/3793257068 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f5dd80672a0 0x7f5dd8069760 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5dec0ad000 tx=0x7f5dec0a6000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:02.160 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.158+0000 7f5de6ffd640 1 -- 172.21.15.164:0/3793257068 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7f5de8052120 con 0x7f5df80b4a00 2024-03-23T14:57:02.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.302+0000 7f5dfc955640 1 -- 172.21.15.164:0/3793257068 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5df80adf50 con 0x7f5df80b4a00 2024-03-23T14:57:02.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.306+0000 7f5de6ffd640 1 -- 172.21.15.164:0/3793257068 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16629 (secure 0 0 0) 0x7f5de80557e0 con 0x7f5df80b4a00 2024-03-23T14:57:02.311 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-23T14:57:02.311 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":20,"fsid":"45c7911b-b69d-4e68-907b-fbfeb780aa3b","created":"2024-03-23T14:56:34.367633+0000","modified":"2024-03-23T14:57:00.611265+0000","last_up_change":"2024-03-23T14:56:46.297403+0000","last_in_change":"2024-03-23T14:56:36.629884+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T14:56:47.186157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-23T14:56:56.994057+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-23T14:56:57.953862+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"80abdc62-bd0d-4e60-a554-7ed6fd67cf54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":984716690},{"type":"v1","addr":"172.21.15.164:6801","nonce":984716690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":984716690},{"type":"v1","addr":"172.21.15.164:6803","nonce":984716690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":984716690},{"type":"v1","addr":"172.21.15.164:6807","nonce":984716690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":984716690},{"type":"v1","addr":"172.21.15.164:6805","nonce":984716690}]},"public_addr":"172.21.15.164:6801/984716690","cluster_addr":"172.21.15.164:6803/984716690","heartbeat_back_addr":"172.21.15.164:6807/984716690","heartbeat_front_addr":"172.21.15.164:6805/984716690","state":["exists","up"]},{"osd":1,"uuid":"f9001b06-be4d-4f42-a0e6-bcef2afa0d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":1349657239},{"type":"v1","addr":"172.21.15.164:6817","nonce":1349657239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":1349657239},{"type":"v1","addr":"172.21.15.164:6819","nonce":1349657239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":1349657239},{"type":"v1","addr":"172.21.15.164:6823","nonce":1349657239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":1349657239},{"type":"v1","addr":"172.21.15.164:6821","nonce":1349657239}]},"public_addr":"172.21.15.164:6817/1349657239","cluster_addr":"172.21.15.164:6819/1349657239","heartbeat_back_addr":"172.21.15.164:6823/1349657239","heartbeat_front_addr":"172.21.15.164:6821/1349657239","state":["exists","up"]},{"osd":2,"uuid":"fa0aeb9f-8b54-4831-a188-3e361c7980d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":887861252},{"type":"v1","addr":"172.21.15.164:6809","nonce":887861252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":887861252},{"type":"v1","addr":"172.21.15.164:6811","nonce":887861252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":887861252},{"type":"v1","addr":"172.21.15.164:6815","nonce":887861252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":887861252},{"type":"v1","addr":"172.21.15.164:6813","nonce":887861252}]},"public_addr":"172.21.15.164:6809/887861252","cluster_addr":"172.21.15.164:6811/887861252","heartbeat_back_addr":"172.21.15.164:6815/887861252","heartbeat_front_addr":"172.21.15.164:6813/887861252","state":["exists","up"]},{"osd":3,"uuid":"7e18c957-3091-46d5-8e17-8f3354c1726c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":812540271},{"type":"v1","addr":"172.21.15.164:6825","nonce":812540271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":812540271},{"type":"v1","addr":"172.21.15.164:6827","nonce":812540271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":812540271},{"type":"v1","addr":"172.21.15.164:6831","nonce":812540271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":812540271},{"type":"v1","addr":"172.21.15.164:6829","nonce":812540271}]},"public_addr":"172.21.15.164:6825/812540271","cluster_addr":"172.21.15.164:6827/812540271","heartbeat_back_addr":"172.21.15.164:6831/812540271","heartbeat_front_addr":"172.21.15.164:6829/812540271","state":["exists","up"]},{"osd":4,"uuid":"8e83f315-42bf-4471-9ce8-d81948da0f9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":2656857421},{"type":"v1","addr":"172.21.15.173:6825","nonce":2656857421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2656857421},{"type":"v1","addr":"172.21.15.173:6827","nonce":2656857421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2656857421},{"type":"v1","addr":"172.21.15.173:6831","nonce":2656857421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":2656857421},{"type":"v1","addr":"172.21.15.173:6829","nonce":2656857421}]},"public_addr":"172.21.15.173:6825/2656857421","cluster_addr":"172.21.15.173:6827/2656857421","heartbeat_back_addr":"172.21.15.173:6831/2656857421","heartbeat_front_addr":"172.21.15.173:6829/2656857421","state":["exists","up"]},{"osd":5,"uuid":"a230c872-71f9-40ab-8999-defd0162d4ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":3676483185},{"type":"v1","addr":"172.21.15.173:6801","nonce":3676483185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":3676483185},{"type":"v1","addr":"172.21.15.173:6803","nonce":3676483185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":3676483185},{"type":"v1","addr":"172.21.15.173:6807","nonce":3676483185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":3676483185},{"type":"v1","addr":"172.21.15.173:6805","nonce":3676483185}]},"public_addr":"172.21.15.173:6801/3676483185","cluster_addr":"172.21.15.173:6803/3676483185","heartbeat_back_addr":"172.21.15.173:6807/3676483185","heartbeat_front_addr":"172.21.15.173:6805/3676483185","state":["exists","up"]},{"osd":6,"uuid":"6193bfd7-8432-4f7f-a117-a3fcee21af99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":3631684087},{"type":"v1","addr":"172.21.15.173:6809","nonce":3631684087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":3631684087},{"type":"v1","addr":"172.21.15.173:6811","nonce":3631684087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":3631684087},{"type":"v1","addr":"172.21.15.173:6815","nonce":3631684087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":3631684087},{"type":"v1","addr":"172.21.15.173:6813","nonce":3631684087}]},"public_addr":"172.21.15.173:6809/3631684087","cluster_addr":"172.21.15.173:6811/3631684087","heartbeat_back_addr":"172.21.15.173:6815/3631684087","heartbeat_front_addr":"172.21.15.173:6813/3631684087","state":["exists","up"]},{"osd":7,"uuid":"b0f1b80a-afc4-4538-83b9-f081d2dcb8cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":3261114948},{"type":"v1","addr":"172.21.15.173:6817","nonce":3261114948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":3261114948},{"type":"v1","addr":"172.21.15.173:6819","nonce":3261114948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":3261114948},{"type":"v1","addr":"172.21.15.173:6823","nonce":3261114948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":3261114948},{"type":"v1","addr":"172.21.15.173:6821","nonce":3261114948}]},"public_addr":"172.21.15.173:6817/3261114948","cluster_addr":"172.21.15.173:6819/3261114948","heartbeat_back_addr":"172.21.15.173:6823/3261114948","heartbeat_front_addr":"172.21.15.173:6821/3261114948","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.188390+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.159932+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.191101+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.181776+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.224365+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.177515+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.153631+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.139972+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T14:57:02.313 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.310+0000 7f5dfc955640 1 -- 172.21.15.164:0/3793257068 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f5dd80672a0 msgr2=0x7f5dd8069760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:02.313 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.310+0000 7f5dfc955640 1 --2- 172.21.15.164:0/3793257068 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f5dd80672a0 0x7f5dd8069760 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5dec0ad000 tx=0x7f5dec0a6000 comp rx=0 tx=0).stop 2024-03-23T14:57:02.313 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.310+0000 7f5dfc955640 1 -- 172.21.15.164:0/3793257068 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5df80b4a00 msgr2=0x7f5df81e32e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:02.313 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.310+0000 7f5dfc955640 1 --2- 172.21.15.164:0/3793257068 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5df80b4a00 0x7f5df81e32e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f5de800d800 tx=0x7f5de800dcc0 comp rx=0 tx=0).stop 2024-03-23T14:57:02.313 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.310+0000 7f5dfc955640 1 -- 172.21.15.164:0/3793257068 shutdown_connections 2024-03-23T14:57:02.313 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.310+0000 7f5dfc955640 1 --2- 172.21.15.164:0/3793257068 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f5dd80672a0 0x7f5dd8069760 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:02.313 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.310+0000 7f5dfc955640 1 --2- 172.21.15.164:0/3793257068 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5df8147420 0x7f5df81e3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:02.313 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.310+0000 7f5dfc955640 1 --2- 172.21.15.164:0/3793257068 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5df80b4a00 0x7f5df81e32e0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:02.313 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.310+0000 7f5dfc955640 1 --2- 172.21.15.164:0/3793257068 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5df809def0 0x7f5df814ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:02.313 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.310+0000 7f5dfc955640 1 -- 172.21.15.164:0/3793257068 >> 172.21.15.164:0/3793257068 conn(0x7f5df80b8ca0 msgr2=0x7f5df80b9080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:02.313 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.310+0000 7f5dfc955640 1 -- 172.21.15.164:0/3793257068 shutdown_connections 2024-03-23T14:57:02.313 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.310+0000 7f5dfc955640 1 -- 172.21.15.164:0/3793257068 wait complete. 2024-03-23T14:57:02.328 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-23T14:57:02.328 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-23T14:57:02.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.434+0000 7f559dc45640 1 Processor -- start 2024-03-23T14:57:02.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.434+0000 7f559dc45640 1 -- start start 2024-03-23T14:57:02.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.434+0000 7f559dc45640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f559802c2a0 0x7f559802c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:02.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.434+0000 7f559dc45640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f559814fc00 0x7f5598151ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:02.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.434+0000 7f559dc45640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5598141a90 con 0x7f559806e8e0 2024-03-23T14:57:02.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.434+0000 7f559dc45640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f55980b9470 con 0x7f559802c2a0 2024-03-23T14:57:02.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.434+0000 7f559dc45640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 -- 0x7f5598072af0 con 0x7f559814fc00 2024-03-23T14:57:02.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.434+0000 7f5596ffd640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f559802c2a0 0x7f559802c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:02.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.434+0000 7f5596ffd640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f559802c2a0 0x7f559802c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:58964/0 (socket says 172.21.15.164:58964) 2024-03-23T14:57:02.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.434+0000 7f5596ffd640 1 -- 172.21.15.164:0/2458633568 learned_addr learned my addr 172.21.15.164:0/2458633568 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:02.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.434+0000 7f5597fff640 1 --2- 172.21.15.164:0/2458633568 >> v2:172.21.15.173:3300/0 conn(0x7f559814fc00 0x7f5598151ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:02.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.434+0000 7f5596ffd640 1 -- 172.21.15.164:0/2458633568 >> v2:172.21.15.173:3300/0 conn(0x7f559814fc00 msgr2=0x7f5598151ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:02.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.434+0000 7f5596ffd640 1 --2- 172.21.15.164:0/2458633568 >> v2:172.21.15.173:3300/0 conn(0x7f559814fc00 0x7f5598151ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:02.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.434+0000 7f5596ffd640 1 -- 172.21.15.164:0/2458633568 >> v1:172.21.15.173:6789/0 conn(0x7f559806e8e0 legacy=0x7f5598141380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:02.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.434+0000 7f5596ffd640 1 -- 172.21.15.164:0/2458633568 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f559806ecc0 con 0x7f559802c2a0 2024-03-23T14:57:02.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.438+0000 7f5596ffd640 1 --2- 172.21.15.164:0/2458633568 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f559802c2a0 0x7f559802c6e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f559802d560 tx=0x7f558801f1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=976e6d6cc7eccf62 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:02.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.438+0000 7f55967fc640 1 -- 172.21.15.164:0/2458633568 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55880af400 con 0x7f559802c2a0 2024-03-23T14:57:02.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.438+0000 7f55967fc640 1 -- 172.21.15.164:0/2458633568 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55880ab050 con 0x7f559802c2a0 2024-03-23T14:57:02.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.438+0000 7f55967fc640 1 -- 172.21.15.164:0/2458633568 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55880af780 con 0x7f559802c2a0 2024-03-23T14:57:02.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.438+0000 7f559dc45640 1 -- 172.21.15.164:0/2458633568 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f559802c2a0 msgr2=0x7f559802c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:02.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.438+0000 7f559dc45640 1 --2- 172.21.15.164:0/2458633568 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f559802c2a0 0x7f559802c6e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f559802d560 tx=0x7f558801f1e0 comp rx=0 tx=0).stop 2024-03-23T14:57:02.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.438+0000 7f559dc45640 1 -- 172.21.15.164:0/2458633568 shutdown_connections 2024-03-23T14:57:02.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.438+0000 7f559dc45640 1 --2- 172.21.15.164:0/2458633568 >> v2:172.21.15.173:3300/0 conn(0x7f559814fc00 0x7f5598151ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:02.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.438+0000 7f559dc45640 1 --2- 172.21.15.164:0/2458633568 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f559802c2a0 0x7f559802c6e0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:02.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.438+0000 7f559dc45640 1 -- 172.21.15.164:0/2458633568 >> 172.21.15.164:0/2458633568 conn(0x7f55980b7970 msgr2=0x7f55980b7d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:02.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.438+0000 7f559dc45640 1 -- 172.21.15.164:0/2458633568 shutdown_connections 2024-03-23T14:57:02.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.438+0000 7f559dc45640 1 -- 172.21.15.164:0/2458633568 wait complete. 2024-03-23T14:57:02.443 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.438+0000 7f559dc45640 1 Processor -- start 2024-03-23T14:57:02.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.438+0000 7f559dc45640 1 -- start start 2024-03-23T14:57:02.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.438+0000 7f559dc45640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f559802c2a0 0x7f559813b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:02.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.438+0000 7f559dc45640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f559806e8e0 0x7f55980b3920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:02.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.438+0000 7f559dc45640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f559814fc00 0x7f5598136c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:02.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.438+0000 7f559dc45640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f55980b22c0 con 0x7f559814fc00 2024-03-23T14:57:02.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.438+0000 7f559dc45640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f55980b3e60 con 0x7f559802c2a0 2024-03-23T14:57:02.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.438+0000 7f559dc45640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f55981469b0 con 0x7f559806e8e0 2024-03-23T14:57:02.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.438+0000 7f5597fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f559814fc00 0x7f5598136c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:02.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.438+0000 7f5597fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f559814fc00 0x7f5598136c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:58968/0 (socket says 172.21.15.164:58968) 2024-03-23T14:57:02.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.438+0000 7f5597fff640 1 -- 172.21.15.164:0/2139979472 learned_addr learned my addr 172.21.15.164:0/2139979472 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:02.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.438+0000 7f5596ffd640 1 --2- 172.21.15.164:0/2139979472 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f559806e8e0 0x7f55980b3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:02.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.438+0000 7f55977fe640 1 --2- 172.21.15.164:0/2139979472 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f559802c2a0 0x7f559813b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:02.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.438+0000 7f5597fff640 1 -- 172.21.15.164:0/2139979472 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f559806e8e0 msgr2=0x7f55980b3920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:02.445 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.438+0000 7f5597fff640 1 --2- 172.21.15.164:0/2139979472 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f559806e8e0 0x7f55980b3920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:02.445 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.438+0000 7f5597fff640 1 -- 172.21.15.164:0/2139979472 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f559802c2a0 msgr2=0x7f559813b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:02.445 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.438+0000 7f5597fff640 1 --2- 172.21.15.164:0/2139979472 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f559802c2a0 0x7f559813b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:02.445 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.438+0000 7f5597fff640 1 -- 172.21.15.164:0/2139979472 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5588009b30 con 0x7f559814fc00 2024-03-23T14:57:02.445 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.438+0000 7f5596ffd640 1 --2- 172.21.15.164:0/2139979472 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f559806e8e0 0x7f55980b3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T14:57:02.445 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.438+0000 7f5597fff640 1 --2- 172.21.15.164:0/2139979472 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f559814fc00 0x7f5598136c60 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5590006e90 tx=0x7f559000d9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:02.445 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.438+0000 7f557ffff640 1 -- 172.21.15.164:0/2139979472 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55900125f0 con 0x7f559814fc00 2024-03-23T14:57:02.445 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.438+0000 7f557ffff640 1 -- 172.21.15.164:0/2139979472 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5590012750 con 0x7f559814fc00 2024-03-23T14:57:02.445 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.438+0000 7f557ffff640 1 -- 172.21.15.164:0/2139979472 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5590012b00 con 0x7f559814fc00 2024-03-23T14:57:02.445 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.438+0000 7f559dc45640 1 -- 172.21.15.164:0/2139979472 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5598146c90 con 0x7f559814fc00 2024-03-23T14:57:02.445 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.438+0000 7f559dc45640 1 -- 172.21.15.164:0/2139979472 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f559813c5f0 con 0x7f559814fc00 2024-03-23T14:57:02.446 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.442+0000 7f559dc45640 1 -- 172.21.15.164:0/2139979472 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5598147190 con 0x7f559814fc00 2024-03-23T14:57:02.446 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.442+0000 7f557ffff640 1 -- 172.21.15.164:0/2139979472 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7f5590012c60 con 0x7f559814fc00 2024-03-23T14:57:02.446 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.442+0000 7f557ffff640 1 --2- 172.21.15.164:0/2139979472 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f55840672a0 0x7f5584069760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:02.446 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.442+0000 7f55977fe640 1 --2- 172.21.15.164:0/2139979472 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f55840672a0 0x7f5584069760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:02.446 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.442+0000 7f557ffff640 1 -- 172.21.15.164:0/2139979472 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f5590085270 con 0x7f559814fc00 2024-03-23T14:57:02.446 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.442+0000 7f55977fe640 1 --2- 172.21.15.164:0/2139979472 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f55840672a0 0x7f5584069760 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f558c001810 tx=0x7f558c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:02.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.446+0000 7f557ffff640 1 -- 172.21.15.164:0/2139979472 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7f5590051eb0 con 0x7f559814fc00 2024-03-23T14:57:02.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.586+0000 7f559dc45640 1 -- 172.21.15.164:0/2139979472 --> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f559802c680 con 0x7f55840672a0 2024-03-23T14:57:02.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.598+0000 7f557ffff640 1 -- 172.21.15.164:0/2139979472 <== mgr.4107 v2:172.21.15.164:6832/2139087985 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+237079 (secure 0 0 0) 0x7f559802c680 con 0x7f55840672a0 2024-03-23T14:57:02.602 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-23T14:57:02.604 INFO:teuthology.orchestra.run.smithi164.stderr:dumped all 2024-03-23T14:57:02.605 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.602+0000 7f559dc45640 1 -- 172.21.15.164:0/2139979472 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f55840672a0 msgr2=0x7f5584069760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:02.605 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.602+0000 7f559dc45640 1 --2- 172.21.15.164:0/2139979472 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f55840672a0 0x7f5584069760 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f558c001810 tx=0x7f558c009040 comp rx=0 tx=0).stop 2024-03-23T14:57:02.605 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.602+0000 7f559dc45640 1 -- 172.21.15.164:0/2139979472 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f559814fc00 msgr2=0x7f5598136c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:02.605 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.602+0000 7f559dc45640 1 --2- 172.21.15.164:0/2139979472 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f559814fc00 0x7f5598136c60 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5590006e90 tx=0x7f559000d9e0 comp rx=0 tx=0).stop 2024-03-23T14:57:02.605 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.602+0000 7f559dc45640 1 -- 172.21.15.164:0/2139979472 shutdown_connections 2024-03-23T14:57:02.605 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.602+0000 7f559dc45640 1 --2- 172.21.15.164:0/2139979472 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f559814fc00 0x7f5598136c60 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:02.605 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.602+0000 7f559dc45640 1 --2- 172.21.15.164:0/2139979472 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f559806e8e0 0x7f55980b3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:02.605 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.602+0000 7f559dc45640 1 --2- 172.21.15.164:0/2139979472 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f55840672a0 0x7f5584069760 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:02.605 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.602+0000 7f559dc45640 1 --2- 172.21.15.164:0/2139979472 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f559802c2a0 0x7f559813b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:02.605 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.602+0000 7f559dc45640 1 -- 172.21.15.164:0/2139979472 >> 172.21.15.164:0/2139979472 conn(0x7f55980b7970 msgr2=0x7f55980b82a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:02.605 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.602+0000 7f559dc45640 1 -- 172.21.15.164:0/2139979472 shutdown_connections 2024-03-23T14:57:02.605 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.602+0000 7f559dc45640 1 -- 172.21.15.164:0/2139979472 wait complete. 2024-03-23T14:57:02.619 INFO:teuthology.orchestra.run.smithi164.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2024-03-23T14:57:01.109719+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":218808,"kb_used_data":2376,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755912,"statfs":{"total":773094113280,"available":772870053888,"internally_reserved":0,"allocated":2433024,"data_stored":1546872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001307"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:58.583412+0000","last_change":"2024-03-23T14:56:58.583412+0000","last_active":"2024-03-23T14:56:58.583412+0000","last_peered":"2024-03-23T14:56:58.583412+0000","last_clean":"2024-03-23T14:56:58.583412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:58.583412+0000","last_undegraded":"2024-03-23T14:56:58.583412+0000","last_fullsized":"2024-03-23T14:56:58.583412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:58.583412+0000","last_change":"2024-03-23T14:56:58.583412+0000","last_active":"2024-03-23T14:56:58.583412+0000","last_peered":"2024-03-23T14:56:58.583412+0000","last_clean":"2024-03-23T14:56:58.583412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:58.583412+0000","last_undegraded":"2024-03-23T14:56:58.583412+0000","last_fullsized":"2024-03-23T14:56:58.583412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:58.583412+0000","last_change":"2024-03-23T14:56:58.583412+0000","last_active":"2024-03-23T14:56:58.583412+0000","last_peered":"2024-03-23T14:56:58.583412+0000","last_clean":"2024-03-23T14:56:58.583412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:58.583412+0000","last_undegraded":"2024-03-23T14:56:58.583412+0000","last_fullsized":"2024-03-23T14:56:58.583412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:58.583412+0000","last_change":"2024-03-23T14:56:58.583412+0000","last_active":"2024-03-23T14:56:58.583412+0000","last_peered":"2024-03-23T14:56:58.583412+0000","last_clean":"2024-03-23T14:56:58.583412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:58.583412+0000","last_undegraded":"2024-03-23T14:56:58.583412+0000","last_fullsized":"2024-03-23T14:56:58.583412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:58.583412+0000","last_change":"2024-03-23T14:56:58.583412+0000","last_active":"2024-03-23T14:56:58.583412+0000","last_peered":"2024-03-23T14:56:58.583412+0000","last_clean":"2024-03-23T14:56:58.583412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:58.583412+0000","last_undegraded":"2024-03-23T14:56:58.583412+0000","last_fullsized":"2024-03-23T14:56:58.583412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:58.583412+0000","last_change":"2024-03-23T14:56:58.583412+0000","last_active":"2024-03-23T14:56:58.583412+0000","last_peered":"2024-03-23T14:56:58.583412+0000","last_clean":"2024-03-23T14:56:58.583412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:58.583412+0000","last_undegraded":"2024-03-23T14:56:58.583412+0000","last_fullsized":"2024-03-23T14:56:58.583412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:58.583412+0000","last_change":"2024-03-23T14:56:58.583412+0000","last_active":"2024-03-23T14:56:58.583412+0000","last_peered":"2024-03-23T14:56:58.583412+0000","last_clean":"2024-03-23T14:56:58.583412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:58.583412+0000","last_undegraded":"2024-03-23T14:56:58.583412+0000","last_fullsized":"2024-03-23T14:56:58.583412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:58.583412+0000","last_change":"2024-03-23T14:56:58.583412+0000","last_active":"2024-03-23T14:56:58.583412+0000","last_peered":"2024-03-23T14:56:58.583412+0000","last_clean":"2024-03-23T14:56:58.583412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:58.583412+0000","last_undegraded":"2024-03-23T14:56:58.583412+0000","last_fullsized":"2024-03-23T14:56:58.583412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:58.583412+0000","last_change":"2024-03-23T14:56:58.583412+0000","last_active":"2024-03-23T14:56:58.583412+0000","last_peered":"2024-03-23T14:56:58.583412+0000","last_clean":"2024-03-23T14:56:58.583412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:58.583412+0000","last_undegraded":"2024-03-23T14:56:58.583412+0000","last_fullsized":"2024-03-23T14:56:58.583412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:58.583412+0000","last_change":"2024-03-23T14:56:58.583412+0000","last_active":"2024-03-23T14:56:58.583412+0000","last_peered":"2024-03-23T14:56:58.583412+0000","last_clean":"2024-03-23T14:56:58.583412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:58.583412+0000","last_undegraded":"2024-03-23T14:56:58.583412+0000","last_fullsized":"2024-03-23T14:56:58.583412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:58.583412+0000","last_change":"2024-03-23T14:56:58.583412+0000","last_active":"2024-03-23T14:56:58.583412+0000","last_peered":"2024-03-23T14:56:58.583412+0000","last_clean":"2024-03-23T14:56:58.583412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:58.583412+0000","last_undegraded":"2024-03-23T14:56:58.583412+0000","last_fullsized":"2024-03-23T14:56:58.583412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:58.583412+0000","last_change":"2024-03-23T14:56:58.583412+0000","last_active":"2024-03-23T14:56:58.583412+0000","last_peered":"2024-03-23T14:56:58.583412+0000","last_clean":"2024-03-23T14:56:58.583412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:58.583412+0000","last_undegraded":"2024-03-23T14:56:58.583412+0000","last_fullsized":"2024-03-23T14:56:58.583412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:58.583412+0000","last_change":"2024-03-23T14:56:58.583412+0000","last_active":"2024-03-23T14:56:58.583412+0000","last_peered":"2024-03-23T14:56:58.583412+0000","last_clean":"2024-03-23T14:56:58.583412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:58.583412+0000","last_undegraded":"2024-03-23T14:56:58.583412+0000","last_fullsized":"2024-03-23T14:56:58.583412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:58.583412+0000","last_change":"2024-03-23T14:56:58.583412+0000","last_active":"2024-03-23T14:56:58.583412+0000","last_peered":"2024-03-23T14:56:58.583412+0000","last_clean":"2024-03-23T14:56:58.583412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:58.583412+0000","last_undegraded":"2024-03-23T14:56:58.583412+0000","last_fullsized":"2024-03-23T14:56:58.583412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:58.583412+0000","last_change":"2024-03-23T14:56:58.583412+0000","last_active":"2024-03-23T14:56:58.583412+0000","last_peered":"2024-03-23T14:56:58.583412+0000","last_clean":"2024-03-23T14:56:58.583412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:58.583412+0000","last_undegraded":"2024-03-23T14:56:58.583412+0000","last_fullsized":"2024-03-23T14:56:58.583412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:58.583412+0000","last_change":"2024-03-23T14:56:58.583412+0000","last_active":"2024-03-23T14:56:58.583412+0000","last_peered":"2024-03-23T14:56:58.583412+0000","last_clean":"2024-03-23T14:56:58.583412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:58.583412+0000","last_undegraded":"2024-03-23T14:56:58.583412+0000","last_fullsized":"2024-03-23T14:56:58.583412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:58.583412+0000","last_change":"2024-03-23T14:56:58.583412+0000","last_active":"2024-03-23T14:56:58.583412+0000","last_peered":"2024-03-23T14:56:58.583412+0000","last_clean":"2024-03-23T14:56:58.583412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:58.583412+0000","last_undegraded":"2024-03-23T14:56:58.583412+0000","last_fullsized":"2024-03-23T14:56:58.583412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:58.583412+0000","last_change":"2024-03-23T14:56:58.583412+0000","last_active":"2024-03-23T14:56:58.583412+0000","last_peered":"2024-03-23T14:56:58.583412+0000","last_clean":"2024-03-23T14:56:58.583412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:58.583412+0000","last_undegraded":"2024-03-23T14:56:58.583412+0000","last_fullsized":"2024-03-23T14:56:58.583412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:58.583412+0000","last_change":"2024-03-23T14:56:58.583412+0000","last_active":"2024-03-23T14:56:58.583412+0000","last_peered":"2024-03-23T14:56:58.583412+0000","last_clean":"2024-03-23T14:56:58.583412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:58.583412+0000","last_undegraded":"2024-03-23T14:56:58.583412+0000","last_fullsized":"2024-03-23T14:56:58.583412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:58.583412+0000","last_change":"2024-03-23T14:56:58.583412+0000","last_active":"2024-03-23T14:56:58.583412+0000","last_peered":"2024-03-23T14:56:58.583412+0000","last_clean":"2024-03-23T14:56:58.583412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:58.583412+0000","last_undegraded":"2024-03-23T14:56:58.583412+0000","last_fullsized":"2024-03-23T14:56:58.583412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:58.583412+0000","last_change":"2024-03-23T14:56:58.583412+0000","last_active":"2024-03-23T14:56:58.583412+0000","last_peered":"2024-03-23T14:56:58.583412+0000","last_clean":"2024-03-23T14:56:58.583412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:58.583412+0000","last_undegraded":"2024-03-23T14:56:58.583412+0000","last_fullsized":"2024-03-23T14:56:58.583412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:58.583412+0000","last_change":"2024-03-23T14:56:58.583412+0000","last_active":"2024-03-23T14:56:58.583412+0000","last_peered":"2024-03-23T14:56:58.583412+0000","last_clean":"2024-03-23T14:56:58.583412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:58.583412+0000","last_undegraded":"2024-03-23T14:56:58.583412+0000","last_fullsized":"2024-03-23T14:56:58.583412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:58.583412+0000","last_change":"2024-03-23T14:56:58.583412+0000","last_active":"2024-03-23T14:56:58.583412+0000","last_peered":"2024-03-23T14:56:58.583412+0000","last_clean":"2024-03-23T14:56:58.583412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:58.583412+0000","last_undegraded":"2024-03-23T14:56:58.583412+0000","last_fullsized":"2024-03-23T14:56:58.583412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:58.583412+0000","last_change":"2024-03-23T14:56:58.583412+0000","last_active":"2024-03-23T14:56:58.583412+0000","last_peered":"2024-03-23T14:56:58.583412+0000","last_clean":"2024-03-23T14:56:58.583412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:58.583412+0000","last_undegraded":"2024-03-23T14:56:58.583412+0000","last_fullsized":"2024-03-23T14:56:58.583412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:58.583412+0000","last_change":"2024-03-23T14:56:58.583412+0000","last_active":"2024-03-23T14:56:58.583412+0000","last_peered":"2024-03-23T14:56:58.583412+0000","last_clean":"2024-03-23T14:56:58.583412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:58.583412+0000","last_undegraded":"2024-03-23T14:56:58.583412+0000","last_fullsized":"2024-03-23T14:56:58.583412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:58.583412+0000","last_change":"2024-03-23T14:56:58.583412+0000","last_active":"2024-03-23T14:56:58.583412+0000","last_peered":"2024-03-23T14:56:58.583412+0000","last_clean":"2024-03-23T14:56:58.583412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:58.583412+0000","last_undegraded":"2024-03-23T14:56:58.583412+0000","last_fullsized":"2024-03-23T14:56:58.583412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:58.583412+0000","last_change":"2024-03-23T14:56:58.583412+0000","last_active":"2024-03-23T14:56:58.583412+0000","last_peered":"2024-03-23T14:56:58.583412+0000","last_clean":"2024-03-23T14:56:58.583412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:58.583412+0000","last_undegraded":"2024-03-23T14:56:58.583412+0000","last_fullsized":"2024-03-23T14:56:58.583412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:58.583412+0000","last_change":"2024-03-23T14:56:58.583412+0000","last_active":"2024-03-23T14:56:58.583412+0000","last_peered":"2024-03-23T14:56:58.583412+0000","last_clean":"2024-03-23T14:56:58.583412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:58.583412+0000","last_undegraded":"2024-03-23T14:56:58.583412+0000","last_fullsized":"2024-03-23T14:56:58.583412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:58.583412+0000","last_change":"2024-03-23T14:56:58.583412+0000","last_active":"2024-03-23T14:56:58.583412+0000","last_peered":"2024-03-23T14:56:58.583412+0000","last_clean":"2024-03-23T14:56:58.583412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:58.583412+0000","last_undegraded":"2024-03-23T14:56:58.583412+0000","last_fullsized":"2024-03-23T14:56:58.583412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:58.583412+0000","last_change":"2024-03-23T14:56:58.583412+0000","last_active":"2024-03-23T14:56:58.583412+0000","last_peered":"2024-03-23T14:56:58.583412+0000","last_clean":"2024-03-23T14:56:58.583412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:58.583412+0000","last_undegraded":"2024-03-23T14:56:58.583412+0000","last_fullsized":"2024-03-23T14:56:58.583412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:58.583412+0000","last_change":"2024-03-23T14:56:58.583412+0000","last_active":"2024-03-23T14:56:58.583412+0000","last_peered":"2024-03-23T14:56:58.583412+0000","last_clean":"2024-03-23T14:56:58.583412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:58.583412+0000","last_undegraded":"2024-03-23T14:56:58.583412+0000","last_fullsized":"2024-03-23T14:56:58.583412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:58.583412+0000","last_change":"2024-03-23T14:56:58.583412+0000","last_active":"2024-03-23T14:56:58.583412+0000","last_peered":"2024-03-23T14:56:58.583412+0000","last_clean":"2024-03-23T14:56:58.583412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:58.583412+0000","last_undegraded":"2024-03-23T14:56:58.583412+0000","last_fullsized":"2024-03-23T14:56:58.583412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:58.583412+0000","last_change":"2024-03-23T14:56:58.583412+0000","last_active":"2024-03-23T14:56:58.583412+0000","last_peered":"2024-03-23T14:56:58.583412+0000","last_clean":"2024-03-23T14:56:58.583412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:58.583412+0000","last_undegraded":"2024-03-23T14:56:58.583412+0000","last_fullsized":"2024-03-23T14:56:58.583412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:57.575127+0000","last_change":"2024-03-23T14:56:57.575127+0000","last_active":"2024-03-23T14:56:57.575127+0000","last_peered":"2024-03-23T14:56:57.575127+0000","last_clean":"2024-03-23T14:56:57.575127+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:57.575127+0000","last_undegraded":"2024-03-23T14:56:57.575127+0000","last_fullsized":"2024-03-23T14:56:57.575127+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:58.583412+0000","last_change":"2024-03-23T14:56:58.583412+0000","last_active":"2024-03-23T14:56:58.583412+0000","last_peered":"2024-03-23T14:56:58.583412+0000","last_clean":"2024-03-23T14:56:58.583412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:58.583412+0000","last_undegraded":"2024-03-23T14:56:58.583412+0000","last_fullsized":"2024-03-23T14:56:58.583412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:57.575127+0000","last_change":"2024-03-23T14:56:57.575127+0000","last_active":"2024-03-23T14:56:57.575127+0000","last_peered":"2024-03-23T14:56:57.575127+0000","last_clean":"2024-03-23T14:56:57.575127+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:57.575127+0000","last_undegraded":"2024-03-23T14:56:57.575127+0000","last_fullsized":"2024-03-23T14:56:57.575127+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:57.575127+0000","last_change":"2024-03-23T14:56:57.575127+0000","last_active":"2024-03-23T14:56:57.575127+0000","last_peered":"2024-03-23T14:56:57.575127+0000","last_clean":"2024-03-23T14:56:57.575127+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:57.575127+0000","last_undegraded":"2024-03-23T14:56:57.575127+0000","last_fullsized":"2024-03-23T14:56:57.575127+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:58.583412+0000","last_change":"2024-03-23T14:56:58.583412+0000","last_active":"2024-03-23T14:56:58.583412+0000","last_peered":"2024-03-23T14:56:58.583412+0000","last_clean":"2024-03-23T14:56:58.583412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:58.583412+0000","last_undegraded":"2024-03-23T14:56:58.583412+0000","last_fullsized":"2024-03-23T14:56:58.583412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:57.575127+0000","last_change":"2024-03-23T14:56:57.575127+0000","last_active":"2024-03-23T14:56:57.575127+0000","last_peered":"2024-03-23T14:56:57.575127+0000","last_clean":"2024-03-23T14:56:57.575127+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:57.575127+0000","last_undegraded":"2024-03-23T14:56:57.575127+0000","last_fullsized":"2024-03-23T14:56:57.575127+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:58.583412+0000","last_change":"2024-03-23T14:56:58.583412+0000","last_active":"2024-03-23T14:56:58.583412+0000","last_peered":"2024-03-23T14:56:58.583412+0000","last_clean":"2024-03-23T14:56:58.583412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:58.583412+0000","last_undegraded":"2024-03-23T14:56:58.583412+0000","last_fullsized":"2024-03-23T14:56:58.583412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:57.575127+0000","last_change":"2024-03-23T14:56:57.575127+0000","last_active":"2024-03-23T14:56:57.575127+0000","last_peered":"2024-03-23T14:56:57.575127+0000","last_clean":"2024-03-23T14:56:57.575127+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:57.575127+0000","last_undegraded":"2024-03-23T14:56:57.575127+0000","last_fullsized":"2024-03-23T14:56:57.575127+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:58.583412+0000","last_change":"2024-03-23T14:56:58.583412+0000","last_active":"2024-03-23T14:56:58.583412+0000","last_peered":"2024-03-23T14:56:58.583412+0000","last_clean":"2024-03-23T14:56:58.583412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:58.583412+0000","last_undegraded":"2024-03-23T14:56:58.583412+0000","last_fullsized":"2024-03-23T14:56:58.583412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:57.575127+0000","last_change":"2024-03-23T14:56:57.575127+0000","last_active":"2024-03-23T14:56:57.575127+0000","last_peered":"2024-03-23T14:56:57.575127+0000","last_clean":"2024-03-23T14:56:57.575127+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:57.575127+0000","last_undegraded":"2024-03-23T14:56:57.575127+0000","last_fullsized":"2024-03-23T14:56:57.575127+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:58.583412+0000","last_change":"2024-03-23T14:56:58.583412+0000","last_active":"2024-03-23T14:56:58.583412+0000","last_peered":"2024-03-23T14:56:58.583412+0000","last_clean":"2024-03-23T14:56:58.583412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:58.583412+0000","last_undegraded":"2024-03-23T14:56:58.583412+0000","last_fullsized":"2024-03-23T14:56:58.583412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:57.575127+0000","last_change":"2024-03-23T14:56:57.575127+0000","last_active":"2024-03-23T14:56:57.575127+0000","last_peered":"2024-03-23T14:56:57.575127+0000","last_clean":"2024-03-23T14:56:57.575127+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:57.575127+0000","last_undegraded":"2024-03-23T14:56:57.575127+0000","last_fullsized":"2024-03-23T14:56:57.575127+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:58.583412+0000","last_change":"2024-03-23T14:56:58.583412+0000","last_active":"2024-03-23T14:56:58.583412+0000","last_peered":"2024-03-23T14:56:58.583412+0000","last_clean":"2024-03-23T14:56:58.583412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:58.583412+0000","last_undegraded":"2024-03-23T14:56:58.583412+0000","last_fullsized":"2024-03-23T14:56:58.583412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:57.575127+0000","last_change":"2024-03-23T14:56:57.575127+0000","last_active":"2024-03-23T14:56:57.575127+0000","last_peered":"2024-03-23T14:56:57.575127+0000","last_clean":"2024-03-23T14:56:57.575127+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:57.575127+0000","last_undegraded":"2024-03-23T14:56:57.575127+0000","last_fullsized":"2024-03-23T14:56:57.575127+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:58.583412+0000","last_change":"2024-03-23T14:56:58.583412+0000","last_active":"2024-03-23T14:56:58.583412+0000","last_peered":"2024-03-23T14:56:58.583412+0000","last_clean":"2024-03-23T14:56:58.583412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:58.583412+0000","last_undegraded":"2024-03-23T14:56:58.583412+0000","last_fullsized":"2024-03-23T14:56:58.583412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:57.575127+0000","last_change":"2024-03-23T14:56:57.575127+0000","last_active":"2024-03-23T14:56:57.575127+0000","last_peered":"2024-03-23T14:56:57.575127+0000","last_clean":"2024-03-23T14:56:57.575127+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:57.575127+0000","last_undegraded":"2024-03-23T14:56:57.575127+0000","last_fullsized":"2024-03-23T14:56:57.575127+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:58.583412+0000","last_change":"2024-03-23T14:56:58.583412+0000","last_active":"2024-03-23T14:56:58.583412+0000","last_peered":"2024-03-23T14:56:58.583412+0000","last_clean":"2024-03-23T14:56:58.583412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:58.583412+0000","last_undegraded":"2024-03-23T14:56:58.583412+0000","last_fullsized":"2024-03-23T14:56:58.583412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:57.575127+0000","last_change":"2024-03-23T14:56:57.575127+0000","last_active":"2024-03-23T14:56:57.575127+0000","last_peered":"2024-03-23T14:56:57.575127+0000","last_clean":"2024-03-23T14:56:57.575127+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:57.575127+0000","last_undegraded":"2024-03-23T14:56:57.575127+0000","last_fullsized":"2024-03-23T14:56:57.575127+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:58.583412+0000","last_change":"2024-03-23T14:56:58.583412+0000","last_active":"2024-03-23T14:56:58.583412+0000","last_peered":"2024-03-23T14:56:58.583412+0000","last_clean":"2024-03-23T14:56:58.583412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:58.583412+0000","last_undegraded":"2024-03-23T14:56:58.583412+0000","last_fullsized":"2024-03-23T14:56:58.583412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:57.575127+0000","last_change":"2024-03-23T14:56:57.575127+0000","last_active":"2024-03-23T14:56:57.575127+0000","last_peered":"2024-03-23T14:56:57.575127+0000","last_clean":"2024-03-23T14:56:57.575127+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:57.575127+0000","last_undegraded":"2024-03-23T14:56:57.575127+0000","last_fullsized":"2024-03-23T14:56:57.575127+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:58.583412+0000","last_change":"2024-03-23T14:56:58.583412+0000","last_active":"2024-03-23T14:56:58.583412+0000","last_peered":"2024-03-23T14:56:58.583412+0000","last_clean":"2024-03-23T14:56:58.583412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:58.583412+0000","last_undegraded":"2024-03-23T14:56:58.583412+0000","last_fullsized":"2024-03-23T14:56:58.583412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:57.575127+0000","last_change":"2024-03-23T14:56:57.575127+0000","last_active":"2024-03-23T14:56:57.575127+0000","last_peered":"2024-03-23T14:56:57.575127+0000","last_clean":"2024-03-23T14:56:57.575127+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:57.575127+0000","last_undegraded":"2024-03-23T14:56:57.575127+0000","last_fullsized":"2024-03-23T14:56:57.575127+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:58.583412+0000","last_change":"2024-03-23T14:56:58.583412+0000","last_active":"2024-03-23T14:56:58.583412+0000","last_peered":"2024-03-23T14:56:58.583412+0000","last_clean":"2024-03-23T14:56:58.583412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:58.583412+0000","last_undegraded":"2024-03-23T14:56:58.583412+0000","last_fullsized":"2024-03-23T14:56:58.583412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:57.575127+0000","last_change":"2024-03-23T14:56:57.575127+0000","last_active":"2024-03-23T14:56:57.575127+0000","last_peered":"2024-03-23T14:56:57.575127+0000","last_clean":"2024-03-23T14:56:57.575127+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:57.575127+0000","last_undegraded":"2024-03-23T14:56:57.575127+0000","last_fullsized":"2024-03-23T14:56:57.575127+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:58.583412+0000","last_change":"2024-03-23T14:56:58.583412+0000","last_active":"2024-03-23T14:56:58.583412+0000","last_peered":"2024-03-23T14:56:58.583412+0000","last_clean":"2024-03-23T14:56:58.583412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:58.583412+0000","last_undegraded":"2024-03-23T14:56:58.583412+0000","last_fullsized":"2024-03-23T14:56:58.583412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:57.575127+0000","last_change":"2024-03-23T14:56:57.575127+0000","last_active":"2024-03-23T14:56:57.575127+0000","last_peered":"2024-03-23T14:56:57.575127+0000","last_clean":"2024-03-23T14:56:57.575127+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:57.575127+0000","last_undegraded":"2024-03-23T14:56:57.575127+0000","last_fullsized":"2024-03-23T14:56:57.575127+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:58.583412+0000","last_change":"2024-03-23T14:56:58.583412+0000","last_active":"2024-03-23T14:56:58.583412+0000","last_peered":"2024-03-23T14:56:58.583412+0000","last_clean":"2024-03-23T14:56:58.583412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:58.583412+0000","last_undegraded":"2024-03-23T14:56:58.583412+0000","last_fullsized":"2024-03-23T14:56:58.583412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:57.575127+0000","last_change":"2024-03-23T14:56:57.575127+0000","last_active":"2024-03-23T14:56:57.575127+0000","last_peered":"2024-03-23T14:56:57.575127+0000","last_clean":"2024-03-23T14:56:57.575127+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:57.575127+0000","last_undegraded":"2024-03-23T14:56:57.575127+0000","last_fullsized":"2024-03-23T14:56:57.575127+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:58.583412+0000","last_change":"2024-03-23T14:56:58.583412+0000","last_active":"2024-03-23T14:56:58.583412+0000","last_peered":"2024-03-23T14:56:58.583412+0000","last_clean":"2024-03-23T14:56:58.583412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:58.583412+0000","last_undegraded":"2024-03-23T14:56:58.583412+0000","last_fullsized":"2024-03-23T14:56:58.583412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:57.575127+0000","last_change":"2024-03-23T14:56:57.575127+0000","last_active":"2024-03-23T14:56:57.575127+0000","last_peered":"2024-03-23T14:56:57.575127+0000","last_clean":"2024-03-23T14:56:57.575127+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:57.575127+0000","last_undegraded":"2024-03-23T14:56:57.575127+0000","last_fullsized":"2024-03-23T14:56:57.575127+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:58.583412+0000","last_change":"2024-03-23T14:56:58.583412+0000","last_active":"2024-03-23T14:56:58.583412+0000","last_peered":"2024-03-23T14:56:58.583412+0000","last_clean":"2024-03-23T14:56:58.583412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:58.583412+0000","last_undegraded":"2024-03-23T14:56:58.583412+0000","last_fullsized":"2024-03-23T14:56:58.583412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":162,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-03-23T14:56:50.310572+0000","last_change":"2024-03-23T14:56:48.722063+0000","last_active":"2024-03-23T14:56:50.310572+0000","last_peered":"2024-03-23T14:56:50.310572+0000","last_clean":"2024-03-23T14:56:50.310572+0000","last_became_active":"2024-03-23T14:56:48.720079+0000","last_became_peered":"2024-03-23T14:56:48.720079+0000","last_unstale":"2024-03-23T14:56:50.310572+0000","last_undegraded":"2024-03-23T14:56:50.310572+0000","last_fullsized":"2024-03-23T14:56:50.310572+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:47.478078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:47.478078+0000","last_clean_scrub_stamp":"2024-03-23T14:56:47.478078+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:55:51.231901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:57.575127+0000","last_change":"2024-03-23T14:56:57.575127+0000","last_active":"2024-03-23T14:56:57.575127+0000","last_peered":"2024-03-23T14:56:57.575127+0000","last_clean":"2024-03-23T14:56:57.575127+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:57.575127+0000","last_undegraded":"2024-03-23T14:56:57.575127+0000","last_fullsized":"2024-03-23T14:56:57.575127+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:58.583412+0000","last_change":"2024-03-23T14:56:58.583412+0000","last_active":"2024-03-23T14:56:58.583412+0000","last_peered":"2024-03-23T14:56:58.583412+0000","last_clean":"2024-03-23T14:56:58.583412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:58.583412+0000","last_undegraded":"2024-03-23T14:56:58.583412+0000","last_fullsized":"2024-03-23T14:56:58.583412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:57.575127+0000","last_change":"2024-03-23T14:56:57.575127+0000","last_active":"2024-03-23T14:56:57.575127+0000","last_peered":"2024-03-23T14:56:57.575127+0000","last_clean":"2024-03-23T14:56:57.575127+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:57.575127+0000","last_undegraded":"2024-03-23T14:56:57.575127+0000","last_fullsized":"2024-03-23T14:56:57.575127+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:58.583412+0000","last_change":"2024-03-23T14:56:58.583412+0000","last_active":"2024-03-23T14:56:58.583412+0000","last_peered":"2024-03-23T14:56:58.583412+0000","last_clean":"2024-03-23T14:56:58.583412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:58.583412+0000","last_undegraded":"2024-03-23T14:56:58.583412+0000","last_fullsized":"2024-03-23T14:56:58.583412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:57.575127+0000","last_change":"2024-03-23T14:56:57.575127+0000","last_active":"2024-03-23T14:56:57.575127+0000","last_peered":"2024-03-23T14:56:57.575127+0000","last_clean":"2024-03-23T14:56:57.575127+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:57.575127+0000","last_undegraded":"2024-03-23T14:56:57.575127+0000","last_fullsized":"2024-03-23T14:56:57.575127+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:58.583412+0000","last_change":"2024-03-23T14:56:58.583412+0000","last_active":"2024-03-23T14:56:58.583412+0000","last_peered":"2024-03-23T14:56:58.583412+0000","last_clean":"2024-03-23T14:56:58.583412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:58.583412+0000","last_undegraded":"2024-03-23T14:56:58.583412+0000","last_fullsized":"2024-03-23T14:56:58.583412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:57.575127+0000","last_change":"2024-03-23T14:56:57.575127+0000","last_active":"2024-03-23T14:56:57.575127+0000","last_peered":"2024-03-23T14:56:57.575127+0000","last_clean":"2024-03-23T14:56:57.575127+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:57.575127+0000","last_undegraded":"2024-03-23T14:56:57.575127+0000","last_fullsized":"2024-03-23T14:56:57.575127+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:58.583412+0000","last_change":"2024-03-23T14:56:58.583412+0000","last_active":"2024-03-23T14:56:58.583412+0000","last_peered":"2024-03-23T14:56:58.583412+0000","last_clean":"2024-03-23T14:56:58.583412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:58.583412+0000","last_undegraded":"2024-03-23T14:56:58.583412+0000","last_fullsized":"2024-03-23T14:56:58.583412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:57.575127+0000","last_change":"2024-03-23T14:56:57.575127+0000","last_active":"2024-03-23T14:56:57.575127+0000","last_peered":"2024-03-23T14:56:57.575127+0000","last_clean":"2024-03-23T14:56:57.575127+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:57.575127+0000","last_undegraded":"2024-03-23T14:56:57.575127+0000","last_fullsized":"2024-03-23T14:56:57.575127+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:58.583412+0000","last_change":"2024-03-23T14:56:58.583412+0000","last_active":"2024-03-23T14:56:58.583412+0000","last_peered":"2024-03-23T14:56:58.583412+0000","last_clean":"2024-03-23T14:56:58.583412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:58.583412+0000","last_undegraded":"2024-03-23T14:56:58.583412+0000","last_fullsized":"2024-03-23T14:56:58.583412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:57.575127+0000","last_change":"2024-03-23T14:56:57.575127+0000","last_active":"2024-03-23T14:56:57.575127+0000","last_peered":"2024-03-23T14:56:57.575127+0000","last_clean":"2024-03-23T14:56:57.575127+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:57.575127+0000","last_undegraded":"2024-03-23T14:56:57.575127+0000","last_fullsized":"2024-03-23T14:56:57.575127+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:58.583412+0000","last_change":"2024-03-23T14:56:58.583412+0000","last_active":"2024-03-23T14:56:58.583412+0000","last_peered":"2024-03-23T14:56:58.583412+0000","last_clean":"2024-03-23T14:56:58.583412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:58.583412+0000","last_undegraded":"2024-03-23T14:56:58.583412+0000","last_fullsized":"2024-03-23T14:56:58.583412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:57.575127+0000","last_change":"2024-03-23T14:56:57.575127+0000","last_active":"2024-03-23T14:56:57.575127+0000","last_peered":"2024-03-23T14:56:57.575127+0000","last_clean":"2024-03-23T14:56:57.575127+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:57.575127+0000","last_undegraded":"2024-03-23T14:56:57.575127+0000","last_fullsized":"2024-03-23T14:56:57.575127+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:58.583412+0000","last_change":"2024-03-23T14:56:58.583412+0000","last_active":"2024-03-23T14:56:58.583412+0000","last_peered":"2024-03-23T14:56:58.583412+0000","last_clean":"2024-03-23T14:56:58.583412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:58.583412+0000","last_undegraded":"2024-03-23T14:56:58.583412+0000","last_fullsized":"2024-03-23T14:56:58.583412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:57.575127+0000","last_change":"2024-03-23T14:56:57.575127+0000","last_active":"2024-03-23T14:56:57.575127+0000","last_peered":"2024-03-23T14:56:57.575127+0000","last_clean":"2024-03-23T14:56:57.575127+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:57.575127+0000","last_undegraded":"2024-03-23T14:56:57.575127+0000","last_fullsized":"2024-03-23T14:56:57.575127+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:58.583412+0000","last_change":"2024-03-23T14:56:58.583412+0000","last_active":"2024-03-23T14:56:58.583412+0000","last_peered":"2024-03-23T14:56:58.583412+0000","last_clean":"2024-03-23T14:56:58.583412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:58.583412+0000","last_undegraded":"2024-03-23T14:56:58.583412+0000","last_fullsized":"2024-03-23T14:56:58.583412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:57.575127+0000","last_change":"2024-03-23T14:56:57.575127+0000","last_active":"2024-03-23T14:56:57.575127+0000","last_peered":"2024-03-23T14:56:57.575127+0000","last_clean":"2024-03-23T14:56:57.575127+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:57.575127+0000","last_undegraded":"2024-03-23T14:56:57.575127+0000","last_fullsized":"2024-03-23T14:56:57.575127+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:58.583412+0000","last_change":"2024-03-23T14:56:58.583412+0000","last_active":"2024-03-23T14:56:58.583412+0000","last_peered":"2024-03-23T14:56:58.583412+0000","last_clean":"2024-03-23T14:56:58.583412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:58.583412+0000","last_undegraded":"2024-03-23T14:56:58.583412+0000","last_fullsized":"2024-03-23T14:56:58.583412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:57.575127+0000","last_change":"2024-03-23T14:56:57.575127+0000","last_active":"2024-03-23T14:56:57.575127+0000","last_peered":"2024-03-23T14:56:57.575127+0000","last_clean":"2024-03-23T14:56:57.575127+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:57.575127+0000","last_undegraded":"2024-03-23T14:56:57.575127+0000","last_fullsized":"2024-03-23T14:56:57.575127+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:58.583412+0000","last_change":"2024-03-23T14:56:58.583412+0000","last_active":"2024-03-23T14:56:58.583412+0000","last_peered":"2024-03-23T14:56:58.583412+0000","last_clean":"2024-03-23T14:56:58.583412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:58.583412+0000","last_undegraded":"2024-03-23T14:56:58.583412+0000","last_fullsized":"2024-03-23T14:56:58.583412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:57.575127+0000","last_change":"2024-03-23T14:56:57.575127+0000","last_active":"2024-03-23T14:56:57.575127+0000","last_peered":"2024-03-23T14:56:57.575127+0000","last_clean":"2024-03-23T14:56:57.575127+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:57.575127+0000","last_undegraded":"2024-03-23T14:56:57.575127+0000","last_fullsized":"2024-03-23T14:56:57.575127+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:58.583412+0000","last_change":"2024-03-23T14:56:58.583412+0000","last_active":"2024-03-23T14:56:58.583412+0000","last_peered":"2024-03-23T14:56:58.583412+0000","last_clean":"2024-03-23T14:56:58.583412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:58.583412+0000","last_undegraded":"2024-03-23T14:56:58.583412+0000","last_fullsized":"2024-03-23T14:56:58.583412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:57.575127+0000","last_change":"2024-03-23T14:56:57.575127+0000","last_active":"2024-03-23T14:56:57.575127+0000","last_peered":"2024-03-23T14:56:57.575127+0000","last_clean":"2024-03-23T14:56:57.575127+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:57.575127+0000","last_undegraded":"2024-03-23T14:56:57.575127+0000","last_fullsized":"2024-03-23T14:56:57.575127+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:58.583412+0000","last_change":"2024-03-23T14:56:58.583412+0000","last_active":"2024-03-23T14:56:58.583412+0000","last_peered":"2024-03-23T14:56:58.583412+0000","last_clean":"2024-03-23T14:56:58.583412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:58.583412+0000","last_undegraded":"2024-03-23T14:56:58.583412+0000","last_fullsized":"2024-03-23T14:56:58.583412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:58.583412+0000","last_change":"2024-03-23T14:56:58.583412+0000","last_active":"2024-03-23T14:56:58.583412+0000","last_peered":"2024-03-23T14:56:58.583412+0000","last_clean":"2024-03-23T14:56:58.583412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:58.583412+0000","last_undegraded":"2024-03-23T14:56:58.583412+0000","last_fullsized":"2024-03-23T14:56:58.583412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:57.575127+0000","last_change":"2024-03-23T14:56:57.575127+0000","last_active":"2024-03-23T14:56:57.575127+0000","last_peered":"2024-03-23T14:56:57.575127+0000","last_clean":"2024-03-23T14:56:57.575127+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:57.575127+0000","last_undegraded":"2024-03-23T14:56:57.575127+0000","last_fullsized":"2024-03-23T14:56:57.575127+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:58.583412+0000","last_change":"2024-03-23T14:56:58.583412+0000","last_active":"2024-03-23T14:56:58.583412+0000","last_peered":"2024-03-23T14:56:58.583412+0000","last_clean":"2024-03-23T14:56:58.583412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:58.583412+0000","last_undegraded":"2024-03-23T14:56:58.583412+0000","last_fullsized":"2024-03-23T14:56:58.583412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:57.575127+0000","last_change":"2024-03-23T14:56:57.575127+0000","last_active":"2024-03-23T14:56:57.575127+0000","last_peered":"2024-03-23T14:56:57.575127+0000","last_clean":"2024-03-23T14:56:57.575127+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:57.575127+0000","last_undegraded":"2024-03-23T14:56:57.575127+0000","last_fullsized":"2024-03-23T14:56:57.575127+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:58.583412+0000","last_change":"2024-03-23T14:56:58.583412+0000","last_active":"2024-03-23T14:56:58.583412+0000","last_peered":"2024-03-23T14:56:58.583412+0000","last_clean":"2024-03-23T14:56:58.583412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:58.583412+0000","last_undegraded":"2024-03-23T14:56:58.583412+0000","last_fullsized":"2024-03-23T14:56:58.583412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:57.575127+0000","last_change":"2024-03-23T14:56:57.575127+0000","last_active":"2024-03-23T14:56:57.575127+0000","last_peered":"2024-03-23T14:56:57.575127+0000","last_clean":"2024-03-23T14:56:57.575127+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:57.575127+0000","last_undegraded":"2024-03-23T14:56:57.575127+0000","last_fullsized":"2024-03-23T14:56:57.575127+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:58.583412+0000","last_change":"2024-03-23T14:56:58.583412+0000","last_active":"2024-03-23T14:56:58.583412+0000","last_peered":"2024-03-23T14:56:58.583412+0000","last_clean":"2024-03-23T14:56:58.583412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:58.583412+0000","last_undegraded":"2024-03-23T14:56:58.583412+0000","last_fullsized":"2024-03-23T14:56:58.583412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:57.575127+0000","last_change":"2024-03-23T14:56:57.575127+0000","last_active":"2024-03-23T14:56:57.575127+0000","last_peered":"2024-03-23T14:56:57.575127+0000","last_clean":"2024-03-23T14:56:57.575127+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:57.575127+0000","last_undegraded":"2024-03-23T14:56:57.575127+0000","last_fullsized":"2024-03-23T14:56:57.575127+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27876,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343964,"statfs":{"total":96636764160,"available":96608219136,"internally_reserved":0,"allocated":749568,"data_stored":636135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27172,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344668,"statfs":{"total":96636764160,"available":96608940032,"internally_reserved":0,"allocated":155648,"data_stored":45767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27172,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344668,"statfs":{"total":96636764160,"available":96608940032,"internally_reserved":0,"allocated":155648,"data_stored":45767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27180,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344660,"statfs":{"total":96636764160,"available":96608931840,"internally_reserved":0,"allocated":155648,"data_stored":45767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27180,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344660,"statfs":{"total":96636764160,"available":96608931840,"internally_reserved":0,"allocated":155648,"data_stored":45767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27172,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344668,"statfs":{"total":96636764160,"available":96608940032,"internally_reserved":0,"allocated":155648,"data_stored":45767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27180,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344660,"statfs":{"total":96636764160,"available":96608931840,"internally_reserved":0,"allocated":155648,"data_stored":45767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27876,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343964,"statfs":{"total":96636764160,"available":96608219136,"internally_reserved":0,"allocated":749568,"data_stored":636135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-23T14:57:02.623 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-23T14:57:02.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.778+0000 7f1a5cd72640 1 Processor -- start 2024-03-23T14:57:02.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.778+0000 7f1a5cd72640 1 -- start start 2024-03-23T14:57:02.783 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.778+0000 7f1a5cd72640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f1a580b4a00 0x7f1a580b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:02.783 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.778+0000 7f1a5cd72640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a5809e2d0 con 0x7f1a5809def0 2024-03-23T14:57:02.783 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.778+0000 7f1a5cd72640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a5806cb70 con 0x7f1a58147420 2024-03-23T14:57:02.783 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.778+0000 7f1a5cd72640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 -- 0x7f1a58146de0 con 0x7f1a580b4a00 2024-03-23T14:57:02.783 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.778+0000 7f1a5659c640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f1a5809def0 0x7f1a580ae620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:42368/0 (socket says 172.21.15.164:42368) 2024-03-23T14:57:02.783 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.778+0000 7f1a5659c640 1 -- 172.21.15.164:0/3840557797 learned_addr learned my addr 172.21.15.164:0/3840557797 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-03-23T14:57:02.783 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.778+0000 7f1a55d9b640 1 --2- 172.21.15.164:0/3840557797 >> v2:172.21.15.173:3300/0 conn(0x7f1a580b4a00 0x7f1a580b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:02.783 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.778+0000 7f1a5559a640 1 -- 172.21.15.164:0/3840557797 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2611555993 0 0) 0x7f1a5809e2d0 con 0x7f1a5809def0 2024-03-23T14:57:02.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.778+0000 7f1a5559a640 1 -- 172.21.15.164:0/3840557797 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1a40003530 con 0x7f1a5809def0 2024-03-23T14:57:02.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.778+0000 7f1a5559a640 1 -- 172.21.15.164:0/3840557797 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1811567475 0 0) 0x7f1a40003530 con 0x7f1a5809def0 2024-03-23T14:57:02.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.778+0000 7f1a5559a640 1 -- 172.21.15.164:0/3840557797 >> v2:172.21.15.173:3300/0 conn(0x7f1a580b4a00 msgr2=0x7f1a580b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:02.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.778+0000 7f1a5559a640 1 --2- 172.21.15.164:0/3840557797 >> v2:172.21.15.173:3300/0 conn(0x7f1a580b4a00 0x7f1a580b4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:02.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.778+0000 7f1a5559a640 1 -- 172.21.15.164:0/3840557797 >> v1:172.21.15.173:6789/0 conn(0x7f1a58147420 legacy=0x7f1a5814e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:02.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.778+0000 7f1a5559a640 1 -- 172.21.15.164:0/3840557797 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a5806ce50 con 0x7f1a5809def0 2024-03-23T14:57:02.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.778+0000 7f1a5559a640 1 -- 172.21.15.164:0/3840557797 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2526390548 0 0) 0x7f1a48001680 con 0x7f1a5809def0 2024-03-23T14:57:02.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.778+0000 7f1a5559a640 1 -- 172.21.15.164:0/3840557797 >> v1:172.21.15.164:6789/0 conn(0x7f1a5809def0 legacy=0x7f1a580ae620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:02.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.778+0000 7f1a5559a640 1 --2- 172.21.15.164:0/3840557797 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1a40003980 0x7f1a40003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:02.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.778+0000 7f1a5559a640 1 --2- 172.21.15.164:0/3840557797 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1a400047d0 0x7f1a40004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:02.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.778+0000 7f1a5559a640 1 --2- 172.21.15.164:0/3840557797 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1a40005190 0x7f1a4000caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:02.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.778+0000 7f1a5559a640 1 -- 172.21.15.164:0/3840557797 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1a4000d210 con 0x7f1a40003980 2024-03-23T14:57:02.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.778+0000 7f1a5559a640 1 -- 172.21.15.164:0/3840557797 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f1a4000d380 con 0x7f1a400047d0 2024-03-23T14:57:02.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.778+0000 7f1a5559a640 1 -- 172.21.15.164:0/3840557797 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f1a4000d540 con 0x7f1a40005190 2024-03-23T14:57:02.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.778+0000 7f1a5659c640 1 --2- 172.21.15.164:0/3840557797 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1a40003980 0x7f1a40003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:02.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.778+0000 7f1a55d9b640 1 --2- 172.21.15.164:0/3840557797 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1a400047d0 0x7f1a40004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:02.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.778+0000 7f1a56d9d640 1 --2- 172.21.15.164:0/3840557797 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1a40005190 0x7f1a4000caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:02.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.778+0000 7f1a5659c640 1 -- 172.21.15.164:0/3840557797 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1a40005190 msgr2=0x7f1a4000caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:02.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.778+0000 7f1a5659c640 1 --2- 172.21.15.164:0/3840557797 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1a40005190 0x7f1a4000caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:02.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.778+0000 7f1a5659c640 1 -- 172.21.15.164:0/3840557797 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1a400047d0 msgr2=0x7f1a40004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:02.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.778+0000 7f1a5659c640 1 --2- 172.21.15.164:0/3840557797 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1a400047d0 0x7f1a40004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:02.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.778+0000 7f1a5659c640 1 -- 172.21.15.164:0/3840557797 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a40003530 con 0x7f1a40003980 2024-03-23T14:57:02.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.778+0000 7f1a56d9d640 1 -- 172.21.15.164:0/3840557797 reap_dead start 2024-03-23T14:57:02.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.782+0000 7f1a5659c640 1 --2- 172.21.15.164:0/3840557797 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1a40003980 0x7f1a40003da0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1a4800c240 tx=0x7f1a4801cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41b11e381faaae2b server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:02.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.782+0000 7f1a5559a640 1 -- 172.21.15.164:0/3840557797 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a48005ae0 con 0x7f1a40003980 2024-03-23T14:57:02.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.782+0000 7f1a5559a640 1 -- 172.21.15.164:0/3840557797 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a48005c60 con 0x7f1a40003980 2024-03-23T14:57:02.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.782+0000 7f1a5559a640 1 -- 172.21.15.164:0/3840557797 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a48007070 con 0x7f1a40003980 2024-03-23T14:57:02.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.782+0000 7f1a5cd72640 1 -- 172.21.15.164:0/3840557797 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1a40003980 msgr2=0x7f1a40003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:02.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.782+0000 7f1a5cd72640 1 --2- 172.21.15.164:0/3840557797 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1a40003980 0x7f1a40003da0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1a4800c240 tx=0x7f1a4801cc60 comp rx=0 tx=0).stop 2024-03-23T14:57:02.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.782+0000 7f1a5cd72640 1 -- 172.21.15.164:0/3840557797 shutdown_connections 2024-03-23T14:57:02.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.782+0000 7f1a5cd72640 1 --2- 172.21.15.164:0/3840557797 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1a40003980 0x7f1a40003da0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:02.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.782+0000 7f1a5cd72640 1 -- 172.21.15.164:0/3840557797 >> 172.21.15.164:0/3840557797 conn(0x7f1a580b8ca0 msgr2=0x7f1a580b90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:02.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.782+0000 7f1a5cd72640 1 -- 172.21.15.164:0/3840557797 shutdown_connections 2024-03-23T14:57:02.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.782+0000 7f1a5cd72640 1 -- 172.21.15.164:0/3840557797 wait complete. 2024-03-23T14:57:02.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.782+0000 7f1a5cd72640 1 Processor -- start 2024-03-23T14:57:02.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.782+0000 7f1a5cd72640 1 -- start start 2024-03-23T14:57:02.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.782+0000 7f1a5cd72640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1a40003980 0x7f1a581d2d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:02.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.782+0000 7f1a5cd72640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1a5809ce80 0x7f1a581d1c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:02.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.782+0000 7f1a5cd72640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1a581d2170 0x7f1a581419e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:02.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.782+0000 7f1a5cd72640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1a5809d300 con 0x7f1a40003980 2024-03-23T14:57:02.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.782+0000 7f1a5cd72640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f1a5809d800 con 0x7f1a5809ce80 2024-03-23T14:57:02.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.782+0000 7f1a5cd72640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f1a581d1970 con 0x7f1a581d2170 2024-03-23T14:57:02.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.782+0000 7f1a5659c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1a40003980 0x7f1a581d2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:02.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.782+0000 7f1a5659c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1a40003980 0x7f1a581d2d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:58978/0 (socket says 172.21.15.164:58978) 2024-03-23T14:57:02.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.782+0000 7f1a5659c640 1 -- 172.21.15.164:0/3314787361 learned_addr learned my addr 172.21.15.164:0/3314787361 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:02.790 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.782+0000 7f1a55d9b640 1 --2- 172.21.15.164:0/3314787361 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1a5809ce80 0x7f1a581d1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:02.790 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.782+0000 7f1a56d9d640 1 --2- 172.21.15.164:0/3314787361 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1a581d2170 0x7f1a581419e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:02.790 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.782+0000 7f1a5659c640 1 -- 172.21.15.164:0/3314787361 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1a581d2170 msgr2=0x7f1a581419e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:02.790 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.782+0000 7f1a5659c640 1 --2- 172.21.15.164:0/3314787361 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1a581d2170 0x7f1a581419e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:02.790 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.782+0000 7f1a5659c640 1 -- 172.21.15.164:0/3314787361 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1a5809ce80 msgr2=0x7f1a581d1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:02.790 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.782+0000 7f1a5659c640 1 --2- 172.21.15.164:0/3314787361 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1a5809ce80 0x7f1a581d1c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:02.790 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.782+0000 7f1a5659c640 1 -- 172.21.15.164:0/3314787361 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a581d3400 con 0x7f1a40003980 2024-03-23T14:57:02.790 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.782+0000 7f1a56d9d640 1 --2- 172.21.15.164:0/3314787361 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1a581d2170 0x7f1a581419e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T14:57:02.790 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.782+0000 7f1a5659c640 1 --2- 172.21.15.164:0/3314787361 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1a40003980 0x7f1a581d2d80 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1a4801ca90 tx=0x7f1a40005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:02.790 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.782+0000 7f1a46ffd640 1 -- 172.21.15.164:0/3314787361 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a480b4070 con 0x7f1a40003980 2024-03-23T14:57:02.790 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.786+0000 7f1a46ffd640 1 -- 172.21.15.164:0/3314787361 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a48005e90 con 0x7f1a40003980 2024-03-23T14:57:02.790 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.786+0000 7f1a5cd72640 1 -- 172.21.15.164:0/3314787361 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a580b1cf0 con 0x7f1a40003980 2024-03-23T14:57:02.790 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.786+0000 7f1a46ffd640 1 -- 172.21.15.164:0/3314787361 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a48001680 con 0x7f1a40003980 2024-03-23T14:57:02.790 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.786+0000 7f1a5cd72640 1 -- 172.21.15.164:0/3314787361 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a581d25f0 con 0x7f1a40003980 2024-03-23T14:57:02.791 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.786+0000 7f1a5cd72640 1 -- 172.21.15.164:0/3314787361 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a581d17d0 con 0x7f1a40003980 2024-03-23T14:57:02.791 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.786+0000 7f1a46ffd640 1 -- 172.21.15.164:0/3314787361 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7f1a48007090 con 0x7f1a40003980 2024-03-23T14:57:02.791 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.786+0000 7f1a46ffd640 1 --2- 172.21.15.164:0/3314787361 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f1a380672d0 0x7f1a38069790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:02.791 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.786+0000 7f1a46ffd640 1 -- 172.21.15.164:0/3314787361 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f1a4811e8f0 con 0x7f1a40003980 2024-03-23T14:57:02.792 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.786+0000 7f1a55d9b640 1 --2- 172.21.15.164:0/3314787361 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f1a380672d0 0x7f1a38069790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:02.792 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.786+0000 7f1a55d9b640 1 --2- 172.21.15.164:0/3314787361 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f1a380672d0 0x7f1a38069790 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1a581e4000 tx=0x7f1a4c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:02.793 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.790+0000 7f1a46ffd640 1 -- 172.21.15.164:0/3314787361 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7f1a581d17d0 con 0x7f1a40003980 2024-03-23T14:57:02.943 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.938+0000 7f1a5cd72640 1 -- 172.21.15.164:0/3314787361 --> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1a581d17d0 con 0x7f1a380672d0 2024-03-23T14:57:02.949 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.946+0000 7f1a46ffd640 1 -- 172.21.15.164:0/3314787361 <== mgr.4107 v2:172.21.15.164:6832/2139087985 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+237079 (secure 0 0 0) 0x7f1a581d17d0 con 0x7f1a380672d0 2024-03-23T14:57:02.951 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-23T14:57:02.952 INFO:teuthology.orchestra.run.smithi164.stderr:dumped all 2024-03-23T14:57:02.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.950+0000 7f1a5cd72640 1 -- 172.21.15.164:0/3314787361 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f1a380672d0 msgr2=0x7f1a38069790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:02.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.950+0000 7f1a5cd72640 1 --2- 172.21.15.164:0/3314787361 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f1a380672d0 0x7f1a38069790 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1a581e4000 tx=0x7f1a4c008040 comp rx=0 tx=0).stop 2024-03-23T14:57:02.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.950+0000 7f1a5cd72640 1 -- 172.21.15.164:0/3314787361 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1a40003980 msgr2=0x7f1a581d2d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:02.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.950+0000 7f1a5cd72640 1 --2- 172.21.15.164:0/3314787361 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1a40003980 0x7f1a581d2d80 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1a4801ca90 tx=0x7f1a40005800 comp rx=0 tx=0).stop 2024-03-23T14:57:02.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.950+0000 7f1a5cd72640 1 -- 172.21.15.164:0/3314787361 shutdown_connections 2024-03-23T14:57:02.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.950+0000 7f1a5cd72640 1 --2- 172.21.15.164:0/3314787361 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1a581d2170 0x7f1a581419e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:02.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.950+0000 7f1a5cd72640 1 --2- 172.21.15.164:0/3314787361 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f1a380672d0 0x7f1a38069790 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:02.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.950+0000 7f1a5cd72640 1 --2- 172.21.15.164:0/3314787361 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1a5809ce80 0x7f1a581d1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:02.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.950+0000 7f1a5cd72640 1 --2- 172.21.15.164:0/3314787361 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1a40003980 0x7f1a581d2d80 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:02.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.950+0000 7f1a5cd72640 1 -- 172.21.15.164:0/3314787361 >> 172.21.15.164:0/3314787361 conn(0x7f1a580b8ca0 msgr2=0x7f1a580b80c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:02.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.950+0000 7f1a5cd72640 1 -- 172.21.15.164:0/3314787361 shutdown_connections 2024-03-23T14:57:02.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:02.950+0000 7f1a5cd72640 1 -- 172.21.15.164:0/3314787361 wait complete. 2024-03-23T14:57:02.968 INFO:teuthology.orchestra.run.smithi164.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2024-03-23T14:57:01.109719+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":218808,"kb_used_data":2376,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755912,"statfs":{"total":773094113280,"available":772870053888,"internally_reserved":0,"allocated":2433024,"data_stored":1546872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001307"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:58.583412+0000","last_change":"2024-03-23T14:56:58.583412+0000","last_active":"2024-03-23T14:56:58.583412+0000","last_peered":"2024-03-23T14:56:58.583412+0000","last_clean":"2024-03-23T14:56:58.583412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:58.583412+0000","last_undegraded":"2024-03-23T14:56:58.583412+0000","last_fullsized":"2024-03-23T14:56:58.583412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:58.583412+0000","last_change":"2024-03-23T14:56:58.583412+0000","last_active":"2024-03-23T14:56:58.583412+0000","last_peered":"2024-03-23T14:56:58.583412+0000","last_clean":"2024-03-23T14:56:58.583412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:58.583412+0000","last_undegraded":"2024-03-23T14:56:58.583412+0000","last_fullsized":"2024-03-23T14:56:58.583412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:58.583412+0000","last_change":"2024-03-23T14:56:58.583412+0000","last_active":"2024-03-23T14:56:58.583412+0000","last_peered":"2024-03-23T14:56:58.583412+0000","last_clean":"2024-03-23T14:56:58.583412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:58.583412+0000","last_undegraded":"2024-03-23T14:56:58.583412+0000","last_fullsized":"2024-03-23T14:56:58.583412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:58.583412+0000","last_change":"2024-03-23T14:56:58.583412+0000","last_active":"2024-03-23T14:56:58.583412+0000","last_peered":"2024-03-23T14:56:58.583412+0000","last_clean":"2024-03-23T14:56:58.583412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:58.583412+0000","last_undegraded":"2024-03-23T14:56:58.583412+0000","last_fullsized":"2024-03-23T14:56:58.583412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:58.583412+0000","last_change":"2024-03-23T14:56:58.583412+0000","last_active":"2024-03-23T14:56:58.583412+0000","last_peered":"2024-03-23T14:56:58.583412+0000","last_clean":"2024-03-23T14:56:58.583412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:58.583412+0000","last_undegraded":"2024-03-23T14:56:58.583412+0000","last_fullsized":"2024-03-23T14:56:58.583412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:58.583412+0000","last_change":"2024-03-23T14:56:58.583412+0000","last_active":"2024-03-23T14:56:58.583412+0000","last_peered":"2024-03-23T14:56:58.583412+0000","last_clean":"2024-03-23T14:56:58.583412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:58.583412+0000","last_undegraded":"2024-03-23T14:56:58.583412+0000","last_fullsized":"2024-03-23T14:56:58.583412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:58.583412+0000","last_change":"2024-03-23T14:56:58.583412+0000","last_active":"2024-03-23T14:56:58.583412+0000","last_peered":"2024-03-23T14:56:58.583412+0000","last_clean":"2024-03-23T14:56:58.583412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:58.583412+0000","last_undegraded":"2024-03-23T14:56:58.583412+0000","last_fullsized":"2024-03-23T14:56:58.583412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:58.583412+0000","last_change":"2024-03-23T14:56:58.583412+0000","last_active":"2024-03-23T14:56:58.583412+0000","last_peered":"2024-03-23T14:56:58.583412+0000","last_clean":"2024-03-23T14:56:58.583412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:58.583412+0000","last_undegraded":"2024-03-23T14:56:58.583412+0000","last_fullsized":"2024-03-23T14:56:58.583412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:58.583412+0000","last_change":"2024-03-23T14:56:58.583412+0000","last_active":"2024-03-23T14:56:58.583412+0000","last_peered":"2024-03-23T14:56:58.583412+0000","last_clean":"2024-03-23T14:56:58.583412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:58.583412+0000","last_undegraded":"2024-03-23T14:56:58.583412+0000","last_fullsized":"2024-03-23T14:56:58.583412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:58.583412+0000","last_change":"2024-03-23T14:56:58.583412+0000","last_active":"2024-03-23T14:56:58.583412+0000","last_peered":"2024-03-23T14:56:58.583412+0000","last_clean":"2024-03-23T14:56:58.583412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:58.583412+0000","last_undegraded":"2024-03-23T14:56:58.583412+0000","last_fullsized":"2024-03-23T14:56:58.583412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:58.583412+0000","last_change":"2024-03-23T14:56:58.583412+0000","last_active":"2024-03-23T14:56:58.583412+0000","last_peered":"2024-03-23T14:56:58.583412+0000","last_clean":"2024-03-23T14:56:58.583412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:58.583412+0000","last_undegraded":"2024-03-23T14:56:58.583412+0000","last_fullsized":"2024-03-23T14:56:58.583412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:58.583412+0000","last_change":"2024-03-23T14:56:58.583412+0000","last_active":"2024-03-23T14:56:58.583412+0000","last_peered":"2024-03-23T14:56:58.583412+0000","last_clean":"2024-03-23T14:56:58.583412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:58.583412+0000","last_undegraded":"2024-03-23T14:56:58.583412+0000","last_fullsized":"2024-03-23T14:56:58.583412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:58.583412+0000","last_change":"2024-03-23T14:56:58.583412+0000","last_active":"2024-03-23T14:56:58.583412+0000","last_peered":"2024-03-23T14:56:58.583412+0000","last_clean":"2024-03-23T14:56:58.583412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:58.583412+0000","last_undegraded":"2024-03-23T14:56:58.583412+0000","last_fullsized":"2024-03-23T14:56:58.583412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:58.583412+0000","last_change":"2024-03-23T14:56:58.583412+0000","last_active":"2024-03-23T14:56:58.583412+0000","last_peered":"2024-03-23T14:56:58.583412+0000","last_clean":"2024-03-23T14:56:58.583412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:58.583412+0000","last_undegraded":"2024-03-23T14:56:58.583412+0000","last_fullsized":"2024-03-23T14:56:58.583412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:58.583412+0000","last_change":"2024-03-23T14:56:58.583412+0000","last_active":"2024-03-23T14:56:58.583412+0000","last_peered":"2024-03-23T14:56:58.583412+0000","last_clean":"2024-03-23T14:56:58.583412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:58.583412+0000","last_undegraded":"2024-03-23T14:56:58.583412+0000","last_fullsized":"2024-03-23T14:56:58.583412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:58.583412+0000","last_change":"2024-03-23T14:56:58.583412+0000","last_active":"2024-03-23T14:56:58.583412+0000","last_peered":"2024-03-23T14:56:58.583412+0000","last_clean":"2024-03-23T14:56:58.583412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:58.583412+0000","last_undegraded":"2024-03-23T14:56:58.583412+0000","last_fullsized":"2024-03-23T14:56:58.583412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:58.583412+0000","last_change":"2024-03-23T14:56:58.583412+0000","last_active":"2024-03-23T14:56:58.583412+0000","last_peered":"2024-03-23T14:56:58.583412+0000","last_clean":"2024-03-23T14:56:58.583412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:58.583412+0000","last_undegraded":"2024-03-23T14:56:58.583412+0000","last_fullsized":"2024-03-23T14:56:58.583412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:58.583412+0000","last_change":"2024-03-23T14:56:58.583412+0000","last_active":"2024-03-23T14:56:58.583412+0000","last_peered":"2024-03-23T14:56:58.583412+0000","last_clean":"2024-03-23T14:56:58.583412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:58.583412+0000","last_undegraded":"2024-03-23T14:56:58.583412+0000","last_fullsized":"2024-03-23T14:56:58.583412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:58.583412+0000","last_change":"2024-03-23T14:56:58.583412+0000","last_active":"2024-03-23T14:56:58.583412+0000","last_peered":"2024-03-23T14:56:58.583412+0000","last_clean":"2024-03-23T14:56:58.583412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:58.583412+0000","last_undegraded":"2024-03-23T14:56:58.583412+0000","last_fullsized":"2024-03-23T14:56:58.583412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:58.583412+0000","last_change":"2024-03-23T14:56:58.583412+0000","last_active":"2024-03-23T14:56:58.583412+0000","last_peered":"2024-03-23T14:56:58.583412+0000","last_clean":"2024-03-23T14:56:58.583412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:58.583412+0000","last_undegraded":"2024-03-23T14:56:58.583412+0000","last_fullsized":"2024-03-23T14:56:58.583412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:58.583412+0000","last_change":"2024-03-23T14:56:58.583412+0000","last_active":"2024-03-23T14:56:58.583412+0000","last_peered":"2024-03-23T14:56:58.583412+0000","last_clean":"2024-03-23T14:56:58.583412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:58.583412+0000","last_undegraded":"2024-03-23T14:56:58.583412+0000","last_fullsized":"2024-03-23T14:56:58.583412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:58.583412+0000","last_change":"2024-03-23T14:56:58.583412+0000","last_active":"2024-03-23T14:56:58.583412+0000","last_peered":"2024-03-23T14:56:58.583412+0000","last_clean":"2024-03-23T14:56:58.583412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:58.583412+0000","last_undegraded":"2024-03-23T14:56:58.583412+0000","last_fullsized":"2024-03-23T14:56:58.583412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:58.583412+0000","last_change":"2024-03-23T14:56:58.583412+0000","last_active":"2024-03-23T14:56:58.583412+0000","last_peered":"2024-03-23T14:56:58.583412+0000","last_clean":"2024-03-23T14:56:58.583412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:58.583412+0000","last_undegraded":"2024-03-23T14:56:58.583412+0000","last_fullsized":"2024-03-23T14:56:58.583412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:58.583412+0000","last_change":"2024-03-23T14:56:58.583412+0000","last_active":"2024-03-23T14:56:58.583412+0000","last_peered":"2024-03-23T14:56:58.583412+0000","last_clean":"2024-03-23T14:56:58.583412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:58.583412+0000","last_undegraded":"2024-03-23T14:56:58.583412+0000","last_fullsized":"2024-03-23T14:56:58.583412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:58.583412+0000","last_change":"2024-03-23T14:56:58.583412+0000","last_active":"2024-03-23T14:56:58.583412+0000","last_peered":"2024-03-23T14:56:58.583412+0000","last_clean":"2024-03-23T14:56:58.583412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:58.583412+0000","last_undegraded":"2024-03-23T14:56:58.583412+0000","last_fullsized":"2024-03-23T14:56:58.583412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:58.583412+0000","last_change":"2024-03-23T14:56:58.583412+0000","last_active":"2024-03-23T14:56:58.583412+0000","last_peered":"2024-03-23T14:56:58.583412+0000","last_clean":"2024-03-23T14:56:58.583412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:58.583412+0000","last_undegraded":"2024-03-23T14:56:58.583412+0000","last_fullsized":"2024-03-23T14:56:58.583412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:58.583412+0000","last_change":"2024-03-23T14:56:58.583412+0000","last_active":"2024-03-23T14:56:58.583412+0000","last_peered":"2024-03-23T14:56:58.583412+0000","last_clean":"2024-03-23T14:56:58.583412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:58.583412+0000","last_undegraded":"2024-03-23T14:56:58.583412+0000","last_fullsized":"2024-03-23T14:56:58.583412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:58.583412+0000","last_change":"2024-03-23T14:56:58.583412+0000","last_active":"2024-03-23T14:56:58.583412+0000","last_peered":"2024-03-23T14:56:58.583412+0000","last_clean":"2024-03-23T14:56:58.583412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:58.583412+0000","last_undegraded":"2024-03-23T14:56:58.583412+0000","last_fullsized":"2024-03-23T14:56:58.583412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:58.583412+0000","last_change":"2024-03-23T14:56:58.583412+0000","last_active":"2024-03-23T14:56:58.583412+0000","last_peered":"2024-03-23T14:56:58.583412+0000","last_clean":"2024-03-23T14:56:58.583412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:58.583412+0000","last_undegraded":"2024-03-23T14:56:58.583412+0000","last_fullsized":"2024-03-23T14:56:58.583412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:58.583412+0000","last_change":"2024-03-23T14:56:58.583412+0000","last_active":"2024-03-23T14:56:58.583412+0000","last_peered":"2024-03-23T14:56:58.583412+0000","last_clean":"2024-03-23T14:56:58.583412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:58.583412+0000","last_undegraded":"2024-03-23T14:56:58.583412+0000","last_fullsized":"2024-03-23T14:56:58.583412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:58.583412+0000","last_change":"2024-03-23T14:56:58.583412+0000","last_active":"2024-03-23T14:56:58.583412+0000","last_peered":"2024-03-23T14:56:58.583412+0000","last_clean":"2024-03-23T14:56:58.583412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:58.583412+0000","last_undegraded":"2024-03-23T14:56:58.583412+0000","last_fullsized":"2024-03-23T14:56:58.583412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:58.583412+0000","last_change":"2024-03-23T14:56:58.583412+0000","last_active":"2024-03-23T14:56:58.583412+0000","last_peered":"2024-03-23T14:56:58.583412+0000","last_clean":"2024-03-23T14:56:58.583412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:58.583412+0000","last_undegraded":"2024-03-23T14:56:58.583412+0000","last_fullsized":"2024-03-23T14:56:58.583412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:58.583412+0000","last_change":"2024-03-23T14:56:58.583412+0000","last_active":"2024-03-23T14:56:58.583412+0000","last_peered":"2024-03-23T14:56:58.583412+0000","last_clean":"2024-03-23T14:56:58.583412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:58.583412+0000","last_undegraded":"2024-03-23T14:56:58.583412+0000","last_fullsized":"2024-03-23T14:56:58.583412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:57.575127+0000","last_change":"2024-03-23T14:56:57.575127+0000","last_active":"2024-03-23T14:56:57.575127+0000","last_peered":"2024-03-23T14:56:57.575127+0000","last_clean":"2024-03-23T14:56:57.575127+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:57.575127+0000","last_undegraded":"2024-03-23T14:56:57.575127+0000","last_fullsized":"2024-03-23T14:56:57.575127+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:58.583412+0000","last_change":"2024-03-23T14:56:58.583412+0000","last_active":"2024-03-23T14:56:58.583412+0000","last_peered":"2024-03-23T14:56:58.583412+0000","last_clean":"2024-03-23T14:56:58.583412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:58.583412+0000","last_undegraded":"2024-03-23T14:56:58.583412+0000","last_fullsized":"2024-03-23T14:56:58.583412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:57.575127+0000","last_change":"2024-03-23T14:56:57.575127+0000","last_active":"2024-03-23T14:56:57.575127+0000","last_peered":"2024-03-23T14:56:57.575127+0000","last_clean":"2024-03-23T14:56:57.575127+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:57.575127+0000","last_undegraded":"2024-03-23T14:56:57.575127+0000","last_fullsized":"2024-03-23T14:56:57.575127+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:57.575127+0000","last_change":"2024-03-23T14:56:57.575127+0000","last_active":"2024-03-23T14:56:57.575127+0000","last_peered":"2024-03-23T14:56:57.575127+0000","last_clean":"2024-03-23T14:56:57.575127+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:57.575127+0000","last_undegraded":"2024-03-23T14:56:57.575127+0000","last_fullsized":"2024-03-23T14:56:57.575127+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:58.583412+0000","last_change":"2024-03-23T14:56:58.583412+0000","last_active":"2024-03-23T14:56:58.583412+0000","last_peered":"2024-03-23T14:56:58.583412+0000","last_clean":"2024-03-23T14:56:58.583412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:58.583412+0000","last_undegraded":"2024-03-23T14:56:58.583412+0000","last_fullsized":"2024-03-23T14:56:58.583412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:57.575127+0000","last_change":"2024-03-23T14:56:57.575127+0000","last_active":"2024-03-23T14:56:57.575127+0000","last_peered":"2024-03-23T14:56:57.575127+0000","last_clean":"2024-03-23T14:56:57.575127+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:57.575127+0000","last_undegraded":"2024-03-23T14:56:57.575127+0000","last_fullsized":"2024-03-23T14:56:57.575127+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:58.583412+0000","last_change":"2024-03-23T14:56:58.583412+0000","last_active":"2024-03-23T14:56:58.583412+0000","last_peered":"2024-03-23T14:56:58.583412+0000","last_clean":"2024-03-23T14:56:58.583412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:58.583412+0000","last_undegraded":"2024-03-23T14:56:58.583412+0000","last_fullsized":"2024-03-23T14:56:58.583412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:57.575127+0000","last_change":"2024-03-23T14:56:57.575127+0000","last_active":"2024-03-23T14:56:57.575127+0000","last_peered":"2024-03-23T14:56:57.575127+0000","last_clean":"2024-03-23T14:56:57.575127+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:57.575127+0000","last_undegraded":"2024-03-23T14:56:57.575127+0000","last_fullsized":"2024-03-23T14:56:57.575127+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:58.583412+0000","last_change":"2024-03-23T14:56:58.583412+0000","last_active":"2024-03-23T14:56:58.583412+0000","last_peered":"2024-03-23T14:56:58.583412+0000","last_clean":"2024-03-23T14:56:58.583412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:58.583412+0000","last_undegraded":"2024-03-23T14:56:58.583412+0000","last_fullsized":"2024-03-23T14:56:58.583412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:57.575127+0000","last_change":"2024-03-23T14:56:57.575127+0000","last_active":"2024-03-23T14:56:57.575127+0000","last_peered":"2024-03-23T14:56:57.575127+0000","last_clean":"2024-03-23T14:56:57.575127+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:57.575127+0000","last_undegraded":"2024-03-23T14:56:57.575127+0000","last_fullsized":"2024-03-23T14:56:57.575127+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:58.583412+0000","last_change":"2024-03-23T14:56:58.583412+0000","last_active":"2024-03-23T14:56:58.583412+0000","last_peered":"2024-03-23T14:56:58.583412+0000","last_clean":"2024-03-23T14:56:58.583412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:58.583412+0000","last_undegraded":"2024-03-23T14:56:58.583412+0000","last_fullsized":"2024-03-23T14:56:58.583412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:57.575127+0000","last_change":"2024-03-23T14:56:57.575127+0000","last_active":"2024-03-23T14:56:57.575127+0000","last_peered":"2024-03-23T14:56:57.575127+0000","last_clean":"2024-03-23T14:56:57.575127+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:57.575127+0000","last_undegraded":"2024-03-23T14:56:57.575127+0000","last_fullsized":"2024-03-23T14:56:57.575127+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:58.583412+0000","last_change":"2024-03-23T14:56:58.583412+0000","last_active":"2024-03-23T14:56:58.583412+0000","last_peered":"2024-03-23T14:56:58.583412+0000","last_clean":"2024-03-23T14:56:58.583412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:58.583412+0000","last_undegraded":"2024-03-23T14:56:58.583412+0000","last_fullsized":"2024-03-23T14:56:58.583412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:57.575127+0000","last_change":"2024-03-23T14:56:57.575127+0000","last_active":"2024-03-23T14:56:57.575127+0000","last_peered":"2024-03-23T14:56:57.575127+0000","last_clean":"2024-03-23T14:56:57.575127+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:57.575127+0000","last_undegraded":"2024-03-23T14:56:57.575127+0000","last_fullsized":"2024-03-23T14:56:57.575127+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:58.583412+0000","last_change":"2024-03-23T14:56:58.583412+0000","last_active":"2024-03-23T14:56:58.583412+0000","last_peered":"2024-03-23T14:56:58.583412+0000","last_clean":"2024-03-23T14:56:58.583412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:58.583412+0000","last_undegraded":"2024-03-23T14:56:58.583412+0000","last_fullsized":"2024-03-23T14:56:58.583412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:57.575127+0000","last_change":"2024-03-23T14:56:57.575127+0000","last_active":"2024-03-23T14:56:57.575127+0000","last_peered":"2024-03-23T14:56:57.575127+0000","last_clean":"2024-03-23T14:56:57.575127+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:57.575127+0000","last_undegraded":"2024-03-23T14:56:57.575127+0000","last_fullsized":"2024-03-23T14:56:57.575127+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:58.583412+0000","last_change":"2024-03-23T14:56:58.583412+0000","last_active":"2024-03-23T14:56:58.583412+0000","last_peered":"2024-03-23T14:56:58.583412+0000","last_clean":"2024-03-23T14:56:58.583412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:58.583412+0000","last_undegraded":"2024-03-23T14:56:58.583412+0000","last_fullsized":"2024-03-23T14:56:58.583412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:57.575127+0000","last_change":"2024-03-23T14:56:57.575127+0000","last_active":"2024-03-23T14:56:57.575127+0000","last_peered":"2024-03-23T14:56:57.575127+0000","last_clean":"2024-03-23T14:56:57.575127+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:57.575127+0000","last_undegraded":"2024-03-23T14:56:57.575127+0000","last_fullsized":"2024-03-23T14:56:57.575127+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:58.583412+0000","last_change":"2024-03-23T14:56:58.583412+0000","last_active":"2024-03-23T14:56:58.583412+0000","last_peered":"2024-03-23T14:56:58.583412+0000","last_clean":"2024-03-23T14:56:58.583412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:58.583412+0000","last_undegraded":"2024-03-23T14:56:58.583412+0000","last_fullsized":"2024-03-23T14:56:58.583412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:57.575127+0000","last_change":"2024-03-23T14:56:57.575127+0000","last_active":"2024-03-23T14:56:57.575127+0000","last_peered":"2024-03-23T14:56:57.575127+0000","last_clean":"2024-03-23T14:56:57.575127+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:57.575127+0000","last_undegraded":"2024-03-23T14:56:57.575127+0000","last_fullsized":"2024-03-23T14:56:57.575127+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:58.583412+0000","last_change":"2024-03-23T14:56:58.583412+0000","last_active":"2024-03-23T14:56:58.583412+0000","last_peered":"2024-03-23T14:56:58.583412+0000","last_clean":"2024-03-23T14:56:58.583412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:58.583412+0000","last_undegraded":"2024-03-23T14:56:58.583412+0000","last_fullsized":"2024-03-23T14:56:58.583412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:57.575127+0000","last_change":"2024-03-23T14:56:57.575127+0000","last_active":"2024-03-23T14:56:57.575127+0000","last_peered":"2024-03-23T14:56:57.575127+0000","last_clean":"2024-03-23T14:56:57.575127+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:57.575127+0000","last_undegraded":"2024-03-23T14:56:57.575127+0000","last_fullsized":"2024-03-23T14:56:57.575127+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:58.583412+0000","last_change":"2024-03-23T14:56:58.583412+0000","last_active":"2024-03-23T14:56:58.583412+0000","last_peered":"2024-03-23T14:56:58.583412+0000","last_clean":"2024-03-23T14:56:58.583412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:58.583412+0000","last_undegraded":"2024-03-23T14:56:58.583412+0000","last_fullsized":"2024-03-23T14:56:58.583412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:57.575127+0000","last_change":"2024-03-23T14:56:57.575127+0000","last_active":"2024-03-23T14:56:57.575127+0000","last_peered":"2024-03-23T14:56:57.575127+0000","last_clean":"2024-03-23T14:56:57.575127+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:57.575127+0000","last_undegraded":"2024-03-23T14:56:57.575127+0000","last_fullsized":"2024-03-23T14:56:57.575127+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:58.583412+0000","last_change":"2024-03-23T14:56:58.583412+0000","last_active":"2024-03-23T14:56:58.583412+0000","last_peered":"2024-03-23T14:56:58.583412+0000","last_clean":"2024-03-23T14:56:58.583412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:58.583412+0000","last_undegraded":"2024-03-23T14:56:58.583412+0000","last_fullsized":"2024-03-23T14:56:58.583412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:57.575127+0000","last_change":"2024-03-23T14:56:57.575127+0000","last_active":"2024-03-23T14:56:57.575127+0000","last_peered":"2024-03-23T14:56:57.575127+0000","last_clean":"2024-03-23T14:56:57.575127+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:57.575127+0000","last_undegraded":"2024-03-23T14:56:57.575127+0000","last_fullsized":"2024-03-23T14:56:57.575127+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:58.583412+0000","last_change":"2024-03-23T14:56:58.583412+0000","last_active":"2024-03-23T14:56:58.583412+0000","last_peered":"2024-03-23T14:56:58.583412+0000","last_clean":"2024-03-23T14:56:58.583412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:58.583412+0000","last_undegraded":"2024-03-23T14:56:58.583412+0000","last_fullsized":"2024-03-23T14:56:58.583412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:57.575127+0000","last_change":"2024-03-23T14:56:57.575127+0000","last_active":"2024-03-23T14:56:57.575127+0000","last_peered":"2024-03-23T14:56:57.575127+0000","last_clean":"2024-03-23T14:56:57.575127+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:57.575127+0000","last_undegraded":"2024-03-23T14:56:57.575127+0000","last_fullsized":"2024-03-23T14:56:57.575127+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:58.583412+0000","last_change":"2024-03-23T14:56:58.583412+0000","last_active":"2024-03-23T14:56:58.583412+0000","last_peered":"2024-03-23T14:56:58.583412+0000","last_clean":"2024-03-23T14:56:58.583412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:58.583412+0000","last_undegraded":"2024-03-23T14:56:58.583412+0000","last_fullsized":"2024-03-23T14:56:58.583412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:57.575127+0000","last_change":"2024-03-23T14:56:57.575127+0000","last_active":"2024-03-23T14:56:57.575127+0000","last_peered":"2024-03-23T14:56:57.575127+0000","last_clean":"2024-03-23T14:56:57.575127+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:57.575127+0000","last_undegraded":"2024-03-23T14:56:57.575127+0000","last_fullsized":"2024-03-23T14:56:57.575127+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:58.583412+0000","last_change":"2024-03-23T14:56:58.583412+0000","last_active":"2024-03-23T14:56:58.583412+0000","last_peered":"2024-03-23T14:56:58.583412+0000","last_clean":"2024-03-23T14:56:58.583412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:58.583412+0000","last_undegraded":"2024-03-23T14:56:58.583412+0000","last_fullsized":"2024-03-23T14:56:58.583412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":162,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-03-23T14:56:50.310572+0000","last_change":"2024-03-23T14:56:48.722063+0000","last_active":"2024-03-23T14:56:50.310572+0000","last_peered":"2024-03-23T14:56:50.310572+0000","last_clean":"2024-03-23T14:56:50.310572+0000","last_became_active":"2024-03-23T14:56:48.720079+0000","last_became_peered":"2024-03-23T14:56:48.720079+0000","last_unstale":"2024-03-23T14:56:50.310572+0000","last_undegraded":"2024-03-23T14:56:50.310572+0000","last_fullsized":"2024-03-23T14:56:50.310572+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:47.478078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:47.478078+0000","last_clean_scrub_stamp":"2024-03-23T14:56:47.478078+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:55:51.231901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:57.575127+0000","last_change":"2024-03-23T14:56:57.575127+0000","last_active":"2024-03-23T14:56:57.575127+0000","last_peered":"2024-03-23T14:56:57.575127+0000","last_clean":"2024-03-23T14:56:57.575127+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:57.575127+0000","last_undegraded":"2024-03-23T14:56:57.575127+0000","last_fullsized":"2024-03-23T14:56:57.575127+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:58.583412+0000","last_change":"2024-03-23T14:56:58.583412+0000","last_active":"2024-03-23T14:56:58.583412+0000","last_peered":"2024-03-23T14:56:58.583412+0000","last_clean":"2024-03-23T14:56:58.583412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:58.583412+0000","last_undegraded":"2024-03-23T14:56:58.583412+0000","last_fullsized":"2024-03-23T14:56:58.583412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:57.575127+0000","last_change":"2024-03-23T14:56:57.575127+0000","last_active":"2024-03-23T14:56:57.575127+0000","last_peered":"2024-03-23T14:56:57.575127+0000","last_clean":"2024-03-23T14:56:57.575127+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:57.575127+0000","last_undegraded":"2024-03-23T14:56:57.575127+0000","last_fullsized":"2024-03-23T14:56:57.575127+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:58.583412+0000","last_change":"2024-03-23T14:56:58.583412+0000","last_active":"2024-03-23T14:56:58.583412+0000","last_peered":"2024-03-23T14:56:58.583412+0000","last_clean":"2024-03-23T14:56:58.583412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:58.583412+0000","last_undegraded":"2024-03-23T14:56:58.583412+0000","last_fullsized":"2024-03-23T14:56:58.583412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:57.575127+0000","last_change":"2024-03-23T14:56:57.575127+0000","last_active":"2024-03-23T14:56:57.575127+0000","last_peered":"2024-03-23T14:56:57.575127+0000","last_clean":"2024-03-23T14:56:57.575127+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:57.575127+0000","last_undegraded":"2024-03-23T14:56:57.575127+0000","last_fullsized":"2024-03-23T14:56:57.575127+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:58.583412+0000","last_change":"2024-03-23T14:56:58.583412+0000","last_active":"2024-03-23T14:56:58.583412+0000","last_peered":"2024-03-23T14:56:58.583412+0000","last_clean":"2024-03-23T14:56:58.583412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:58.583412+0000","last_undegraded":"2024-03-23T14:56:58.583412+0000","last_fullsized":"2024-03-23T14:56:58.583412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:57.575127+0000","last_change":"2024-03-23T14:56:57.575127+0000","last_active":"2024-03-23T14:56:57.575127+0000","last_peered":"2024-03-23T14:56:57.575127+0000","last_clean":"2024-03-23T14:56:57.575127+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:57.575127+0000","last_undegraded":"2024-03-23T14:56:57.575127+0000","last_fullsized":"2024-03-23T14:56:57.575127+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:58.583412+0000","last_change":"2024-03-23T14:56:58.583412+0000","last_active":"2024-03-23T14:56:58.583412+0000","last_peered":"2024-03-23T14:56:58.583412+0000","last_clean":"2024-03-23T14:56:58.583412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:58.583412+0000","last_undegraded":"2024-03-23T14:56:58.583412+0000","last_fullsized":"2024-03-23T14:56:58.583412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:57.575127+0000","last_change":"2024-03-23T14:56:57.575127+0000","last_active":"2024-03-23T14:56:57.575127+0000","last_peered":"2024-03-23T14:56:57.575127+0000","last_clean":"2024-03-23T14:56:57.575127+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:57.575127+0000","last_undegraded":"2024-03-23T14:56:57.575127+0000","last_fullsized":"2024-03-23T14:56:57.575127+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:58.583412+0000","last_change":"2024-03-23T14:56:58.583412+0000","last_active":"2024-03-23T14:56:58.583412+0000","last_peered":"2024-03-23T14:56:58.583412+0000","last_clean":"2024-03-23T14:56:58.583412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:58.583412+0000","last_undegraded":"2024-03-23T14:56:58.583412+0000","last_fullsized":"2024-03-23T14:56:58.583412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:57.575127+0000","last_change":"2024-03-23T14:56:57.575127+0000","last_active":"2024-03-23T14:56:57.575127+0000","last_peered":"2024-03-23T14:56:57.575127+0000","last_clean":"2024-03-23T14:56:57.575127+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:57.575127+0000","last_undegraded":"2024-03-23T14:56:57.575127+0000","last_fullsized":"2024-03-23T14:56:57.575127+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:58.583412+0000","last_change":"2024-03-23T14:56:58.583412+0000","last_active":"2024-03-23T14:56:58.583412+0000","last_peered":"2024-03-23T14:56:58.583412+0000","last_clean":"2024-03-23T14:56:58.583412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:58.583412+0000","last_undegraded":"2024-03-23T14:56:58.583412+0000","last_fullsized":"2024-03-23T14:56:58.583412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:57.575127+0000","last_change":"2024-03-23T14:56:57.575127+0000","last_active":"2024-03-23T14:56:57.575127+0000","last_peered":"2024-03-23T14:56:57.575127+0000","last_clean":"2024-03-23T14:56:57.575127+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:57.575127+0000","last_undegraded":"2024-03-23T14:56:57.575127+0000","last_fullsized":"2024-03-23T14:56:57.575127+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:58.583412+0000","last_change":"2024-03-23T14:56:58.583412+0000","last_active":"2024-03-23T14:56:58.583412+0000","last_peered":"2024-03-23T14:56:58.583412+0000","last_clean":"2024-03-23T14:56:58.583412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:58.583412+0000","last_undegraded":"2024-03-23T14:56:58.583412+0000","last_fullsized":"2024-03-23T14:56:58.583412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:57.575127+0000","last_change":"2024-03-23T14:56:57.575127+0000","last_active":"2024-03-23T14:56:57.575127+0000","last_peered":"2024-03-23T14:56:57.575127+0000","last_clean":"2024-03-23T14:56:57.575127+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:57.575127+0000","last_undegraded":"2024-03-23T14:56:57.575127+0000","last_fullsized":"2024-03-23T14:56:57.575127+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:58.583412+0000","last_change":"2024-03-23T14:56:58.583412+0000","last_active":"2024-03-23T14:56:58.583412+0000","last_peered":"2024-03-23T14:56:58.583412+0000","last_clean":"2024-03-23T14:56:58.583412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:58.583412+0000","last_undegraded":"2024-03-23T14:56:58.583412+0000","last_fullsized":"2024-03-23T14:56:58.583412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:57.575127+0000","last_change":"2024-03-23T14:56:57.575127+0000","last_active":"2024-03-23T14:56:57.575127+0000","last_peered":"2024-03-23T14:56:57.575127+0000","last_clean":"2024-03-23T14:56:57.575127+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:57.575127+0000","last_undegraded":"2024-03-23T14:56:57.575127+0000","last_fullsized":"2024-03-23T14:56:57.575127+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:58.583412+0000","last_change":"2024-03-23T14:56:58.583412+0000","last_active":"2024-03-23T14:56:58.583412+0000","last_peered":"2024-03-23T14:56:58.583412+0000","last_clean":"2024-03-23T14:56:58.583412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:58.583412+0000","last_undegraded":"2024-03-23T14:56:58.583412+0000","last_fullsized":"2024-03-23T14:56:58.583412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:57.575127+0000","last_change":"2024-03-23T14:56:57.575127+0000","last_active":"2024-03-23T14:56:57.575127+0000","last_peered":"2024-03-23T14:56:57.575127+0000","last_clean":"2024-03-23T14:56:57.575127+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:57.575127+0000","last_undegraded":"2024-03-23T14:56:57.575127+0000","last_fullsized":"2024-03-23T14:56:57.575127+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:58.583412+0000","last_change":"2024-03-23T14:56:58.583412+0000","last_active":"2024-03-23T14:56:58.583412+0000","last_peered":"2024-03-23T14:56:58.583412+0000","last_clean":"2024-03-23T14:56:58.583412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:58.583412+0000","last_undegraded":"2024-03-23T14:56:58.583412+0000","last_fullsized":"2024-03-23T14:56:58.583412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:57.575127+0000","last_change":"2024-03-23T14:56:57.575127+0000","last_active":"2024-03-23T14:56:57.575127+0000","last_peered":"2024-03-23T14:56:57.575127+0000","last_clean":"2024-03-23T14:56:57.575127+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:57.575127+0000","last_undegraded":"2024-03-23T14:56:57.575127+0000","last_fullsized":"2024-03-23T14:56:57.575127+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:58.583412+0000","last_change":"2024-03-23T14:56:58.583412+0000","last_active":"2024-03-23T14:56:58.583412+0000","last_peered":"2024-03-23T14:56:58.583412+0000","last_clean":"2024-03-23T14:56:58.583412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:58.583412+0000","last_undegraded":"2024-03-23T14:56:58.583412+0000","last_fullsized":"2024-03-23T14:56:58.583412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:57.575127+0000","last_change":"2024-03-23T14:56:57.575127+0000","last_active":"2024-03-23T14:56:57.575127+0000","last_peered":"2024-03-23T14:56:57.575127+0000","last_clean":"2024-03-23T14:56:57.575127+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:57.575127+0000","last_undegraded":"2024-03-23T14:56:57.575127+0000","last_fullsized":"2024-03-23T14:56:57.575127+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:58.583412+0000","last_change":"2024-03-23T14:56:58.583412+0000","last_active":"2024-03-23T14:56:58.583412+0000","last_peered":"2024-03-23T14:56:58.583412+0000","last_clean":"2024-03-23T14:56:58.583412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:58.583412+0000","last_undegraded":"2024-03-23T14:56:58.583412+0000","last_fullsized":"2024-03-23T14:56:58.583412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:58.583412+0000","last_change":"2024-03-23T14:56:58.583412+0000","last_active":"2024-03-23T14:56:58.583412+0000","last_peered":"2024-03-23T14:56:58.583412+0000","last_clean":"2024-03-23T14:56:58.583412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:58.583412+0000","last_undegraded":"2024-03-23T14:56:58.583412+0000","last_fullsized":"2024-03-23T14:56:58.583412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:57.575127+0000","last_change":"2024-03-23T14:56:57.575127+0000","last_active":"2024-03-23T14:56:57.575127+0000","last_peered":"2024-03-23T14:56:57.575127+0000","last_clean":"2024-03-23T14:56:57.575127+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:57.575127+0000","last_undegraded":"2024-03-23T14:56:57.575127+0000","last_fullsized":"2024-03-23T14:56:57.575127+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:58.583412+0000","last_change":"2024-03-23T14:56:58.583412+0000","last_active":"2024-03-23T14:56:58.583412+0000","last_peered":"2024-03-23T14:56:58.583412+0000","last_clean":"2024-03-23T14:56:58.583412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:58.583412+0000","last_undegraded":"2024-03-23T14:56:58.583412+0000","last_fullsized":"2024-03-23T14:56:58.583412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:57.575127+0000","last_change":"2024-03-23T14:56:57.575127+0000","last_active":"2024-03-23T14:56:57.575127+0000","last_peered":"2024-03-23T14:56:57.575127+0000","last_clean":"2024-03-23T14:56:57.575127+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:57.575127+0000","last_undegraded":"2024-03-23T14:56:57.575127+0000","last_fullsized":"2024-03-23T14:56:57.575127+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:58.583412+0000","last_change":"2024-03-23T14:56:58.583412+0000","last_active":"2024-03-23T14:56:58.583412+0000","last_peered":"2024-03-23T14:56:58.583412+0000","last_clean":"2024-03-23T14:56:58.583412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:58.583412+0000","last_undegraded":"2024-03-23T14:56:58.583412+0000","last_fullsized":"2024-03-23T14:56:58.583412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:57.575127+0000","last_change":"2024-03-23T14:56:57.575127+0000","last_active":"2024-03-23T14:56:57.575127+0000","last_peered":"2024-03-23T14:56:57.575127+0000","last_clean":"2024-03-23T14:56:57.575127+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:57.575127+0000","last_undegraded":"2024-03-23T14:56:57.575127+0000","last_fullsized":"2024-03-23T14:56:57.575127+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:58.583412+0000","last_change":"2024-03-23T14:56:58.583412+0000","last_active":"2024-03-23T14:56:58.583412+0000","last_peered":"2024-03-23T14:56:58.583412+0000","last_clean":"2024-03-23T14:56:58.583412+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:58.583412+0000","last_undegraded":"2024-03-23T14:56:58.583412+0000","last_fullsized":"2024-03-23T14:56:58.583412+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-23T14:56:57.575127+0000","last_change":"2024-03-23T14:56:57.575127+0000","last_active":"2024-03-23T14:56:57.575127+0000","last_peered":"2024-03-23T14:56:57.575127+0000","last_clean":"2024-03-23T14:56:57.575127+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-23T14:56:57.575127+0000","last_undegraded":"2024-03-23T14:56:57.575127+0000","last_fullsized":"2024-03-23T14:56:57.575127+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27876,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343964,"statfs":{"total":96636764160,"available":96608219136,"internally_reserved":0,"allocated":749568,"data_stored":636135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27172,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344668,"statfs":{"total":96636764160,"available":96608940032,"internally_reserved":0,"allocated":155648,"data_stored":45767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27172,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344668,"statfs":{"total":96636764160,"available":96608940032,"internally_reserved":0,"allocated":155648,"data_stored":45767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27180,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344660,"statfs":{"total":96636764160,"available":96608931840,"internally_reserved":0,"allocated":155648,"data_stored":45767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27180,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344660,"statfs":{"total":96636764160,"available":96608931840,"internally_reserved":0,"allocated":155648,"data_stored":45767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27172,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344668,"statfs":{"total":96636764160,"available":96608940032,"internally_reserved":0,"allocated":155648,"data_stored":45767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27180,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344660,"statfs":{"total":96636764160,"available":96608931840,"internally_reserved":0,"allocated":155648,"data_stored":45767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27876,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343964,"statfs":{"total":96636764160,"available":96608219136,"internally_reserved":0,"allocated":749568,"data_stored":636135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-23T14:57:02.972 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-23T14:57:03.127 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.122+0000 7f2af280c640 1 Processor -- start 2024-03-23T14:57:03.127 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.122+0000 7f2af280c640 1 -- start start 2024-03-23T14:57:03.130 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.122+0000 7f2af280c640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2aec148cf0 0x7f2aec149180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:03.130 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.122+0000 7f2af280c640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f2aec14db70 0x7f2aec14ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:03.130 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.122+0000 7f2af280c640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2aec0b51d0 con 0x7f2aec0b0230 2024-03-23T14:57:03.130 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.126+0000 7f2af280c640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 -- 0x7f2aec0b0610 con 0x7f2aec14db70 2024-03-23T14:57:03.130 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.126+0000 7f2af280c640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f2aec0b1500 con 0x7f2aec148cf0 2024-03-23T14:57:03.130 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.126+0000 7f2aebfff640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f2aec0b0230 0x7f2aec0b34d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:42372/0 (socket says 172.21.15.164:42372) 2024-03-23T14:57:03.130 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.126+0000 7f2aebfff640 1 -- 172.21.15.164:0/3632231705 learned_addr learned my addr 172.21.15.164:0/3632231705 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-03-23T14:57:03.130 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.126+0000 7f2aeb7fe640 1 --2- 172.21.15.164:0/3632231705 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2aec148cf0 0x7f2aec149180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:03.130 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.126+0000 7f2af0da9640 1 --2- 172.21.15.164:0/3632231705 >> v2:172.21.15.173:3300/0 conn(0x7f2aec14db70 0x7f2aec14ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:03.130 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.126+0000 7f2aeaffd640 1 -- 172.21.15.164:0/3632231705 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1941369011 0 0) 0x7f2aec0b51d0 con 0x7f2aec0b0230 2024-03-23T14:57:03.130 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.126+0000 7f2aeaffd640 1 -- 172.21.15.164:0/3632231705 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2ad8003530 con 0x7f2aec0b0230 2024-03-23T14:57:03.130 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.126+0000 7f2aeaffd640 1 -- 172.21.15.164:0/3632231705 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2002827246 0 0) 0x7f2ad8003530 con 0x7f2aec0b0230 2024-03-23T14:57:03.130 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.126+0000 7f2aeaffd640 1 -- 172.21.15.164:0/3632231705 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2aec148cf0 msgr2=0x7f2aec149180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:03.131 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.126+0000 7f2aeaffd640 1 --2- 172.21.15.164:0/3632231705 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2aec148cf0 0x7f2aec149180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:03.131 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.126+0000 7f2aeaffd640 1 -- 172.21.15.164:0/3632231705 >> v2:172.21.15.173:3300/0 conn(0x7f2aec14db70 msgr2=0x7f2aec14ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:03.131 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.126+0000 7f2aeaffd640 1 --2- 172.21.15.164:0/3632231705 >> v2:172.21.15.173:3300/0 conn(0x7f2aec14db70 0x7f2aec14ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:03.131 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.126+0000 7f2aeaffd640 1 -- 172.21.15.164:0/3632231705 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2aec06d250 con 0x7f2aec0b0230 2024-03-23T14:57:03.131 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.126+0000 7f2aeaffd640 1 -- 172.21.15.164:0/3632231705 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2526390548 0 0) 0x7f2ae4001a00 con 0x7f2aec0b0230 2024-03-23T14:57:03.131 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.126+0000 7f2aeaffd640 1 -- 172.21.15.164:0/3632231705 >> v1:172.21.15.164:6789/0 conn(0x7f2aec0b0230 legacy=0x7f2aec0b34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:03.131 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.126+0000 7f2aeaffd640 1 --2- 172.21.15.164:0/3632231705 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2ad8003980 0x7f2ad8003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:03.131 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.126+0000 7f2aeaffd640 1 --2- 172.21.15.164:0/3632231705 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2ad80047d0 0x7f2ad8004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:03.131 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.126+0000 7f2aeaffd640 1 --2- 172.21.15.164:0/3632231705 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2aec148cf0 0x7f2ad8005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:03.131 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.126+0000 7f2aeaffd640 1 -- 172.21.15.164:0/3632231705 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2aec0b0610 con 0x7f2ad8003980 2024-03-23T14:57:03.131 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.126+0000 7f2aeaffd640 1 -- 172.21.15.164:0/3632231705 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f2aec0b1500 con 0x7f2ad80047d0 2024-03-23T14:57:03.131 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.126+0000 7f2aeaffd640 1 -- 172.21.15.164:0/3632231705 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f2ad800b9c0 con 0x7f2aec148cf0 2024-03-23T14:57:03.131 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.126+0000 7f2aebfff640 1 --2- 172.21.15.164:0/3632231705 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2ad8003980 0x7f2ad8003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:03.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.126+0000 7f2aeb7fe640 1 --2- 172.21.15.164:0/3632231705 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2ad80047d0 0x7f2ad8004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:03.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.126+0000 7f2af0da9640 1 --2- 172.21.15.164:0/3632231705 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2aec148cf0 0x7f2ad8005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:03.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.126+0000 7f2aebfff640 1 -- 172.21.15.164:0/3632231705 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2aec148cf0 msgr2=0x7f2ad8005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:03.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.126+0000 7f2aebfff640 1 --2- 172.21.15.164:0/3632231705 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2aec148cf0 0x7f2ad8005190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:03.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.126+0000 7f2aebfff640 1 -- 172.21.15.164:0/3632231705 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2ad80047d0 msgr2=0x7f2ad8004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:03.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.126+0000 7f2aebfff640 1 --2- 172.21.15.164:0/3632231705 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2ad80047d0 0x7f2ad8004c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:03.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.126+0000 7f2aebfff640 1 -- 172.21.15.164:0/3632231705 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ad8003530 con 0x7f2ad8003980 2024-03-23T14:57:03.136 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.126+0000 7f2aebfff640 1 --2- 172.21.15.164:0/3632231705 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2ad8003980 0x7f2ad8003da0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2ae400a8d0 tx=0x7f2ae401d340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee99d84032250961 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:03.136 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.126+0000 7f2aeaffd640 1 -- 172.21.15.164:0/3632231705 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ae40ad630 con 0x7f2ad8003980 2024-03-23T14:57:03.136 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.126+0000 7f2aeaffd640 1 -- 172.21.15.164:0/3632231705 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ae40a8070 con 0x7f2ad8003980 2024-03-23T14:57:03.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.126+0000 7f2aeaffd640 1 -- 172.21.15.164:0/3632231705 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ae40ad9d0 con 0x7f2ad8003980 2024-03-23T14:57:03.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.130+0000 7f2af280c640 1 -- 172.21.15.164:0/3632231705 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2ad8003980 msgr2=0x7f2ad8003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:03.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.130+0000 7f2af280c640 1 --2- 172.21.15.164:0/3632231705 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2ad8003980 0x7f2ad8003da0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2ae400a8d0 tx=0x7f2ae401d340 comp rx=0 tx=0).stop 2024-03-23T14:57:03.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.130+0000 7f2af0da9640 1 -- 172.21.15.164:0/3632231705 reap_dead start 2024-03-23T14:57:03.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.130+0000 7f2af280c640 1 -- 172.21.15.164:0/3632231705 shutdown_connections 2024-03-23T14:57:03.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.130+0000 7f2af280c640 1 -- 172.21.15.164:0/3632231705 >> 172.21.15.164:0/3632231705 conn(0x7f2aec0b8ca0 msgr2=0x7f2aec0b90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:03.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.130+0000 7f2af280c640 1 -- 172.21.15.164:0/3632231705 shutdown_connections 2024-03-23T14:57:03.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.130+0000 7f2af280c640 1 -- 172.21.15.164:0/3632231705 wait complete. 2024-03-23T14:57:03.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.130+0000 7f2af280c640 1 Processor -- start 2024-03-23T14:57:03.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.130+0000 7f2af280c640 1 -- start start 2024-03-23T14:57:03.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.130+0000 7f2af280c640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2ad8003980 0x7f2aec1d3340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:03.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.130+0000 7f2af280c640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2aec1e1170 0x7f2aec141000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:03.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.130+0000 7f2af280c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2aec142a70 0x7f2aec141540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:03.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.130+0000 7f2af280c640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2aec1e1c10 con 0x7f2aec142a70 2024-03-23T14:57:03.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.130+0000 7f2af280c640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f2aec140220 con 0x7f2ad8003980 2024-03-23T14:57:03.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.130+0000 7f2af280c640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f2aec1426d0 con 0x7f2aec1e1170 2024-03-23T14:57:03.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.130+0000 7f2af0da9640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2aec142a70 0x7f2aec141540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:03.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.130+0000 7f2af0da9640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2aec142a70 0x7f2aec141540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:58994/0 (socket says 172.21.15.164:58994) 2024-03-23T14:57:03.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.130+0000 7f2af0da9640 1 -- 172.21.15.164:0/3551625982 learned_addr learned my addr 172.21.15.164:0/3551625982 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:03.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.130+0000 7f2aeb7fe640 1 --2- 172.21.15.164:0/3551625982 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2aec1e1170 0x7f2aec141000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:03.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.130+0000 7f2aebfff640 1 --2- 172.21.15.164:0/3551625982 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2ad8003980 0x7f2aec1d3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:03.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.130+0000 7f2af0da9640 1 -- 172.21.15.164:0/3551625982 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2aec1e1170 msgr2=0x7f2aec141000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:03.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.130+0000 7f2af0da9640 1 --2- 172.21.15.164:0/3551625982 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2aec1e1170 0x7f2aec141000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:03.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.130+0000 7f2af0da9640 1 -- 172.21.15.164:0/3551625982 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2ad8003980 msgr2=0x7f2aec1d3340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:03.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.130+0000 7f2af0da9640 1 --2- 172.21.15.164:0/3551625982 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2ad8003980 0x7f2aec1d3340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:03.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.130+0000 7f2af0da9640 1 -- 172.21.15.164:0/3551625982 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2aec0b0f70 con 0x7f2aec142a70 2024-03-23T14:57:03.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.130+0000 7f2af0da9640 1 --2- 172.21.15.164:0/3551625982 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2aec142a70 0x7f2aec141540 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2aec0afea0 tx=0x7f2adc010b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:03.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.130+0000 7f2ae8ff9640 1 -- 172.21.15.164:0/3551625982 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2adc00e8a0 con 0x7f2aec142a70 2024-03-23T14:57:03.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.130+0000 7f2ae8ff9640 1 -- 172.21.15.164:0/3551625982 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2adc00ea00 con 0x7f2aec142a70 2024-03-23T14:57:03.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.130+0000 7f2af280c640 1 -- 172.21.15.164:0/3551625982 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2aec140420 con 0x7f2aec142a70 2024-03-23T14:57:03.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.130+0000 7f2ae8ff9640 1 -- 172.21.15.164:0/3551625982 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2adc00edb0 con 0x7f2aec142a70 2024-03-23T14:57:03.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.130+0000 7f2af280c640 1 -- 172.21.15.164:0/3551625982 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2aec1e1930 con 0x7f2aec142a70 2024-03-23T14:57:03.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.130+0000 7f2af280c640 1 -- 172.21.15.164:0/3551625982 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2aec140d20 con 0x7f2aec142a70 2024-03-23T14:57:03.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.134+0000 7f2ae8ff9640 1 -- 172.21.15.164:0/3551625982 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7f2adc020410 con 0x7f2aec142a70 2024-03-23T14:57:03.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.134+0000 7f2ae8ff9640 1 --2- 172.21.15.164:0/3551625982 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f2acc0672a0 0x7f2acc069760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:03.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.134+0000 7f2ae8ff9640 1 -- 172.21.15.164:0/3551625982 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f2adc0848a0 con 0x7f2aec142a70 2024-03-23T14:57:03.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.134+0000 7f2aebfff640 1 --2- 172.21.15.164:0/3551625982 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f2acc0672a0 0x7f2acc069760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:03.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.134+0000 7f2aebfff640 1 --2- 172.21.15.164:0/3551625982 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f2acc0672a0 0x7f2acc069760 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2ae40ad000 tx=0x7f2ae4001d90 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:03.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.138+0000 7f2ae8ff9640 1 -- 172.21.15.164:0/3551625982 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7f2aec140d20 con 0x7f2aec142a70 2024-03-23T14:57:03.294 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.290+0000 7f2af280c640 1 -- 172.21.15.164:0/3551625982 --> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2aec0b0df0 con 0x7f2acc0672a0 2024-03-23T14:57:03.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.306+0000 7f2ae8ff9640 1 -- 172.21.15.164:0/3551625982 <== mgr.4107 v2:172.21.15.164:6832/2139087985 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251952 (secure 0 0 0) 0x7f2ae4004d30 con 0x7f2acc0672a0 2024-03-23T14:57:03.311 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-23T14:57:03.314 INFO:teuthology.orchestra.run.smithi164.stderr:dumped all 2024-03-23T14:57:03.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.310+0000 7f2af280c640 1 -- 172.21.15.164:0/3551625982 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f2acc0672a0 msgr2=0x7f2acc069760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:03.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.310+0000 7f2af280c640 1 --2- 172.21.15.164:0/3551625982 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f2acc0672a0 0x7f2acc069760 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2ae40ad000 tx=0x7f2ae4001d90 comp rx=0 tx=0).stop 2024-03-23T14:57:03.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.310+0000 7f2af280c640 1 -- 172.21.15.164:0/3551625982 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2aec142a70 msgr2=0x7f2aec141540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:03.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.310+0000 7f2af280c640 1 --2- 172.21.15.164:0/3551625982 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2aec142a70 0x7f2aec141540 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2aec0afea0 tx=0x7f2adc010b70 comp rx=0 tx=0).stop 2024-03-23T14:57:03.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.310+0000 7f2af280c640 1 -- 172.21.15.164:0/3551625982 shutdown_connections 2024-03-23T14:57:03.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.310+0000 7f2af280c640 1 --2- 172.21.15.164:0/3551625982 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2aec142a70 0x7f2aec141540 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:03.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.310+0000 7f2af280c640 1 --2- 172.21.15.164:0/3551625982 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2aec1e1170 0x7f2aec141000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:03.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.310+0000 7f2af280c640 1 --2- 172.21.15.164:0/3551625982 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f2acc0672a0 0x7f2acc069760 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:03.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.310+0000 7f2af280c640 1 --2- 172.21.15.164:0/3551625982 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2ad8003980 0x7f2aec1d3340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:03.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.310+0000 7f2af280c640 1 -- 172.21.15.164:0/3551625982 >> 172.21.15.164:0/3551625982 conn(0x7f2aec0b8ca0 msgr2=0x7f2aec0b5560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:03.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.310+0000 7f2af280c640 1 -- 172.21.15.164:0/3551625982 shutdown_connections 2024-03-23T14:57:03.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.310+0000 7f2af280c640 1 -- 172.21.15.164:0/3551625982 wait complete. 2024-03-23T14:57:03.332 INFO:teuthology.orchestra.run.smithi164.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-03-23T14:57:03.110303+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":185,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219000,"kb_used_data":2800,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755720,"statfs":{"total":773094113280,"available":772869857280,"internally_reserved":0,"allocated":2867200,"data_stored":1740428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.525781"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.518848+0000","last_change":"2024-03-23T14:56:59.652438+0000","last_active":"2024-03-23T14:57:01.518848+0000","last_peered":"2024-03-23T14:57:01.518848+0000","last_clean":"2024-03-23T14:57:01.518848+0000","last_became_active":"2024-03-23T14:56:59.652169+0000","last_became_peered":"2024-03-23T14:56:59.652169+0000","last_unstale":"2024-03-23T14:57:01.518848+0000","last_undegraded":"2024-03-23T14:57:01.518848+0000","last_fullsized":"2024-03-23T14:57:01.518848+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:58:31.542399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.621127+0000","last_change":"2024-03-23T14:56:59.632308+0000","last_active":"2024-03-23T14:57:00.621127+0000","last_peered":"2024-03-23T14:57:00.621127+0000","last_clean":"2024-03-23T14:57:00.621127+0000","last_became_active":"2024-03-23T14:56:59.631895+0000","last_became_peered":"2024-03-23T14:56:59.631895+0000","last_unstale":"2024-03-23T14:57:00.621127+0000","last_undegraded":"2024-03-23T14:57:00.621127+0000","last_fullsized":"2024-03-23T14:57:00.621127+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:59:19.727729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.555798+0000","last_change":"2024-03-23T14:56:59.636862+0000","last_active":"2024-03-23T14:57:01.555798+0000","last_peered":"2024-03-23T14:57:01.555798+0000","last_clean":"2024-03-23T14:57:01.555798+0000","last_became_active":"2024-03-23T14:56:59.636209+0000","last_became_peered":"2024-03-23T14:56:59.636209+0000","last_unstale":"2024-03-23T14:57:01.555798+0000","last_undegraded":"2024-03-23T14:57:01.555798+0000","last_fullsized":"2024-03-23T14:57:01.555798+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:11:59.565543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.551115+0000","last_change":"2024-03-23T14:56:59.662732+0000","last_active":"2024-03-23T14:57:01.551115+0000","last_peered":"2024-03-23T14:57:01.551115+0000","last_clean":"2024-03-23T14:57:01.551115+0000","last_became_active":"2024-03-23T14:56:59.661656+0000","last_became_peered":"2024-03-23T14:56:59.661656+0000","last_unstale":"2024-03-23T14:57:01.551115+0000","last_undegraded":"2024-03-23T14:57:01.551115+0000","last_fullsized":"2024-03-23T14:57:01.551115+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:35:32.710122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.518258+0000","last_change":"2024-03-23T14:56:59.651141+0000","last_active":"2024-03-23T14:57:01.518258+0000","last_peered":"2024-03-23T14:57:01.518258+0000","last_clean":"2024-03-23T14:57:01.518258+0000","last_became_active":"2024-03-23T14:56:59.650922+0000","last_became_peered":"2024-03-23T14:56:59.650922+0000","last_unstale":"2024-03-23T14:57:01.518258+0000","last_undegraded":"2024-03-23T14:57:01.518258+0000","last_fullsized":"2024-03-23T14:57:01.518258+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:14:04.996547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.551288+0000","last_change":"2024-03-23T14:56:59.662551+0000","last_active":"2024-03-23T14:57:01.551288+0000","last_peered":"2024-03-23T14:57:01.551288+0000","last_clean":"2024-03-23T14:57:01.551288+0000","last_became_active":"2024-03-23T14:56:59.661732+0000","last_became_peered":"2024-03-23T14:56:59.661732+0000","last_unstale":"2024-03-23T14:57:01.551288+0000","last_undegraded":"2024-03-23T14:57:01.551288+0000","last_fullsized":"2024-03-23T14:57:01.551288+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:35:26.329408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.518244+0000","last_change":"2024-03-23T14:56:59.646745+0000","last_active":"2024-03-23T14:57:01.518244+0000","last_peered":"2024-03-23T14:57:01.518244+0000","last_clean":"2024-03-23T14:57:01.518244+0000","last_became_active":"2024-03-23T14:56:59.646354+0000","last_became_peered":"2024-03-23T14:56:59.646354+0000","last_unstale":"2024-03-23T14:57:01.518244+0000","last_undegraded":"2024-03-23T14:57:01.518244+0000","last_fullsized":"2024-03-23T14:57:01.518244+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:58:40.996298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.617783+0000","last_change":"2024-03-23T14:56:59.659502+0000","last_active":"2024-03-23T14:57:00.617783+0000","last_peered":"2024-03-23T14:57:00.617783+0000","last_clean":"2024-03-23T14:57:00.617783+0000","last_became_active":"2024-03-23T14:56:59.659222+0000","last_became_peered":"2024-03-23T14:56:59.659222+0000","last_unstale":"2024-03-23T14:57:00.617783+0000","last_undegraded":"2024-03-23T14:57:00.617783+0000","last_fullsized":"2024-03-23T14:57:00.617783+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:07:26.887157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.519017+0000","last_change":"2024-03-23T14:56:59.659935+0000","last_active":"2024-03-23T14:57:01.519017+0000","last_peered":"2024-03-23T14:57:01.519017+0000","last_clean":"2024-03-23T14:57:01.519017+0000","last_became_active":"2024-03-23T14:56:59.659501+0000","last_became_peered":"2024-03-23T14:56:59.659501+0000","last_unstale":"2024-03-23T14:57:01.519017+0000","last_undegraded":"2024-03-23T14:57:01.519017+0000","last_fullsized":"2024-03-23T14:57:01.519017+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:21:16.376914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.529493+0000","last_change":"2024-03-23T14:56:59.659013+0000","last_active":"2024-03-23T14:57:01.529493+0000","last_peered":"2024-03-23T14:57:01.529493+0000","last_clean":"2024-03-23T14:57:01.529493+0000","last_became_active":"2024-03-23T14:56:59.658667+0000","last_became_peered":"2024-03-23T14:56:59.658667+0000","last_unstale":"2024-03-23T14:57:01.529493+0000","last_undegraded":"2024-03-23T14:57:01.529493+0000","last_fullsized":"2024-03-23T14:57:01.529493+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:12:59.763661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.529535+0000","last_change":"2024-03-23T14:56:59.663729+0000","last_active":"2024-03-23T14:57:01.529535+0000","last_peered":"2024-03-23T14:57:01.529535+0000","last_clean":"2024-03-23T14:57:01.529535+0000","last_became_active":"2024-03-23T14:56:59.663313+0000","last_became_peered":"2024-03-23T14:56:59.663313+0000","last_unstale":"2024-03-23T14:57:01.529535+0000","last_undegraded":"2024-03-23T14:57:01.529535+0000","last_fullsized":"2024-03-23T14:57:01.529535+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:57:06.681361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.619818+0000","last_change":"2024-03-23T14:56:59.640819+0000","last_active":"2024-03-23T14:57:00.619818+0000","last_peered":"2024-03-23T14:57:00.619818+0000","last_clean":"2024-03-23T14:57:00.619818+0000","last_became_active":"2024-03-23T14:56:59.640330+0000","last_became_peered":"2024-03-23T14:56:59.640330+0000","last_unstale":"2024-03-23T14:57:00.619818+0000","last_undegraded":"2024-03-23T14:57:00.619818+0000","last_fullsized":"2024-03-23T14:57:00.619818+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:01:26.168657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.618487+0000","last_change":"2024-03-23T14:56:59.645597+0000","last_active":"2024-03-23T14:57:00.618487+0000","last_peered":"2024-03-23T14:57:00.618487+0000","last_clean":"2024-03-23T14:57:00.618487+0000","last_became_active":"2024-03-23T14:56:59.644700+0000","last_became_peered":"2024-03-23T14:56:59.644700+0000","last_unstale":"2024-03-23T14:57:00.618487+0000","last_undegraded":"2024-03-23T14:57:00.618487+0000","last_fullsized":"2024-03-23T14:57:00.618487+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:59:19.727729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.556462+0000","last_change":"2024-03-23T14:56:59.636241+0000","last_active":"2024-03-23T14:57:01.556462+0000","last_peered":"2024-03-23T14:57:01.556462+0000","last_clean":"2024-03-23T14:57:01.556462+0000","last_became_active":"2024-03-23T14:56:59.635803+0000","last_became_peered":"2024-03-23T14:56:59.635803+0000","last_unstale":"2024-03-23T14:57:01.556462+0000","last_undegraded":"2024-03-23T14:57:01.556462+0000","last_fullsized":"2024-03-23T14:57:01.556462+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:07:26.887157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.620719+0000","last_change":"2024-03-23T14:56:59.642239+0000","last_active":"2024-03-23T14:57:00.620719+0000","last_peered":"2024-03-23T14:57:00.620719+0000","last_clean":"2024-03-23T14:57:00.620719+0000","last_became_active":"2024-03-23T14:56:59.641872+0000","last_became_peered":"2024-03-23T14:56:59.641872+0000","last_unstale":"2024-03-23T14:57:00.620719+0000","last_undegraded":"2024-03-23T14:57:00.620719+0000","last_fullsized":"2024-03-23T14:57:00.620719+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:09:00.536071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.551044+0000","last_change":"2024-03-23T14:56:59.634551+0000","last_active":"2024-03-23T14:57:01.551044+0000","last_peered":"2024-03-23T14:57:01.551044+0000","last_clean":"2024-03-23T14:57:01.551044+0000","last_became_active":"2024-03-23T14:56:59.634255+0000","last_became_peered":"2024-03-23T14:56:59.634255+0000","last_unstale":"2024-03-23T14:57:01.551044+0000","last_undegraded":"2024-03-23T14:57:01.551044+0000","last_fullsized":"2024-03-23T14:57:01.551044+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:39:22.051918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.552128+0000","last_change":"2024-03-23T14:56:59.657535+0000","last_active":"2024-03-23T14:57:01.552128+0000","last_peered":"2024-03-23T14:57:01.552128+0000","last_clean":"2024-03-23T14:57:01.552128+0000","last_became_active":"2024-03-23T14:56:59.656959+0000","last_became_peered":"2024-03-23T14:56:59.656959+0000","last_unstale":"2024-03-23T14:57:01.552128+0000","last_undegraded":"2024-03-23T14:57:01.552128+0000","last_fullsized":"2024-03-23T14:57:01.552128+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:58:31.542399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.555866+0000","last_change":"2024-03-23T14:56:59.639104+0000","last_active":"2024-03-23T14:57:01.555866+0000","last_peered":"2024-03-23T14:57:01.555866+0000","last_clean":"2024-03-23T14:57:01.555866+0000","last_became_active":"2024-03-23T14:56:59.638744+0000","last_became_peered":"2024-03-23T14:56:59.638744+0000","last_unstale":"2024-03-23T14:57:01.555866+0000","last_undegraded":"2024-03-23T14:57:01.555866+0000","last_fullsized":"2024-03-23T14:57:01.555866+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:29:08.139571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.518933+0000","last_change":"2024-03-23T14:56:59.647334+0000","last_active":"2024-03-23T14:57:01.518933+0000","last_peered":"2024-03-23T14:57:01.518933+0000","last_clean":"2024-03-23T14:57:01.518933+0000","last_became_active":"2024-03-23T14:56:59.646801+0000","last_became_peered":"2024-03-23T14:56:59.646801+0000","last_unstale":"2024-03-23T14:57:01.518933+0000","last_undegraded":"2024-03-23T14:57:01.518933+0000","last_fullsized":"2024-03-23T14:57:01.518933+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:02:12.733995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.529380+0000","last_change":"2024-03-23T14:56:59.639076+0000","last_active":"2024-03-23T14:57:01.529380+0000","last_peered":"2024-03-23T14:57:01.529380+0000","last_clean":"2024-03-23T14:57:01.529380+0000","last_became_active":"2024-03-23T14:56:59.638267+0000","last_became_peered":"2024-03-23T14:56:59.638267+0000","last_unstale":"2024-03-23T14:57:01.529380+0000","last_undegraded":"2024-03-23T14:57:01.529380+0000","last_fullsized":"2024-03-23T14:57:01.529380+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:59:19.727729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.556655+0000","last_change":"2024-03-23T14:56:59.647220+0000","last_active":"2024-03-23T14:57:01.556655+0000","last_peered":"2024-03-23T14:57:01.556655+0000","last_clean":"2024-03-23T14:57:01.556655+0000","last_became_active":"2024-03-23T14:56:59.646833+0000","last_became_peered":"2024-03-23T14:56:59.646833+0000","last_unstale":"2024-03-23T14:57:01.556655+0000","last_undegraded":"2024-03-23T14:57:01.556655+0000","last_fullsized":"2024-03-23T14:57:01.556655+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:38:24.739838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.551049+0000","last_change":"2024-03-23T14:56:59.662969+0000","last_active":"2024-03-23T14:57:01.551049+0000","last_peered":"2024-03-23T14:57:01.551049+0000","last_clean":"2024-03-23T14:57:01.551049+0000","last_became_active":"2024-03-23T14:56:59.661554+0000","last_became_peered":"2024-03-23T14:56:59.661554+0000","last_unstale":"2024-03-23T14:57:01.551049+0000","last_undegraded":"2024-03-23T14:57:01.551049+0000","last_fullsized":"2024-03-23T14:57:01.551049+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:52:36.323039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.618010+0000","last_change":"2024-03-23T14:56:59.643866+0000","last_active":"2024-03-23T14:57:00.618010+0000","last_peered":"2024-03-23T14:57:00.618010+0000","last_clean":"2024-03-23T14:57:00.618010+0000","last_became_active":"2024-03-23T14:56:59.643577+0000","last_became_peered":"2024-03-23T14:56:59.643577+0000","last_unstale":"2024-03-23T14:57:00.618010+0000","last_undegraded":"2024-03-23T14:57:00.618010+0000","last_fullsized":"2024-03-23T14:57:00.618010+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:37:56.745366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.550754+0000","last_change":"2024-03-23T14:56:59.651798+0000","last_active":"2024-03-23T14:57:01.550754+0000","last_peered":"2024-03-23T14:57:01.550754+0000","last_clean":"2024-03-23T14:57:01.550754+0000","last_became_active":"2024-03-23T14:56:59.651534+0000","last_became_peered":"2024-03-23T14:56:59.651534+0000","last_unstale":"2024-03-23T14:57:01.550754+0000","last_undegraded":"2024-03-23T14:57:01.550754+0000","last_fullsized":"2024-03-23T14:57:01.550754+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:57:06.681361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.619373+0000","last_change":"2024-03-23T14:56:59.645160+0000","last_active":"2024-03-23T14:57:00.619373+0000","last_peered":"2024-03-23T14:57:00.619373+0000","last_clean":"2024-03-23T14:57:00.619373+0000","last_became_active":"2024-03-23T14:56:59.643402+0000","last_became_peered":"2024-03-23T14:56:59.643402+0000","last_unstale":"2024-03-23T14:57:00.619373+0000","last_undegraded":"2024-03-23T14:57:00.619373+0000","last_fullsized":"2024-03-23T14:57:00.619373+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:35:32.710122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.529573+0000","last_change":"2024-03-23T14:56:59.640718+0000","last_active":"2024-03-23T14:57:01.529573+0000","last_peered":"2024-03-23T14:57:01.529573+0000","last_clean":"2024-03-23T14:57:01.529573+0000","last_became_active":"2024-03-23T14:56:59.640386+0000","last_became_peered":"2024-03-23T14:56:59.640386+0000","last_unstale":"2024-03-23T14:57:01.529573+0000","last_undegraded":"2024-03-23T14:57:01.529573+0000","last_fullsized":"2024-03-23T14:57:01.529573+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:39:22.051918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.621420+0000","last_change":"2024-03-23T14:56:59.650572+0000","last_active":"2024-03-23T14:57:00.621420+0000","last_peered":"2024-03-23T14:57:00.621420+0000","last_clean":"2024-03-23T14:57:00.621420+0000","last_became_active":"2024-03-23T14:56:59.650260+0000","last_became_peered":"2024-03-23T14:56:59.650260+0000","last_unstale":"2024-03-23T14:57:00.621420+0000","last_undegraded":"2024-03-23T14:57:00.621420+0000","last_fullsized":"2024-03-23T14:57:00.621420+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:12:59.763661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.619035+0000","last_change":"2024-03-23T14:56:59.646883+0000","last_active":"2024-03-23T14:57:00.619035+0000","last_peered":"2024-03-23T14:57:00.619035+0000","last_clean":"2024-03-23T14:57:00.619035+0000","last_became_active":"2024-03-23T14:56:59.643427+0000","last_became_peered":"2024-03-23T14:56:59.643427+0000","last_unstale":"2024-03-23T14:57:00.619035+0000","last_undegraded":"2024-03-23T14:57:00.619035+0000","last_fullsized":"2024-03-23T14:57:00.619035+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:52:36.323039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.557036+0000","last_change":"2024-03-23T14:56:59.646524+0000","last_active":"2024-03-23T14:57:01.557036+0000","last_peered":"2024-03-23T14:57:01.557036+0000","last_clean":"2024-03-23T14:57:01.557036+0000","last_became_active":"2024-03-23T14:56:59.646037+0000","last_became_peered":"2024-03-23T14:56:59.646037+0000","last_unstale":"2024-03-23T14:57:01.557036+0000","last_undegraded":"2024-03-23T14:57:01.557036+0000","last_fullsized":"2024-03-23T14:57:01.557036+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:57:06.681361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.551413+0000","last_change":"2024-03-23T14:56:59.658015+0000","last_active":"2024-03-23T14:57:01.551413+0000","last_peered":"2024-03-23T14:57:01.551413+0000","last_clean":"2024-03-23T14:57:01.551413+0000","last_became_active":"2024-03-23T14:56:59.657623+0000","last_became_peered":"2024-03-23T14:56:59.657623+0000","last_unstale":"2024-03-23T14:57:01.551413+0000","last_undegraded":"2024-03-23T14:57:01.551413+0000","last_fullsized":"2024-03-23T14:57:01.551413+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:14:04.996547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.555261+0000","last_change":"2024-03-23T14:56:59.660146+0000","last_active":"2024-03-23T14:57:01.555261+0000","last_peered":"2024-03-23T14:57:01.555261+0000","last_clean":"2024-03-23T14:57:01.555261+0000","last_became_active":"2024-03-23T14:56:59.659765+0000","last_became_peered":"2024-03-23T14:56:59.659765+0000","last_unstale":"2024-03-23T14:57:01.555261+0000","last_undegraded":"2024-03-23T14:57:01.555261+0000","last_fullsized":"2024-03-23T14:57:01.555261+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:49:20.982742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.529104+0000","last_change":"2024-03-23T14:56:59.637937+0000","last_active":"2024-03-23T14:57:01.529104+0000","last_peered":"2024-03-23T14:57:01.529104+0000","last_clean":"2024-03-23T14:57:01.529104+0000","last_became_active":"2024-03-23T14:56:59.637465+0000","last_became_peered":"2024-03-23T14:56:59.637465+0000","last_unstale":"2024-03-23T14:57:01.529104+0000","last_undegraded":"2024-03-23T14:57:01.529104+0000","last_fullsized":"2024-03-23T14:57:01.529104+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:38:24.739838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.619136+0000","last_change":"2024-03-23T14:56:59.633433+0000","last_active":"2024-03-23T14:57:00.619136+0000","last_peered":"2024-03-23T14:57:00.619136+0000","last_clean":"2024-03-23T14:57:00.619136+0000","last_became_active":"2024-03-23T14:56:59.633171+0000","last_became_peered":"2024-03-23T14:56:59.633171+0000","last_unstale":"2024-03-23T14:57:00.619136+0000","last_undegraded":"2024-03-23T14:57:00.619136+0000","last_fullsized":"2024-03-23T14:57:00.619136+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:36:11.384485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.618779+0000","last_change":"2024-03-23T14:56:58.621210+0000","last_active":"2024-03-23T14:57:00.618779+0000","last_peered":"2024-03-23T14:57:00.618779+0000","last_clean":"2024-03-23T14:57:00.618779+0000","last_became_active":"2024-03-23T14:56:58.620750+0000","last_became_peered":"2024-03-23T14:56:58.620750+0000","last_unstale":"2024-03-23T14:57:00.618779+0000","last_undegraded":"2024-03-23T14:57:00.618779+0000","last_fullsized":"2024-03-23T14:57:00.618779+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:24:18.799175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.556969+0000","last_change":"2024-03-23T14:56:59.633767+0000","last_active":"2024-03-23T14:57:01.556969+0000","last_peered":"2024-03-23T14:57:01.556969+0000","last_clean":"2024-03-23T14:57:01.556969+0000","last_became_active":"2024-03-23T14:56:59.633390+0000","last_became_peered":"2024-03-23T14:56:59.633390+0000","last_unstale":"2024-03-23T14:57:01.556969+0000","last_undegraded":"2024-03-23T14:57:01.556969+0000","last_fullsized":"2024-03-23T14:57:01.556969+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:12:59.763661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.529138+0000","last_change":"2024-03-23T14:56:58.613611+0000","last_active":"2024-03-23T14:57:01.529138+0000","last_peered":"2024-03-23T14:57:01.529138+0000","last_clean":"2024-03-23T14:57:01.529138+0000","last_became_active":"2024-03-23T14:56:58.608339+0000","last_became_peered":"2024-03-23T14:56:58.608339+0000","last_unstale":"2024-03-23T14:57:01.529138+0000","last_undegraded":"2024-03-23T14:57:01.529138+0000","last_fullsized":"2024-03-23T14:57:01.529138+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:48:12.389105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.618094+0000","last_change":"2024-03-23T14:56:58.621716+0000","last_active":"2024-03-23T14:57:00.618094+0000","last_peered":"2024-03-23T14:57:00.618094+0000","last_clean":"2024-03-23T14:57:00.618094+0000","last_became_active":"2024-03-23T14:56:58.620675+0000","last_became_peered":"2024-03-23T14:56:58.620675+0000","last_unstale":"2024-03-23T14:57:00.618094+0000","last_undegraded":"2024-03-23T14:57:00.618094+0000","last_fullsized":"2024-03-23T14:57:00.618094+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:07:10.915534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.617954+0000","last_change":"2024-03-23T14:56:59.644784+0000","last_active":"2024-03-23T14:57:00.617954+0000","last_peered":"2024-03-23T14:57:00.617954+0000","last_clean":"2024-03-23T14:57:00.617954+0000","last_became_active":"2024-03-23T14:56:59.644312+0000","last_became_peered":"2024-03-23T14:56:59.644312+0000","last_unstale":"2024-03-23T14:57:00.617954+0000","last_undegraded":"2024-03-23T14:57:00.617954+0000","last_fullsized":"2024-03-23T14:57:00.617954+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:38:24.739838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.550888+0000","last_change":"2024-03-23T14:56:58.611793+0000","last_active":"2024-03-23T14:57:01.550888+0000","last_peered":"2024-03-23T14:57:01.550888+0000","last_clean":"2024-03-23T14:57:01.550888+0000","last_became_active":"2024-03-23T14:56:58.611131+0000","last_became_peered":"2024-03-23T14:56:58.611131+0000","last_unstale":"2024-03-23T14:57:01.550888+0000","last_undegraded":"2024-03-23T14:57:01.550888+0000","last_fullsized":"2024-03-23T14:57:01.550888+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:36:10.376200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.620461+0000","last_change":"2024-03-23T14:56:59.649256+0000","last_active":"2024-03-23T14:57:00.620461+0000","last_peered":"2024-03-23T14:57:00.620461+0000","last_clean":"2024-03-23T14:57:00.620461+0000","last_became_active":"2024-03-23T14:56:59.648957+0000","last_became_peered":"2024-03-23T14:56:59.648957+0000","last_unstale":"2024-03-23T14:57:00.620461+0000","last_undegraded":"2024-03-23T14:57:00.620461+0000","last_fullsized":"2024-03-23T14:57:00.620461+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:07:26.887157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.529168+0000","last_change":"2024-03-23T14:56:58.630337+0000","last_active":"2024-03-23T14:57:01.529168+0000","last_peered":"2024-03-23T14:57:01.529168+0000","last_clean":"2024-03-23T14:57:01.529168+0000","last_became_active":"2024-03-23T14:56:58.629899+0000","last_became_peered":"2024-03-23T14:56:58.629899+0000","last_unstale":"2024-03-23T14:57:01.529168+0000","last_undegraded":"2024-03-23T14:57:01.529168+0000","last_fullsized":"2024-03-23T14:57:01.529168+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:24:18.799175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.551507+0000","last_change":"2024-03-23T14:56:59.657632+0000","last_active":"2024-03-23T14:57:01.551507+0000","last_peered":"2024-03-23T14:57:01.551507+0000","last_clean":"2024-03-23T14:57:01.551507+0000","last_became_active":"2024-03-23T14:56:59.655136+0000","last_became_peered":"2024-03-23T14:56:59.655136+0000","last_unstale":"2024-03-23T14:57:01.551507+0000","last_undegraded":"2024-03-23T14:57:01.551507+0000","last_fullsized":"2024-03-23T14:57:01.551507+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:46:57.866240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.551534+0000","last_change":"2024-03-23T14:56:58.627359+0000","last_active":"2024-03-23T14:57:01.551534+0000","last_peered":"2024-03-23T14:57:01.551534+0000","last_clean":"2024-03-23T14:57:01.551534+0000","last_became_active":"2024-03-23T14:56:58.626614+0000","last_became_peered":"2024-03-23T14:56:58.626614+0000","last_unstale":"2024-03-23T14:57:01.551534+0000","last_undegraded":"2024-03-23T14:57:01.551534+0000","last_fullsized":"2024-03-23T14:57:01.551534+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:05:00.726517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.556310+0000","last_change":"2024-03-23T14:56:59.659806+0000","last_active":"2024-03-23T14:57:01.556310+0000","last_peered":"2024-03-23T14:57:01.556310+0000","last_clean":"2024-03-23T14:57:01.556310+0000","last_became_active":"2024-03-23T14:56:59.658983+0000","last_became_peered":"2024-03-23T14:56:59.658983+0000","last_unstale":"2024-03-23T14:57:01.556310+0000","last_undegraded":"2024-03-23T14:57:01.556310+0000","last_fullsized":"2024-03-23T14:57:01.556310+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:58:29.591659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.621558+0000","last_change":"2024-03-23T14:56:58.625877+0000","last_active":"2024-03-23T14:57:00.621558+0000","last_peered":"2024-03-23T14:57:00.621558+0000","last_clean":"2024-03-23T14:57:00.621558+0000","last_became_active":"2024-03-23T14:56:58.625410+0000","last_became_peered":"2024-03-23T14:56:58.625410+0000","last_unstale":"2024-03-23T14:57:00.621558+0000","last_undegraded":"2024-03-23T14:57:00.621558+0000","last_fullsized":"2024-03-23T14:57:00.621558+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:05:00.726517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.619847+0000","last_change":"2024-03-23T14:56:59.647811+0000","last_active":"2024-03-23T14:57:00.619847+0000","last_peered":"2024-03-23T14:57:00.619847+0000","last_clean":"2024-03-23T14:57:00.619847+0000","last_became_active":"2024-03-23T14:56:59.647138+0000","last_became_peered":"2024-03-23T14:56:59.647138+0000","last_unstale":"2024-03-23T14:57:00.619847+0000","last_undegraded":"2024-03-23T14:57:00.619847+0000","last_fullsized":"2024-03-23T14:57:00.619847+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:49:20.982742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.529463+0000","last_change":"2024-03-23T14:56:58.630507+0000","last_active":"2024-03-23T14:57:01.529463+0000","last_peered":"2024-03-23T14:57:01.529463+0000","last_clean":"2024-03-23T14:57:01.529463+0000","last_became_active":"2024-03-23T14:56:58.629991+0000","last_became_peered":"2024-03-23T14:56:58.629991+0000","last_unstale":"2024-03-23T14:57:01.529463+0000","last_undegraded":"2024-03-23T14:57:01.529463+0000","last_fullsized":"2024-03-23T14:57:01.529463+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:01:25.160372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.518629+0000","last_change":"2024-03-23T14:56:59.649839+0000","last_active":"2024-03-23T14:57:01.518629+0000","last_peered":"2024-03-23T14:57:01.518629+0000","last_clean":"2024-03-23T14:57:01.518629+0000","last_became_active":"2024-03-23T14:56:59.648333+0000","last_became_peered":"2024-03-23T14:56:59.648333+0000","last_unstale":"2024-03-23T14:57:01.518629+0000","last_undegraded":"2024-03-23T14:57:01.518629+0000","last_fullsized":"2024-03-23T14:57:01.518629+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:46:57.866240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.557119+0000","last_change":"2024-03-23T14:56:58.609669+0000","last_active":"2024-03-23T14:57:01.557119+0000","last_peered":"2024-03-23T14:57:01.557119+0000","last_clean":"2024-03-23T14:57:01.557119+0000","last_became_active":"2024-03-23T14:56:58.608906+0000","last_became_peered":"2024-03-23T14:56:58.608906+0000","last_unstale":"2024-03-23T14:57:01.557119+0000","last_undegraded":"2024-03-23T14:57:01.557119+0000","last_fullsized":"2024-03-23T14:57:01.557119+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:34:51.972168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.556854+0000","last_change":"2024-03-23T14:56:59.648070+0000","last_active":"2024-03-23T14:57:01.556854+0000","last_peered":"2024-03-23T14:57:01.556854+0000","last_clean":"2024-03-23T14:57:01.556854+0000","last_became_active":"2024-03-23T14:56:59.647803+0000","last_became_peered":"2024-03-23T14:56:59.647803+0000","last_unstale":"2024-03-23T14:57:01.556854+0000","last_undegraded":"2024-03-23T14:57:01.556854+0000","last_fullsized":"2024-03-23T14:57:01.556854+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:37:56.745366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.551552+0000","last_change":"2024-03-23T14:56:58.626829+0000","last_active":"2024-03-23T14:57:01.551552+0000","last_peered":"2024-03-23T14:57:01.551552+0000","last_clean":"2024-03-23T14:57:01.551552+0000","last_became_active":"2024-03-23T14:56:58.626375+0000","last_became_peered":"2024-03-23T14:56:58.626375+0000","last_unstale":"2024-03-23T14:57:01.551552+0000","last_undegraded":"2024-03-23T14:57:01.551552+0000","last_fullsized":"2024-03-23T14:57:01.551552+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:48:12.389105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.518496+0000","last_change":"2024-03-23T14:56:59.653570+0000","last_active":"2024-03-23T14:57:01.518496+0000","last_peered":"2024-03-23T14:57:01.518496+0000","last_clean":"2024-03-23T14:57:01.518496+0000","last_became_active":"2024-03-23T14:56:59.653284+0000","last_became_peered":"2024-03-23T14:56:59.653284+0000","last_unstale":"2024-03-23T14:57:01.518496+0000","last_undegraded":"2024-03-23T14:57:01.518496+0000","last_fullsized":"2024-03-23T14:57:01.518496+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:34:30.083232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.528879+0000","last_change":"2024-03-23T14:56:58.614950+0000","last_active":"2024-03-23T14:57:01.528879+0000","last_peered":"2024-03-23T14:57:01.528879+0000","last_clean":"2024-03-23T14:57:01.528879+0000","last_became_active":"2024-03-23T14:56:58.614441+0000","last_became_peered":"2024-03-23T14:56:58.614441+0000","last_unstale":"2024-03-23T14:57:01.528879+0000","last_undegraded":"2024-03-23T14:57:01.528879+0000","last_fullsized":"2024-03-23T14:57:01.528879+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:05:00.726517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.528950+0000","last_change":"2024-03-23T14:56:59.636998+0000","last_active":"2024-03-23T14:57:01.528950+0000","last_peered":"2024-03-23T14:57:01.528950+0000","last_clean":"2024-03-23T14:57:01.528950+0000","last_became_active":"2024-03-23T14:56:59.636444+0000","last_became_peered":"2024-03-23T14:56:59.636444+0000","last_unstale":"2024-03-23T14:57:01.528950+0000","last_undegraded":"2024-03-23T14:57:01.528950+0000","last_fullsized":"2024-03-23T14:57:01.528950+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:09:00.536071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.555118+0000","last_change":"2024-03-23T14:56:58.604409+0000","last_active":"2024-03-23T14:57:01.555118+0000","last_peered":"2024-03-23T14:57:01.555118+0000","last_clean":"2024-03-23T14:57:01.555118+0000","last_became_active":"2024-03-23T14:56:58.604175+0000","last_became_peered":"2024-03-23T14:56:58.604175+0000","last_unstale":"2024-03-23T14:57:01.555118+0000","last_undegraded":"2024-03-23T14:57:01.555118+0000","last_fullsized":"2024-03-23T14:57:01.555118+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:07:10.915534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.518264+0000","last_change":"2024-03-23T14:56:59.651120+0000","last_active":"2024-03-23T14:57:01.518264+0000","last_peered":"2024-03-23T14:57:01.518264+0000","last_clean":"2024-03-23T14:57:01.518264+0000","last_became_active":"2024-03-23T14:56:59.650822+0000","last_became_peered":"2024-03-23T14:56:59.650822+0000","last_unstale":"2024-03-23T14:57:01.518264+0000","last_undegraded":"2024-03-23T14:57:01.518264+0000","last_fullsized":"2024-03-23T14:57:01.518264+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:35:26.329408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.619806+0000","last_change":"2024-03-23T14:56:58.606372+0000","last_active":"2024-03-23T14:57:00.619806+0000","last_peered":"2024-03-23T14:57:00.619806+0000","last_clean":"2024-03-23T14:57:00.619806+0000","last_became_active":"2024-03-23T14:56:58.606011+0000","last_became_peered":"2024-03-23T14:56:58.606011+0000","last_unstale":"2024-03-23T14:57:00.619806+0000","last_undegraded":"2024-03-23T14:57:00.619806+0000","last_fullsized":"2024-03-23T14:57:00.619806+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:34:51.972168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.518669+0000","last_change":"2024-03-23T14:56:59.652521+0000","last_active":"2024-03-23T14:57:01.518669+0000","last_peered":"2024-03-23T14:57:01.518669+0000","last_clean":"2024-03-23T14:57:01.518669+0000","last_became_active":"2024-03-23T14:56:59.652270+0000","last_became_peered":"2024-03-23T14:56:59.652270+0000","last_unstale":"2024-03-23T14:57:01.518669+0000","last_undegraded":"2024-03-23T14:57:01.518669+0000","last_fullsized":"2024-03-23T14:57:01.518669+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:51:59.393587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.556006+0000","last_change":"2024-03-23T14:56:58.610985+0000","last_active":"2024-03-23T14:57:01.556006+0000","last_peered":"2024-03-23T14:57:01.556006+0000","last_clean":"2024-03-23T14:57:01.556006+0000","last_became_active":"2024-03-23T14:56:58.610717+0000","last_became_peered":"2024-03-23T14:56:58.610717+0000","last_unstale":"2024-03-23T14:57:01.556006+0000","last_undegraded":"2024-03-23T14:57:01.556006+0000","last_fullsized":"2024-03-23T14:57:01.556006+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:24:18.799175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.519128+0000","last_change":"2024-03-23T14:56:59.652993+0000","last_active":"2024-03-23T14:57:01.519128+0000","last_peered":"2024-03-23T14:57:01.519128+0000","last_clean":"2024-03-23T14:57:01.519128+0000","last_became_active":"2024-03-23T14:56:59.652718+0000","last_became_peered":"2024-03-23T14:56:59.652718+0000","last_unstale":"2024-03-23T14:57:01.519128+0000","last_undegraded":"2024-03-23T14:57:01.519128+0000","last_fullsized":"2024-03-23T14:57:01.519128+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:06:34.693789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.621685+0000","last_change":"2024-03-23T14:56:58.608418+0000","last_active":"2024-03-23T14:57:00.621685+0000","last_peered":"2024-03-23T14:57:00.621685+0000","last_clean":"2024-03-23T14:57:00.621685+0000","last_became_active":"2024-03-23T14:56:58.607992+0000","last_became_peered":"2024-03-23T14:56:58.607992+0000","last_unstale":"2024-03-23T14:57:00.621685+0000","last_undegraded":"2024-03-23T14:57:00.621685+0000","last_fullsized":"2024-03-23T14:57:00.621685+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:36:10.376200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.551352+0000","last_change":"2024-03-23T14:56:59.657882+0000","last_active":"2024-03-23T14:57:01.551352+0000","last_peered":"2024-03-23T14:57:01.551352+0000","last_clean":"2024-03-23T14:57:01.551352+0000","last_became_active":"2024-03-23T14:56:59.657004+0000","last_became_peered":"2024-03-23T14:56:59.657004+0000","last_unstale":"2024-03-23T14:57:01.551352+0000","last_undegraded":"2024-03-23T14:57:01.551352+0000","last_fullsized":"2024-03-23T14:57:01.551352+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:58:40.996298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.557066+0000","last_change":"2024-03-23T14:56:58.617074+0000","last_active":"2024-03-23T14:57:01.557066+0000","last_peered":"2024-03-23T14:57:01.557066+0000","last_clean":"2024-03-23T14:57:01.557066+0000","last_became_active":"2024-03-23T14:56:58.616732+0000","last_became_peered":"2024-03-23T14:56:58.616732+0000","last_unstale":"2024-03-23T14:57:01.557066+0000","last_undegraded":"2024-03-23T14:57:01.557066+0000","last_fullsized":"2024-03-23T14:57:01.557066+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:05:00.726517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.518726+0000","last_change":"2024-03-23T14:56:59.664625+0000","last_active":"2024-03-23T14:57:01.518726+0000","last_peered":"2024-03-23T14:57:01.518726+0000","last_clean":"2024-03-23T14:57:01.518726+0000","last_became_active":"2024-03-23T14:56:59.664268+0000","last_became_peered":"2024-03-23T14:56:59.664268+0000","last_unstale":"2024-03-23T14:57:01.518726+0000","last_undegraded":"2024-03-23T14:57:01.518726+0000","last_fullsized":"2024-03-23T14:57:01.518726+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:53:55.152886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.517881+0000","last_change":"2024-03-23T14:56:48.722063+0000","last_active":"2024-03-23T14:57:01.517881+0000","last_peered":"2024-03-23T14:57:01.517881+0000","last_clean":"2024-03-23T14:57:01.517881+0000","last_became_active":"2024-03-23T14:56:48.720079+0000","last_became_peered":"2024-03-23T14:56:48.720079+0000","last_unstale":"2024-03-23T14:57:01.517881+0000","last_undegraded":"2024-03-23T14:57:01.517881+0000","last_fullsized":"2024-03-23T14:57:01.517881+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:47.478078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:47.478078+0000","last_clean_scrub_stamp":"2024-03-23T14:56:47.478078+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:55:51.231901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.551138+0000","last_change":"2024-03-23T14:56:58.611796+0000","last_active":"2024-03-23T14:57:01.551138+0000","last_peered":"2024-03-23T14:57:01.551138+0000","last_clean":"2024-03-23T14:57:01.551138+0000","last_became_active":"2024-03-23T14:56:58.611243+0000","last_became_peered":"2024-03-23T14:56:58.611243+0000","last_unstale":"2024-03-23T14:57:01.551138+0000","last_undegraded":"2024-03-23T14:57:01.551138+0000","last_fullsized":"2024-03-23T14:57:01.551138+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:24:18.799175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.618598+0000","last_change":"2024-03-23T14:56:59.664611+0000","last_active":"2024-03-23T14:57:00.618598+0000","last_peered":"2024-03-23T14:57:00.618598+0000","last_clean":"2024-03-23T14:57:00.618598+0000","last_became_active":"2024-03-23T14:56:59.664275+0000","last_became_peered":"2024-03-23T14:56:59.664275+0000","last_unstale":"2024-03-23T14:57:00.618598+0000","last_undegraded":"2024-03-23T14:57:00.618598+0000","last_fullsized":"2024-03-23T14:57:00.618598+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:12:59.763661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.517853+0000","last_change":"2024-03-23T14:56:58.620960+0000","last_active":"2024-03-23T14:57:01.517853+0000","last_peered":"2024-03-23T14:57:01.517853+0000","last_clean":"2024-03-23T14:57:01.517853+0000","last_became_active":"2024-03-23T14:56:58.620592+0000","last_became_peered":"2024-03-23T14:56:58.620592+0000","last_unstale":"2024-03-23T14:57:01.517853+0000","last_undegraded":"2024-03-23T14:57:01.517853+0000","last_fullsized":"2024-03-23T14:57:01.517853+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:01:25.160372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.556871+0000","last_change":"2024-03-23T14:56:59.621260+0000","last_active":"2024-03-23T14:57:01.556871+0000","last_peered":"2024-03-23T14:57:01.556871+0000","last_clean":"2024-03-23T14:57:01.556871+0000","last_became_active":"2024-03-23T14:56:59.620924+0000","last_became_peered":"2024-03-23T14:56:59.620924+0000","last_unstale":"2024-03-23T14:57:01.556871+0000","last_undegraded":"2024-03-23T14:57:01.556871+0000","last_fullsized":"2024-03-23T14:57:01.556871+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:59:19.727729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.555522+0000","last_change":"2024-03-23T14:56:58.611902+0000","last_active":"2024-03-23T14:57:01.555522+0000","last_peered":"2024-03-23T14:57:01.555522+0000","last_clean":"2024-03-23T14:57:01.555522+0000","last_became_active":"2024-03-23T14:56:58.611208+0000","last_became_peered":"2024-03-23T14:56:58.611208+0000","last_unstale":"2024-03-23T14:57:01.555522+0000","last_undegraded":"2024-03-23T14:57:01.555522+0000","last_fullsized":"2024-03-23T14:57:01.555522+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:36:10.376200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.620759+0000","last_change":"2024-03-23T14:56:59.640818+0000","last_active":"2024-03-23T14:57:00.620759+0000","last_peered":"2024-03-23T14:57:00.620759+0000","last_clean":"2024-03-23T14:57:00.620759+0000","last_became_active":"2024-03-23T14:56:59.640493+0000","last_became_peered":"2024-03-23T14:56:59.640493+0000","last_unstale":"2024-03-23T14:57:00.620759+0000","last_undegraded":"2024-03-23T14:57:00.620759+0000","last_fullsized":"2024-03-23T14:57:00.620759+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:38:24.739838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.551684+0000","last_change":"2024-03-23T14:56:58.616782+0000","last_active":"2024-03-23T14:57:01.551684+0000","last_peered":"2024-03-23T14:57:01.551684+0000","last_clean":"2024-03-23T14:57:01.551684+0000","last_became_active":"2024-03-23T14:56:58.615957+0000","last_became_peered":"2024-03-23T14:56:58.615957+0000","last_unstale":"2024-03-23T14:57:01.551684+0000","last_undegraded":"2024-03-23T14:57:01.551684+0000","last_fullsized":"2024-03-23T14:57:01.551684+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:01:25.160372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.619632+0000","last_change":"2024-03-23T14:56:59.623480+0000","last_active":"2024-03-23T14:57:00.619632+0000","last_peered":"2024-03-23T14:57:00.619632+0000","last_clean":"2024-03-23T14:57:00.619632+0000","last_became_active":"2024-03-23T14:56:59.623224+0000","last_became_peered":"2024-03-23T14:56:59.623224+0000","last_unstale":"2024-03-23T14:57:00.619632+0000","last_undegraded":"2024-03-23T14:57:00.619632+0000","last_fullsized":"2024-03-23T14:57:00.619632+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:48:13.397390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.621161+0000","last_change":"2024-03-23T14:56:58.604398+0000","last_active":"2024-03-23T14:57:00.621161+0000","last_peered":"2024-03-23T14:57:00.621161+0000","last_clean":"2024-03-23T14:57:00.621161+0000","last_became_active":"2024-03-23T14:56:58.603192+0000","last_became_peered":"2024-03-23T14:56:58.603192+0000","last_unstale":"2024-03-23T14:57:00.621161+0000","last_undegraded":"2024-03-23T14:57:00.621161+0000","last_fullsized":"2024-03-23T14:57:00.621161+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:24:18.799175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.551563+0000","last_change":"2024-03-23T14:56:59.661728+0000","last_active":"2024-03-23T14:57:01.551563+0000","last_peered":"2024-03-23T14:57:01.551563+0000","last_clean":"2024-03-23T14:57:01.551563+0000","last_became_active":"2024-03-23T14:56:59.656955+0000","last_became_peered":"2024-03-23T14:56:59.656955+0000","last_unstale":"2024-03-23T14:57:01.551563+0000","last_undegraded":"2024-03-23T14:57:01.551563+0000","last_fullsized":"2024-03-23T14:57:01.551563+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:51:59.393587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.518923+0000","last_change":"2024-03-23T14:56:58.619283+0000","last_active":"2024-03-23T14:57:01.518923+0000","last_peered":"2024-03-23T14:57:01.518923+0000","last_clean":"2024-03-23T14:57:01.518923+0000","last_became_active":"2024-03-23T14:56:58.618922+0000","last_became_peered":"2024-03-23T14:56:58.618922+0000","last_unstale":"2024-03-23T14:57:01.518923+0000","last_undegraded":"2024-03-23T14:57:01.518923+0000","last_fullsized":"2024-03-23T14:57:01.518923+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:29:07.131287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.620920+0000","last_change":"2024-03-23T14:56:59.636699+0000","last_active":"2024-03-23T14:57:00.620920+0000","last_peered":"2024-03-23T14:57:00.620920+0000","last_clean":"2024-03-23T14:57:00.620920+0000","last_became_active":"2024-03-23T14:56:59.636376+0000","last_became_peered":"2024-03-23T14:56:59.636376+0000","last_unstale":"2024-03-23T14:57:00.620920+0000","last_undegraded":"2024-03-23T14:57:00.620920+0000","last_fullsized":"2024-03-23T14:57:00.620920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:37:56.745366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.622171+0000","last_change":"2024-03-23T14:56:58.627211+0000","last_active":"2024-03-23T14:57:01.622171+0000","last_peered":"2024-03-23T14:57:01.622171+0000","last_clean":"2024-03-23T14:57:01.622171+0000","last_became_active":"2024-03-23T14:56:58.626779+0000","last_became_peered":"2024-03-23T14:56:58.626779+0000","last_unstale":"2024-03-23T14:57:01.622171+0000","last_undegraded":"2024-03-23T14:57:01.622171+0000","last_fullsized":"2024-03-23T14:57:01.622171+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:01:25.160372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.619749+0000","last_change":"2024-03-23T14:56:59.643640+0000","last_active":"2024-03-23T14:57:00.619749+0000","last_peered":"2024-03-23T14:57:00.619749+0000","last_clean":"2024-03-23T14:57:00.619749+0000","last_became_active":"2024-03-23T14:56:59.643255+0000","last_became_peered":"2024-03-23T14:56:59.643255+0000","last_unstale":"2024-03-23T14:57:00.619749+0000","last_undegraded":"2024-03-23T14:57:00.619749+0000","last_fullsized":"2024-03-23T14:57:00.619749+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:35:26.329408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.529191+0000","last_change":"2024-03-23T14:56:58.623880+0000","last_active":"2024-03-23T14:57:01.529191+0000","last_peered":"2024-03-23T14:57:01.529191+0000","last_clean":"2024-03-23T14:57:01.529191+0000","last_became_active":"2024-03-23T14:56:58.623291+0000","last_became_peered":"2024-03-23T14:56:58.623291+0000","last_unstale":"2024-03-23T14:57:01.529191+0000","last_undegraded":"2024-03-23T14:57:01.529191+0000","last_fullsized":"2024-03-23T14:57:01.529191+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:36:10.376200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.556526+0000","last_change":"2024-03-23T14:56:59.634446+0000","last_active":"2024-03-23T14:57:01.556526+0000","last_peered":"2024-03-23T14:57:01.556526+0000","last_clean":"2024-03-23T14:57:01.556526+0000","last_became_active":"2024-03-23T14:56:59.633946+0000","last_became_peered":"2024-03-23T14:56:59.633946+0000","last_unstale":"2024-03-23T14:57:01.556526+0000","last_undegraded":"2024-03-23T14:57:01.556526+0000","last_fullsized":"2024-03-23T14:57:01.556526+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:09:00.536071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.517807+0000","last_change":"2024-03-23T14:56:58.618776+0000","last_active":"2024-03-23T14:57:01.517807+0000","last_peered":"2024-03-23T14:57:01.517807+0000","last_clean":"2024-03-23T14:57:01.517807+0000","last_became_active":"2024-03-23T14:56:58.618418+0000","last_became_peered":"2024-03-23T14:56:58.618418+0000","last_unstale":"2024-03-23T14:57:01.517807+0000","last_undegraded":"2024-03-23T14:57:01.517807+0000","last_fullsized":"2024-03-23T14:57:01.517807+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:48:12.389105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.619949+0000","last_change":"2024-03-23T14:56:59.633845+0000","last_active":"2024-03-23T14:57:00.619949+0000","last_peered":"2024-03-23T14:57:00.619949+0000","last_clean":"2024-03-23T14:57:00.619949+0000","last_became_active":"2024-03-23T14:56:59.633409+0000","last_became_peered":"2024-03-23T14:56:59.633409+0000","last_unstale":"2024-03-23T14:57:00.619949+0000","last_undegraded":"2024-03-23T14:57:00.619949+0000","last_fullsized":"2024-03-23T14:57:00.619949+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:29:08.139571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.620668+0000","last_change":"2024-03-23T14:56:58.616951+0000","last_active":"2024-03-23T14:57:00.620668+0000","last_peered":"2024-03-23T14:57:00.620668+0000","last_clean":"2024-03-23T14:57:00.620668+0000","last_became_active":"2024-03-23T14:56:58.616383+0000","last_became_peered":"2024-03-23T14:56:58.616383+0000","last_unstale":"2024-03-23T14:57:00.620668+0000","last_undegraded":"2024-03-23T14:57:00.620668+0000","last_fullsized":"2024-03-23T14:57:00.620668+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:48:12.389105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.617942+0000","last_change":"2024-03-23T14:56:59.645384+0000","last_active":"2024-03-23T14:57:00.617942+0000","last_peered":"2024-03-23T14:57:00.617942+0000","last_clean":"2024-03-23T14:57:00.617942+0000","last_became_active":"2024-03-23T14:56:59.643723+0000","last_became_peered":"2024-03-23T14:56:59.643723+0000","last_unstale":"2024-03-23T14:57:00.617942+0000","last_undegraded":"2024-03-23T14:57:00.617942+0000","last_fullsized":"2024-03-23T14:57:00.617942+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:09:00.536071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.620676+0000","last_change":"2024-03-23T14:56:58.622814+0000","last_active":"2024-03-23T14:57:00.620676+0000","last_peered":"2024-03-23T14:57:00.620676+0000","last_clean":"2024-03-23T14:57:00.620676+0000","last_became_active":"2024-03-23T14:56:58.621745+0000","last_became_peered":"2024-03-23T14:56:58.621745+0000","last_unstale":"2024-03-23T14:57:00.620676+0000","last_undegraded":"2024-03-23T14:57:00.620676+0000","last_fullsized":"2024-03-23T14:57:00.620676+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:49:19.974457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.618866+0000","last_change":"2024-03-23T14:56:59.645315+0000","last_active":"2024-03-23T14:57:00.618866+0000","last_peered":"2024-03-23T14:57:00.618866+0000","last_clean":"2024-03-23T14:57:00.618866+0000","last_became_active":"2024-03-23T14:56:59.644287+0000","last_became_peered":"2024-03-23T14:56:59.644287+0000","last_unstale":"2024-03-23T14:57:00.618866+0000","last_undegraded":"2024-03-23T14:57:00.618866+0000","last_fullsized":"2024-03-23T14:57:00.618866+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:57:06.681361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.518700+0000","last_change":"2024-03-23T14:56:58.625848+0000","last_active":"2024-03-23T14:57:01.518700+0000","last_peered":"2024-03-23T14:57:01.518700+0000","last_clean":"2024-03-23T14:57:01.518700+0000","last_became_active":"2024-03-23T14:56:58.625453+0000","last_became_peered":"2024-03-23T14:56:58.625453+0000","last_unstale":"2024-03-23T14:57:01.518700+0000","last_undegraded":"2024-03-23T14:57:01.518700+0000","last_fullsized":"2024-03-23T14:57:01.518700+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:06:05.434722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.529162+0000","last_change":"2024-03-23T14:56:59.635812+0000","last_active":"2024-03-23T14:57:01.529162+0000","last_peered":"2024-03-23T14:57:01.529162+0000","last_clean":"2024-03-23T14:57:01.529162+0000","last_became_active":"2024-03-23T14:56:59.635519+0000","last_became_peered":"2024-03-23T14:56:59.635519+0000","last_unstale":"2024-03-23T14:57:01.529162+0000","last_undegraded":"2024-03-23T14:57:01.529162+0000","last_fullsized":"2024-03-23T14:57:01.529162+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:37:56.745366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.551417+0000","last_change":"2024-03-23T14:56:59.662434+0000","last_active":"2024-03-23T14:57:01.551417+0000","last_peered":"2024-03-23T14:57:01.551417+0000","last_clean":"2024-03-23T14:57:01.551417+0000","last_became_active":"2024-03-23T14:56:59.661514+0000","last_became_peered":"2024-03-23T14:56:59.661514+0000","last_unstale":"2024-03-23T14:57:01.551417+0000","last_undegraded":"2024-03-23T14:57:01.551417+0000","last_fullsized":"2024-03-23T14:57:01.551417+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:34:30.083232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.518270+0000","last_change":"2024-03-23T14:56:58.629744+0000","last_active":"2024-03-23T14:57:01.518270+0000","last_peered":"2024-03-23T14:57:01.518270+0000","last_clean":"2024-03-23T14:57:01.518270+0000","last_became_active":"2024-03-23T14:56:58.629401+0000","last_became_peered":"2024-03-23T14:56:58.629401+0000","last_unstale":"2024-03-23T14:57:01.518270+0000","last_undegraded":"2024-03-23T14:57:01.518270+0000","last_fullsized":"2024-03-23T14:57:01.518270+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:13:45.673576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.551735+0000","last_change":"2024-03-23T14:56:59.658250+0000","last_active":"2024-03-23T14:57:01.551735+0000","last_peered":"2024-03-23T14:57:01.551735+0000","last_clean":"2024-03-23T14:57:01.551735+0000","last_became_active":"2024-03-23T14:56:59.657085+0000","last_became_peered":"2024-03-23T14:56:59.657085+0000","last_unstale":"2024-03-23T14:57:01.551735+0000","last_undegraded":"2024-03-23T14:57:01.551735+0000","last_fullsized":"2024-03-23T14:57:01.551735+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:53:55.152886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.518837+0000","last_change":"2024-03-23T14:56:58.624886+0000","last_active":"2024-03-23T14:57:01.518837+0000","last_peered":"2024-03-23T14:57:01.518837+0000","last_clean":"2024-03-23T14:57:01.518837+0000","last_became_active":"2024-03-23T14:56:58.624478+0000","last_became_peered":"2024-03-23T14:56:58.624478+0000","last_unstale":"2024-03-23T14:57:01.518837+0000","last_undegraded":"2024-03-23T14:57:01.518837+0000","last_fullsized":"2024-03-23T14:57:01.518837+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:49:19.974457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.555868+0000","last_change":"2024-03-23T14:56:59.660089+0000","last_active":"2024-03-23T14:57:01.555868+0000","last_peered":"2024-03-23T14:57:01.555868+0000","last_clean":"2024-03-23T14:57:01.555868+0000","last_became_active":"2024-03-23T14:56:59.659469+0000","last_became_peered":"2024-03-23T14:56:59.659469+0000","last_unstale":"2024-03-23T14:57:01.555868+0000","last_undegraded":"2024-03-23T14:57:01.555868+0000","last_fullsized":"2024-03-23T14:57:01.555868+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:37:14.115808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.619024+0000","last_change":"2024-03-23T14:56:58.608147+0000","last_active":"2024-03-23T14:57:00.619024+0000","last_peered":"2024-03-23T14:57:00.619024+0000","last_clean":"2024-03-23T14:57:00.619024+0000","last_became_active":"2024-03-23T14:56:58.603630+0000","last_became_peered":"2024-03-23T14:56:58.603630+0000","last_unstale":"2024-03-23T14:57:00.619024+0000","last_undegraded":"2024-03-23T14:57:00.619024+0000","last_fullsized":"2024-03-23T14:57:00.619024+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:08:31.251287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.618985+0000","last_change":"2024-03-23T14:56:59.645116+0000","last_active":"2024-03-23T14:57:00.618985+0000","last_peered":"2024-03-23T14:57:00.618985+0000","last_clean":"2024-03-23T14:57:00.618985+0000","last_became_active":"2024-03-23T14:56:59.644484+0000","last_became_peered":"2024-03-23T14:56:59.644484+0000","last_unstale":"2024-03-23T14:57:00.618985+0000","last_undegraded":"2024-03-23T14:57:00.618985+0000","last_fullsized":"2024-03-23T14:57:00.618985+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:39:22.051918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.556818+0000","last_change":"2024-03-23T14:56:58.619314+0000","last_active":"2024-03-23T14:57:01.556818+0000","last_peered":"2024-03-23T14:57:01.556818+0000","last_clean":"2024-03-23T14:57:01.556818+0000","last_became_active":"2024-03-23T14:56:58.618751+0000","last_became_peered":"2024-03-23T14:56:58.618751+0000","last_unstale":"2024-03-23T14:57:01.556818+0000","last_undegraded":"2024-03-23T14:57:01.556818+0000","last_fullsized":"2024-03-23T14:57:01.556818+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:16:22.644801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27800,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344040,"statfs":{"total":96636764160,"available":96608296960,"internally_reserved":0,"allocated":790528,"data_stored":655584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":780,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":798720,"data_stored":655640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-23T14:57:03.336 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-03-23T14:57:03.498 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.494+0000 7f1d46c8f640 1 Processor -- start 2024-03-23T14:57:03.498 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.494+0000 7f1d46c8f640 1 -- start start 2024-03-23T14:57:03.500 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.494+0000 7f1d46c8f640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f1d400b6360 0x7f1d40145a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:03.500 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.494+0000 7f1d46c8f640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1d400b6740 0x7f1d400b41d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:03.500 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.494+0000 7f1d46c8f640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d400b5530 con 0x7f1d40145f60 2024-03-23T14:57:03.500 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.494+0000 7f1d46c8f640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 -- 0x7f1d400b5a60 con 0x7f1d400b6360 2024-03-23T14:57:03.500 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.494+0000 7f1d46c8f640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f1d40071260 con 0x7f1d400b6740 2024-03-23T14:57:03.500 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.494+0000 7f1d44a2b640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f1d400b6360 0x7f1d40145a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:03.500 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.494+0000 7f1d4522c640 1 --1- >> v1:172.21.15.173:6789/0 conn(0x7f1d40145f60 0x7f1d40148860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.173:6789/0 says I am v1:172.21.15.164:35080/0 (socket says 172.21.15.164:35080) 2024-03-23T14:57:03.500 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.494+0000 7f1d4522c640 1 -- 172.21.15.164:0/2206173245 learned_addr learned my addr 172.21.15.164:0/2206173245 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-03-23T14:57:03.501 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.494+0000 7f1d3ffff640 1 --2- 172.21.15.164:0/2206173245 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1d400b6740 0x7f1d400b41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:03.501 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.498+0000 7f1d44a2b640 1 -- 172.21.15.164:0/2206173245 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1d400b6740 msgr2=0x7f1d400b41d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:03.501 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.498+0000 7f1d44a2b640 1 --2- 172.21.15.164:0/2206173245 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1d400b6740 0x7f1d400b41d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:03.501 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.498+0000 7f1d44a2b640 1 -- 172.21.15.164:0/2206173245 >> v1:172.21.15.173:6789/0 conn(0x7f1d40145f60 legacy=0x7f1d40148860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:03.501 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.498+0000 7f1d44a2b640 1 -- 172.21.15.164:0/2206173245 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d400b2a70 con 0x7f1d400b6360 2024-03-23T14:57:03.502 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.498+0000 7f1d44a2b640 1 --2- 172.21.15.164:0/2206173245 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1d400b6360 0x7f1d40145a20 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1d300097c0 tx=0x7f1d3001c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=290f518c3528e3e2 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:03.502 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.498+0000 7f1d3f7fe640 1 -- 172.21.15.164:0/2206173245 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d300a4ea0 con 0x7f1d400b6360 2024-03-23T14:57:03.502 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.498+0000 7f1d3f7fe640 1 -- 172.21.15.164:0/2206173245 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d300a7050 con 0x7f1d400b6360 2024-03-23T14:57:03.503 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.498+0000 7f1d3f7fe640 1 -- 172.21.15.164:0/2206173245 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d300ab820 con 0x7f1d400b6360 2024-03-23T14:57:03.503 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.498+0000 7f1d46c8f640 1 -- 172.21.15.164:0/2206173245 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1d400b6360 msgr2=0x7f1d40145a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:03.503 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.498+0000 7f1d46c8f640 1 --2- 172.21.15.164:0/2206173245 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1d400b6360 0x7f1d40145a20 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1d300097c0 tx=0x7f1d3001c410 comp rx=0 tx=0).stop 2024-03-23T14:57:03.504 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.498+0000 7f1d46c8f640 1 -- 172.21.15.164:0/2206173245 shutdown_connections 2024-03-23T14:57:03.504 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.498+0000 7f1d46c8f640 1 --2- 172.21.15.164:0/2206173245 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1d400b6740 0x7f1d400b41d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:03.504 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.498+0000 7f1d46c8f640 1 --2- 172.21.15.164:0/2206173245 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1d400b6360 0x7f1d40145a20 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:03.504 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.498+0000 7f1d46c8f640 1 -- 172.21.15.164:0/2206173245 >> 172.21.15.164:0/2206173245 conn(0x7f1d400b7630 msgr2=0x7f1d400b7a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:03.504 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.498+0000 7f1d46c8f640 1 -- 172.21.15.164:0/2206173245 shutdown_connections 2024-03-23T14:57:03.504 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.498+0000 7f1d46c8f640 1 -- 172.21.15.164:0/2206173245 wait complete. 2024-03-23T14:57:03.506 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.502+0000 7f1d46c8f640 1 Processor -- start 2024-03-23T14:57:03.506 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.502+0000 7f1d46c8f640 1 -- start start 2024-03-23T14:57:03.506 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.502+0000 7f1d46c8f640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1d400b6360 0x7f1d401373a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:03.506 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.502+0000 7f1d46c8f640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1d400b6740 0x7f1d4006b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:03.506 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.502+0000 7f1d46c8f640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1d40145f60 0x7f1d4013dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:03.506 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.502+0000 7f1d46c8f640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1d401391d0 con 0x7f1d400b6740 2024-03-23T14:57:03.506 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.502+0000 7f1d46c8f640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f1d4013a820 con 0x7f1d40145f60 2024-03-23T14:57:03.506 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.502+0000 7f1d46c8f640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f1d401385f0 con 0x7f1d400b6360 2024-03-23T14:57:03.506 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.502+0000 7f1d3ffff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1d400b6740 0x7f1d4006b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:03.506 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.502+0000 7f1d3ffff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1d400b6740 0x7f1d4006b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:59002/0 (socket says 172.21.15.164:59002) 2024-03-23T14:57:03.507 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.502+0000 7f1d3ffff640 1 -- 172.21.15.164:0/2303509174 learned_addr learned my addr 172.21.15.164:0/2303509174 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:03.507 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.502+0000 7f1d44a2b640 1 --2- 172.21.15.164:0/2303509174 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1d400b6360 0x7f1d401373a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:03.507 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.502+0000 7f1d4522c640 1 --2- 172.21.15.164:0/2303509174 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1d40145f60 0x7f1d4013dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:03.507 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.502+0000 7f1d3ffff640 1 -- 172.21.15.164:0/2303509174 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1d400b6360 msgr2=0x7f1d401373a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:03.507 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.502+0000 7f1d3ffff640 1 --2- 172.21.15.164:0/2303509174 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1d400b6360 0x7f1d401373a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:03.507 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.502+0000 7f1d3ffff640 1 -- 172.21.15.164:0/2303509174 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1d40145f60 msgr2=0x7f1d4013dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:03.507 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.502+0000 7f1d3ffff640 1 --2- 172.21.15.164:0/2303509174 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1d40145f60 0x7f1d4013dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:03.507 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.502+0000 7f1d3ffff640 1 -- 172.21.15.164:0/2303509174 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d30009420 con 0x7f1d400b6740 2024-03-23T14:57:03.507 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.502+0000 7f1d3ffff640 1 --2- 172.21.15.164:0/2303509174 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1d400b6740 0x7f1d4006b1a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1d3400e960 tx=0x7f1d3400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:03.507 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.502+0000 7f1d3d7fa640 1 -- 172.21.15.164:0/2303509174 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d34009690 con 0x7f1d400b6740 2024-03-23T14:57:03.507 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.502+0000 7f1d3d7fa640 1 -- 172.21.15.164:0/2303509174 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d3400ca00 con 0x7f1d400b6740 2024-03-23T14:57:03.507 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.502+0000 7f1d46c8f640 1 -- 172.21.15.164:0/2303509174 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d4013f340 con 0x7f1d400b6740 2024-03-23T14:57:03.507 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.502+0000 7f1d3d7fa640 1 -- 172.21.15.164:0/2303509174 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d3400cd70 con 0x7f1d400b6740 2024-03-23T14:57:03.507 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.502+0000 7f1d46c8f640 1 -- 172.21.15.164:0/2303509174 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d4013c140 con 0x7f1d400b6740 2024-03-23T14:57:03.508 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.502+0000 7f1d46c8f640 1 -- 172.21.15.164:0/2303509174 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d400690a0 con 0x7f1d400b6740 2024-03-23T14:57:03.508 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.502+0000 7f1d3d7fa640 1 -- 172.21.15.164:0/2303509174 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7f1d34013470 con 0x7f1d400b6740 2024-03-23T14:57:03.508 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.506+0000 7f1d3d7fa640 1 --2- 172.21.15.164:0/2303509174 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f1d2c0672a0 0x7f1d2c069760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:03.508 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.506+0000 7f1d44a2b640 1 --2- 172.21.15.164:0/2303509174 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f1d2c0672a0 0x7f1d2c069760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:03.509 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.506+0000 7f1d3d7fa640 1 -- 172.21.15.164:0/2303509174 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f1d34085280 con 0x7f1d400b6740 2024-03-23T14:57:03.509 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.506+0000 7f1d44a2b640 1 --2- 172.21.15.164:0/2303509174 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f1d2c0672a0 0x7f1d2c069760 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f1d300ad000 tx=0x7f1d300a6000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:03.512 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.510+0000 7f1d3d7fa640 1 -- 172.21.15.164:0/2303509174 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7f1d34051e20 con 0x7f1d400b6740 2024-03-23T14:57:03.714 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.710+0000 7f1d46c8f640 1 -- 172.21.15.164:0/2303509174 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f1d400b44f0 con 0x7f1d400b6740 2024-03-23T14:57:03.714 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.710+0000 7f1d3d7fa640 1 -- 172.21.15.164:0/2303509174 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1011 (secure 0 0 0) 0x7f1d340554e0 con 0x7f1d400b6740 2024-03-23T14:57:03.715 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-23T14:57:03.715 INFO:teuthology.orchestra.run.smithi164.stdout:{"fsid":"45c7911b-b69d-4e68-907b-fbfeb780aa3b","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":29,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":20,"num_osds":8,"num_up_osds":8,"osd_up_since":1711205806,"num_in_osds":8,"osd_in_since":1711205796,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":24,"data_bytes":592778,"bytes_used":224256000,"bytes_avail":772869857280,"bytes_total":773094113280,"write_bytes_sec":2594,"read_op_per_sec":0,"write_op_per_sec":8},"fsmap":{"epoch":6,"id":1,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":1,"rank":0,"name":"b","status":"up:active","gid":4279}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":1,"modified":"2024-03-23T14:56:34.370184+0000","services":{}},"progress_events":{}} 2024-03-23T14:57:03.717 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.714+0000 7f1d46c8f640 1 -- 172.21.15.164:0/2303509174 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f1d2c0672a0 msgr2=0x7f1d2c069760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:03.717 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.714+0000 7f1d46c8f640 1 --2- 172.21.15.164:0/2303509174 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f1d2c0672a0 0x7f1d2c069760 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f1d300ad000 tx=0x7f1d300a6000 comp rx=0 tx=0).stop 2024-03-23T14:57:03.717 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.714+0000 7f1d46c8f640 1 -- 172.21.15.164:0/2303509174 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1d400b6740 msgr2=0x7f1d4006b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:03.717 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.714+0000 7f1d46c8f640 1 --2- 172.21.15.164:0/2303509174 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1d400b6740 0x7f1d4006b1a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1d3400e960 tx=0x7f1d3400ee20 comp rx=0 tx=0).stop 2024-03-23T14:57:03.717 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.714+0000 7f1d46c8f640 1 -- 172.21.15.164:0/2303509174 shutdown_connections 2024-03-23T14:57:03.717 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.714+0000 7f1d46c8f640 1 --2- 172.21.15.164:0/2303509174 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f1d2c0672a0 0x7f1d2c069760 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:03.717 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.714+0000 7f1d46c8f640 1 --2- 172.21.15.164:0/2303509174 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1d40145f60 0x7f1d4013dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:03.717 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.714+0000 7f1d46c8f640 1 --2- 172.21.15.164:0/2303509174 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1d400b6740 0x7f1d4006b1a0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:03.717 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.714+0000 7f1d46c8f640 1 --2- 172.21.15.164:0/2303509174 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1d400b6360 0x7f1d401373a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:03.717 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.714+0000 7f1d46c8f640 1 -- 172.21.15.164:0/2303509174 >> 172.21.15.164:0/2303509174 conn(0x7f1d400b7630 msgr2=0x7f1d400713b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:03.717 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.714+0000 7f1d46c8f640 1 -- 172.21.15.164:0/2303509174 shutdown_connections 2024-03-23T14:57:03.717 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.714+0000 7f1d46c8f640 1 -- 172.21.15.164:0/2303509174 wait complete. 2024-03-23T14:57:03.732 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-03-23T14:57:03.732 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-23T14:57:03.851 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.846+0000 7f8c57240640 1 Processor -- start 2024-03-23T14:57:03.852 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.846+0000 7f8c57240640 1 -- start start 2024-03-23T14:57:03.852 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.850+0000 7f8c57240640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8c500b6360 0x7f8c50145a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:03.852 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.850+0000 7f8c57240640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c500b5530 con 0x7f8c50145f60 2024-03-23T14:57:03.853 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.850+0000 7f8c57240640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c500b2a70 con 0x7f8c500b6740 2024-03-23T14:57:03.853 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.850+0000 7f8c57240640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f8c500b5a60 con 0x7f8c500b6360 2024-03-23T14:57:03.853 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.850+0000 7f8c557dd640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f8c50145f60 0x7f8c5014fb90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:42386/0 (socket says 172.21.15.164:42386) 2024-03-23T14:57:03.853 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.850+0000 7f8c557dd640 1 -- 172.21.15.164:0/315914331 learned_addr learned my addr 172.21.15.164:0/315914331 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-03-23T14:57:03.854 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.850+0000 7f8c54fdc640 1 --2- 172.21.15.164:0/315914331 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8c500b6360 0x7f8c50145a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:03.855 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.850+0000 7f8c4f7fe640 1 -- 172.21.15.164:0/315914331 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3842314686 0 0) 0x7f8c500b5530 con 0x7f8c50145f60 2024-03-23T14:57:03.855 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.850+0000 7f8c4f7fe640 1 -- 172.21.15.164:0/315914331 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8c40003530 con 0x7f8c50145f60 2024-03-23T14:57:03.855 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.850+0000 7f8c4f7fe640 1 -- 172.21.15.164:0/315914331 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 747465922 0 0) 0x7f8c40003530 con 0x7f8c50145f60 2024-03-23T14:57:03.855 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.850+0000 7f8c4f7fe640 1 -- 172.21.15.164:0/315914331 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8c500b6360 msgr2=0x7f8c50145a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:03.855 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.850+0000 7f8c4f7fe640 1 --2- 172.21.15.164:0/315914331 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8c500b6360 0x7f8c50145a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:03.855 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.850+0000 7f8c4f7fe640 1 -- 172.21.15.164:0/315914331 >> v1:172.21.15.173:6789/0 conn(0x7f8c500b6740 legacy=0x7f8c500b41d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:03.855 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.850+0000 7f8c4f7fe640 1 -- 172.21.15.164:0/315914331 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c500b2800 con 0x7f8c50145f60 2024-03-23T14:57:03.855 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.850+0000 7f8c4f7fe640 1 -- 172.21.15.164:0/315914331 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2526390548 0 0) 0x7f8c48001a00 con 0x7f8c50145f60 2024-03-23T14:57:03.855 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.850+0000 7f8c4f7fe640 1 -- 172.21.15.164:0/315914331 >> v1:172.21.15.164:6789/0 conn(0x7f8c50145f60 legacy=0x7f8c5014fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:03.855 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.850+0000 7f8c4f7fe640 1 --2- 172.21.15.164:0/315914331 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8c400039b0 0x7f8c40003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:03.855 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.850+0000 7f8c4f7fe640 1 --2- 172.21.15.164:0/315914331 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8c400047d0 0x7f8c40004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:03.855 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.850+0000 7f8c4f7fe640 1 --2- 172.21.15.164:0/315914331 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8c500b6360 0x7f8c400051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:03.855 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.850+0000 7f8c4f7fe640 1 -- 172.21.15.164:0/315914331 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8c500b5a60 con 0x7f8c400047d0 2024-03-23T14:57:03.855 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.850+0000 7f8c4f7fe640 1 -- 172.21.15.164:0/315914331 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f8c4000bac0 con 0x7f8c400039b0 2024-03-23T14:57:03.856 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.850+0000 7f8c4f7fe640 1 -- 172.21.15.164:0/315914331 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f8c4000bc80 con 0x7f8c500b6360 2024-03-23T14:57:03.856 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.850+0000 7f8c4ffff640 1 --2- 172.21.15.164:0/315914331 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8c400047d0 0x7f8c40004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:03.856 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.850+0000 7f8c54fdc640 1 --2- 172.21.15.164:0/315914331 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8c400039b0 0x7f8c40003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:03.856 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.850+0000 7f8c557dd640 1 --2- 172.21.15.164:0/315914331 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8c500b6360 0x7f8c400051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:03.856 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.850+0000 7f8c4ffff640 1 -- 172.21.15.164:0/315914331 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8c500b6360 msgr2=0x7f8c400051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:03.856 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.850+0000 7f8c4ffff640 1 --2- 172.21.15.164:0/315914331 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8c500b6360 0x7f8c400051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:03.856 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.850+0000 7f8c4ffff640 1 -- 172.21.15.164:0/315914331 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8c400039b0 msgr2=0x7f8c40003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:03.856 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.850+0000 7f8c4ffff640 1 --2- 172.21.15.164:0/315914331 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8c400039b0 0x7f8c40003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:03.856 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.850+0000 7f8c4ffff640 1 -- 172.21.15.164:0/315914331 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c40003530 con 0x7f8c400047d0 2024-03-23T14:57:03.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.854+0000 7f8c4ffff640 1 --2- 172.21.15.164:0/315914331 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8c400047d0 0x7f8c40004c50 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c001780 tx=0x7f8c3c01c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77f1469d22e2c54e server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:03.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.854+0000 7f8c4f7fe640 1 -- 172.21.15.164:0/315914331 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c3c0052d0 con 0x7f8c400047d0 2024-03-23T14:57:03.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.854+0000 7f8c4f7fe640 1 -- 172.21.15.164:0/315914331 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c3c0a7050 con 0x7f8c400047d0 2024-03-23T14:57:03.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.854+0000 7f8c4f7fe640 1 -- 172.21.15.164:0/315914331 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c3c003600 con 0x7f8c400047d0 2024-03-23T14:57:03.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.854+0000 7f8c57240640 1 -- 172.21.15.164:0/315914331 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8c400047d0 msgr2=0x7f8c40004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:03.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.854+0000 7f8c57240640 1 --2- 172.21.15.164:0/315914331 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8c400047d0 0x7f8c40004c50 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c001780 tx=0x7f8c3c01c420 comp rx=0 tx=0).stop 2024-03-23T14:57:03.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.854+0000 7f8c557dd640 1 -- 172.21.15.164:0/315914331 reap_dead start 2024-03-23T14:57:03.858 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.854+0000 7f8c57240640 1 -- 172.21.15.164:0/315914331 shutdown_connections 2024-03-23T14:57:03.858 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.854+0000 7f8c57240640 1 -- 172.21.15.164:0/315914331 >> 172.21.15.164:0/315914331 conn(0x7f8c500b7630 msgr2=0x7f8c500b7a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:03.858 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.854+0000 7f8c57240640 1 -- 172.21.15.164:0/315914331 shutdown_connections 2024-03-23T14:57:03.858 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.854+0000 7f8c57240640 1 -- 172.21.15.164:0/315914331 wait complete. 2024-03-23T14:57:03.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.854+0000 7f8c57240640 1 Processor -- start 2024-03-23T14:57:03.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.854+0000 7f8c57240640 1 -- start start 2024-03-23T14:57:03.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.854+0000 7f8c57240640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8c400047d0 0x7f8c500b9640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:03.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.854+0000 7f8c57240640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8c500b1200 0x7f8c50146450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:03.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.854+0000 7f8c57240640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8c50146990 0x7f8c50140c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:03.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.854+0000 7f8c57240640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8c50147c40 con 0x7f8c500b1200 2024-03-23T14:57:03.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.854+0000 7f8c57240640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f8c50148390 con 0x7f8c50146990 2024-03-23T14:57:03.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.854+0000 7f8c57240640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f8c50146e10 con 0x7f8c400047d0 2024-03-23T14:57:03.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.854+0000 7f8c4ffff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8c500b1200 0x7f8c50146450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:03.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.854+0000 7f8c4ffff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8c500b1200 0x7f8c50146450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:59010/0 (socket says 172.21.15.164:59010) 2024-03-23T14:57:03.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.854+0000 7f8c4ffff640 1 -- 172.21.15.164:0/4138291801 learned_addr learned my addr 172.21.15.164:0/4138291801 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:03.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.854+0000 7f8c54fdc640 1 --2- 172.21.15.164:0/4138291801 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8c400047d0 0x7f8c500b9640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:03.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.854+0000 7f8c557dd640 1 --2- 172.21.15.164:0/4138291801 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8c50146990 0x7f8c50140c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:03.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.854+0000 7f8c4ffff640 1 -- 172.21.15.164:0/4138291801 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8c400047d0 msgr2=0x7f8c500b9640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:03.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.854+0000 7f8c4ffff640 1 --2- 172.21.15.164:0/4138291801 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8c400047d0 0x7f8c500b9640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:03.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.854+0000 7f8c4ffff640 1 -- 172.21.15.164:0/4138291801 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8c50146990 msgr2=0x7f8c50140c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:03.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.854+0000 7f8c4ffff640 1 --2- 172.21.15.164:0/4138291801 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8c50146990 0x7f8c50140c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:03.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.854+0000 7f8c4ffff640 1 -- 172.21.15.164:0/4138291801 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c50148ff0 con 0x7f8c500b1200 2024-03-23T14:57:03.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.854+0000 7f8c54fdc640 1 --2- 172.21.15.164:0/4138291801 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8c400047d0 0x7f8c500b9640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T14:57:03.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.854+0000 7f8c557dd640 1 --2- 172.21.15.164:0/4138291801 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8c50146990 0x7f8c50140c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T14:57:03.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.854+0000 7f8c4ffff640 1 --2- 172.21.15.164:0/4138291801 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8c500b1200 0x7f8c50146450 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c005040 tx=0x7f8c40005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:03.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.854+0000 7f8c4d7fa640 1 -- 172.21.15.164:0/4138291801 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c3c007180 con 0x7f8c500b1200 2024-03-23T14:57:03.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.854+0000 7f8c4d7fa640 1 -- 172.21.15.164:0/4138291801 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c3c0072e0 con 0x7f8c500b1200 2024-03-23T14:57:03.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.854+0000 7f8c57240640 1 -- 172.21.15.164:0/4138291801 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c50147580 con 0x7f8c500b1200 2024-03-23T14:57:03.862 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.854+0000 7f8c4d7fa640 1 -- 172.21.15.164:0/4138291801 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c3c007670 con 0x7f8c500b1200 2024-03-23T14:57:03.862 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.854+0000 7f8c57240640 1 -- 172.21.15.164:0/4138291801 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c50142c80 con 0x7f8c500b1200 2024-03-23T14:57:03.862 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.858+0000 7f8c57240640 1 -- 172.21.15.164:0/4138291801 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c50147d80 con 0x7f8c500b1200 2024-03-23T14:57:03.862 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.858+0000 7f8c4d7fa640 1 -- 172.21.15.164:0/4138291801 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7f8c3c0067e0 con 0x7f8c500b1200 2024-03-23T14:57:03.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.858+0000 7f8c4d7fa640 1 --2- 172.21.15.164:0/4138291801 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f8c340672d0 0x7f8c34069790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:03.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.858+0000 7f8c4d7fa640 1 -- 172.21.15.164:0/4138291801 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f8c3c11f2c0 con 0x7f8c500b1200 2024-03-23T14:57:03.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.858+0000 7f8c54fdc640 1 --2- 172.21.15.164:0/4138291801 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f8c340672d0 0x7f8c34069790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:03.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.858+0000 7f8c54fdc640 1 --2- 172.21.15.164:0/4138291801 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f8c340672d0 0x7f8c34069790 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8c44009a20 tx=0x7f8c44008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:03.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:03.862+0000 7f8c4d7fa640 1 -- 172.21.15.164:0/4138291801 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7f8c50147d80 con 0x7f8c500b1200 2024-03-23T14:57:04.019 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:04.014+0000 7f8c57240640 1 -- 172.21.15.164:0/4138291801 --> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8c50151ac0 con 0x7f8c340672d0 2024-03-23T14:57:04.028 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:04.026+0000 7f8c4d7fa640 1 -- 172.21.15.164:0/4138291801 <== mgr.4107 v2:172.21.15.164:6832/2139087985 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251952 (secure 0 0 0) 0x7f8c44012950 con 0x7f8c340672d0 2024-03-23T14:57:04.031 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-23T14:57:04.032 INFO:teuthology.orchestra.run.smithi164.stderr:dumped all 2024-03-23T14:57:04.033 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:04.030+0000 7f8c57240640 1 -- 172.21.15.164:0/4138291801 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f8c340672d0 msgr2=0x7f8c34069790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:04.033 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:04.030+0000 7f8c57240640 1 --2- 172.21.15.164:0/4138291801 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f8c340672d0 0x7f8c34069790 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8c44009a20 tx=0x7f8c44008040 comp rx=0 tx=0).stop 2024-03-23T14:57:04.033 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:04.030+0000 7f8c57240640 1 -- 172.21.15.164:0/4138291801 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8c500b1200 msgr2=0x7f8c50146450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:04.033 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:04.030+0000 7f8c57240640 1 --2- 172.21.15.164:0/4138291801 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8c500b1200 0x7f8c50146450 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c005040 tx=0x7f8c40005800 comp rx=0 tx=0).stop 2024-03-23T14:57:04.033 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:04.030+0000 7f8c57240640 1 -- 172.21.15.164:0/4138291801 shutdown_connections 2024-03-23T14:57:04.033 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:04.030+0000 7f8c57240640 1 --2- 172.21.15.164:0/4138291801 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f8c340672d0 0x7f8c34069790 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:04.033 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:04.030+0000 7f8c57240640 1 --2- 172.21.15.164:0/4138291801 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8c50146990 0x7f8c50140c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:04.033 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:04.030+0000 7f8c57240640 1 --2- 172.21.15.164:0/4138291801 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8c500b1200 0x7f8c50146450 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:04.033 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:04.030+0000 7f8c57240640 1 --2- 172.21.15.164:0/4138291801 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8c400047d0 0x7f8c500b9640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:04.034 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:04.030+0000 7f8c57240640 1 -- 172.21.15.164:0/4138291801 >> 172.21.15.164:0/4138291801 conn(0x7f8c500b7630 msgr2=0x7f8c5013ee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:04.034 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:04.030+0000 7f8c57240640 1 -- 172.21.15.164:0/4138291801 shutdown_connections 2024-03-23T14:57:04.034 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:04.030+0000 7f8c57240640 1 -- 172.21.15.164:0/4138291801 wait complete. 2024-03-23T14:57:04.049 INFO:teuthology.orchestra.run.smithi164.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-03-23T14:57:03.110303+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":185,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219000,"kb_used_data":2800,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755720,"statfs":{"total":773094113280,"available":772869857280,"internally_reserved":0,"allocated":2867200,"data_stored":1740428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.525781"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.518848+0000","last_change":"2024-03-23T14:56:59.652438+0000","last_active":"2024-03-23T14:57:01.518848+0000","last_peered":"2024-03-23T14:57:01.518848+0000","last_clean":"2024-03-23T14:57:01.518848+0000","last_became_active":"2024-03-23T14:56:59.652169+0000","last_became_peered":"2024-03-23T14:56:59.652169+0000","last_unstale":"2024-03-23T14:57:01.518848+0000","last_undegraded":"2024-03-23T14:57:01.518848+0000","last_fullsized":"2024-03-23T14:57:01.518848+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:58:31.542399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.621127+0000","last_change":"2024-03-23T14:56:59.632308+0000","last_active":"2024-03-23T14:57:00.621127+0000","last_peered":"2024-03-23T14:57:00.621127+0000","last_clean":"2024-03-23T14:57:00.621127+0000","last_became_active":"2024-03-23T14:56:59.631895+0000","last_became_peered":"2024-03-23T14:56:59.631895+0000","last_unstale":"2024-03-23T14:57:00.621127+0000","last_undegraded":"2024-03-23T14:57:00.621127+0000","last_fullsized":"2024-03-23T14:57:00.621127+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:59:19.727729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.555798+0000","last_change":"2024-03-23T14:56:59.636862+0000","last_active":"2024-03-23T14:57:01.555798+0000","last_peered":"2024-03-23T14:57:01.555798+0000","last_clean":"2024-03-23T14:57:01.555798+0000","last_became_active":"2024-03-23T14:56:59.636209+0000","last_became_peered":"2024-03-23T14:56:59.636209+0000","last_unstale":"2024-03-23T14:57:01.555798+0000","last_undegraded":"2024-03-23T14:57:01.555798+0000","last_fullsized":"2024-03-23T14:57:01.555798+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:11:59.565543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.551115+0000","last_change":"2024-03-23T14:56:59.662732+0000","last_active":"2024-03-23T14:57:01.551115+0000","last_peered":"2024-03-23T14:57:01.551115+0000","last_clean":"2024-03-23T14:57:01.551115+0000","last_became_active":"2024-03-23T14:56:59.661656+0000","last_became_peered":"2024-03-23T14:56:59.661656+0000","last_unstale":"2024-03-23T14:57:01.551115+0000","last_undegraded":"2024-03-23T14:57:01.551115+0000","last_fullsized":"2024-03-23T14:57:01.551115+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:35:32.710122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.518258+0000","last_change":"2024-03-23T14:56:59.651141+0000","last_active":"2024-03-23T14:57:01.518258+0000","last_peered":"2024-03-23T14:57:01.518258+0000","last_clean":"2024-03-23T14:57:01.518258+0000","last_became_active":"2024-03-23T14:56:59.650922+0000","last_became_peered":"2024-03-23T14:56:59.650922+0000","last_unstale":"2024-03-23T14:57:01.518258+0000","last_undegraded":"2024-03-23T14:57:01.518258+0000","last_fullsized":"2024-03-23T14:57:01.518258+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:14:04.996547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.551288+0000","last_change":"2024-03-23T14:56:59.662551+0000","last_active":"2024-03-23T14:57:01.551288+0000","last_peered":"2024-03-23T14:57:01.551288+0000","last_clean":"2024-03-23T14:57:01.551288+0000","last_became_active":"2024-03-23T14:56:59.661732+0000","last_became_peered":"2024-03-23T14:56:59.661732+0000","last_unstale":"2024-03-23T14:57:01.551288+0000","last_undegraded":"2024-03-23T14:57:01.551288+0000","last_fullsized":"2024-03-23T14:57:01.551288+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:35:26.329408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.518244+0000","last_change":"2024-03-23T14:56:59.646745+0000","last_active":"2024-03-23T14:57:01.518244+0000","last_peered":"2024-03-23T14:57:01.518244+0000","last_clean":"2024-03-23T14:57:01.518244+0000","last_became_active":"2024-03-23T14:56:59.646354+0000","last_became_peered":"2024-03-23T14:56:59.646354+0000","last_unstale":"2024-03-23T14:57:01.518244+0000","last_undegraded":"2024-03-23T14:57:01.518244+0000","last_fullsized":"2024-03-23T14:57:01.518244+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:58:40.996298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.617783+0000","last_change":"2024-03-23T14:56:59.659502+0000","last_active":"2024-03-23T14:57:00.617783+0000","last_peered":"2024-03-23T14:57:00.617783+0000","last_clean":"2024-03-23T14:57:00.617783+0000","last_became_active":"2024-03-23T14:56:59.659222+0000","last_became_peered":"2024-03-23T14:56:59.659222+0000","last_unstale":"2024-03-23T14:57:00.617783+0000","last_undegraded":"2024-03-23T14:57:00.617783+0000","last_fullsized":"2024-03-23T14:57:00.617783+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:07:26.887157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.519017+0000","last_change":"2024-03-23T14:56:59.659935+0000","last_active":"2024-03-23T14:57:01.519017+0000","last_peered":"2024-03-23T14:57:01.519017+0000","last_clean":"2024-03-23T14:57:01.519017+0000","last_became_active":"2024-03-23T14:56:59.659501+0000","last_became_peered":"2024-03-23T14:56:59.659501+0000","last_unstale":"2024-03-23T14:57:01.519017+0000","last_undegraded":"2024-03-23T14:57:01.519017+0000","last_fullsized":"2024-03-23T14:57:01.519017+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:21:16.376914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.529493+0000","last_change":"2024-03-23T14:56:59.659013+0000","last_active":"2024-03-23T14:57:01.529493+0000","last_peered":"2024-03-23T14:57:01.529493+0000","last_clean":"2024-03-23T14:57:01.529493+0000","last_became_active":"2024-03-23T14:56:59.658667+0000","last_became_peered":"2024-03-23T14:56:59.658667+0000","last_unstale":"2024-03-23T14:57:01.529493+0000","last_undegraded":"2024-03-23T14:57:01.529493+0000","last_fullsized":"2024-03-23T14:57:01.529493+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:12:59.763661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.529535+0000","last_change":"2024-03-23T14:56:59.663729+0000","last_active":"2024-03-23T14:57:01.529535+0000","last_peered":"2024-03-23T14:57:01.529535+0000","last_clean":"2024-03-23T14:57:01.529535+0000","last_became_active":"2024-03-23T14:56:59.663313+0000","last_became_peered":"2024-03-23T14:56:59.663313+0000","last_unstale":"2024-03-23T14:57:01.529535+0000","last_undegraded":"2024-03-23T14:57:01.529535+0000","last_fullsized":"2024-03-23T14:57:01.529535+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:57:06.681361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.619818+0000","last_change":"2024-03-23T14:56:59.640819+0000","last_active":"2024-03-23T14:57:00.619818+0000","last_peered":"2024-03-23T14:57:00.619818+0000","last_clean":"2024-03-23T14:57:00.619818+0000","last_became_active":"2024-03-23T14:56:59.640330+0000","last_became_peered":"2024-03-23T14:56:59.640330+0000","last_unstale":"2024-03-23T14:57:00.619818+0000","last_undegraded":"2024-03-23T14:57:00.619818+0000","last_fullsized":"2024-03-23T14:57:00.619818+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:01:26.168657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.618487+0000","last_change":"2024-03-23T14:56:59.645597+0000","last_active":"2024-03-23T14:57:00.618487+0000","last_peered":"2024-03-23T14:57:00.618487+0000","last_clean":"2024-03-23T14:57:00.618487+0000","last_became_active":"2024-03-23T14:56:59.644700+0000","last_became_peered":"2024-03-23T14:56:59.644700+0000","last_unstale":"2024-03-23T14:57:00.618487+0000","last_undegraded":"2024-03-23T14:57:00.618487+0000","last_fullsized":"2024-03-23T14:57:00.618487+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:59:19.727729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.556462+0000","last_change":"2024-03-23T14:56:59.636241+0000","last_active":"2024-03-23T14:57:01.556462+0000","last_peered":"2024-03-23T14:57:01.556462+0000","last_clean":"2024-03-23T14:57:01.556462+0000","last_became_active":"2024-03-23T14:56:59.635803+0000","last_became_peered":"2024-03-23T14:56:59.635803+0000","last_unstale":"2024-03-23T14:57:01.556462+0000","last_undegraded":"2024-03-23T14:57:01.556462+0000","last_fullsized":"2024-03-23T14:57:01.556462+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:07:26.887157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.620719+0000","last_change":"2024-03-23T14:56:59.642239+0000","last_active":"2024-03-23T14:57:00.620719+0000","last_peered":"2024-03-23T14:57:00.620719+0000","last_clean":"2024-03-23T14:57:00.620719+0000","last_became_active":"2024-03-23T14:56:59.641872+0000","last_became_peered":"2024-03-23T14:56:59.641872+0000","last_unstale":"2024-03-23T14:57:00.620719+0000","last_undegraded":"2024-03-23T14:57:00.620719+0000","last_fullsized":"2024-03-23T14:57:00.620719+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:09:00.536071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.551044+0000","last_change":"2024-03-23T14:56:59.634551+0000","last_active":"2024-03-23T14:57:01.551044+0000","last_peered":"2024-03-23T14:57:01.551044+0000","last_clean":"2024-03-23T14:57:01.551044+0000","last_became_active":"2024-03-23T14:56:59.634255+0000","last_became_peered":"2024-03-23T14:56:59.634255+0000","last_unstale":"2024-03-23T14:57:01.551044+0000","last_undegraded":"2024-03-23T14:57:01.551044+0000","last_fullsized":"2024-03-23T14:57:01.551044+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:39:22.051918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.552128+0000","last_change":"2024-03-23T14:56:59.657535+0000","last_active":"2024-03-23T14:57:01.552128+0000","last_peered":"2024-03-23T14:57:01.552128+0000","last_clean":"2024-03-23T14:57:01.552128+0000","last_became_active":"2024-03-23T14:56:59.656959+0000","last_became_peered":"2024-03-23T14:56:59.656959+0000","last_unstale":"2024-03-23T14:57:01.552128+0000","last_undegraded":"2024-03-23T14:57:01.552128+0000","last_fullsized":"2024-03-23T14:57:01.552128+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:58:31.542399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.555866+0000","last_change":"2024-03-23T14:56:59.639104+0000","last_active":"2024-03-23T14:57:01.555866+0000","last_peered":"2024-03-23T14:57:01.555866+0000","last_clean":"2024-03-23T14:57:01.555866+0000","last_became_active":"2024-03-23T14:56:59.638744+0000","last_became_peered":"2024-03-23T14:56:59.638744+0000","last_unstale":"2024-03-23T14:57:01.555866+0000","last_undegraded":"2024-03-23T14:57:01.555866+0000","last_fullsized":"2024-03-23T14:57:01.555866+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:29:08.139571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.518933+0000","last_change":"2024-03-23T14:56:59.647334+0000","last_active":"2024-03-23T14:57:01.518933+0000","last_peered":"2024-03-23T14:57:01.518933+0000","last_clean":"2024-03-23T14:57:01.518933+0000","last_became_active":"2024-03-23T14:56:59.646801+0000","last_became_peered":"2024-03-23T14:56:59.646801+0000","last_unstale":"2024-03-23T14:57:01.518933+0000","last_undegraded":"2024-03-23T14:57:01.518933+0000","last_fullsized":"2024-03-23T14:57:01.518933+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:02:12.733995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.529380+0000","last_change":"2024-03-23T14:56:59.639076+0000","last_active":"2024-03-23T14:57:01.529380+0000","last_peered":"2024-03-23T14:57:01.529380+0000","last_clean":"2024-03-23T14:57:01.529380+0000","last_became_active":"2024-03-23T14:56:59.638267+0000","last_became_peered":"2024-03-23T14:56:59.638267+0000","last_unstale":"2024-03-23T14:57:01.529380+0000","last_undegraded":"2024-03-23T14:57:01.529380+0000","last_fullsized":"2024-03-23T14:57:01.529380+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:59:19.727729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.556655+0000","last_change":"2024-03-23T14:56:59.647220+0000","last_active":"2024-03-23T14:57:01.556655+0000","last_peered":"2024-03-23T14:57:01.556655+0000","last_clean":"2024-03-23T14:57:01.556655+0000","last_became_active":"2024-03-23T14:56:59.646833+0000","last_became_peered":"2024-03-23T14:56:59.646833+0000","last_unstale":"2024-03-23T14:57:01.556655+0000","last_undegraded":"2024-03-23T14:57:01.556655+0000","last_fullsized":"2024-03-23T14:57:01.556655+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:38:24.739838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.551049+0000","last_change":"2024-03-23T14:56:59.662969+0000","last_active":"2024-03-23T14:57:01.551049+0000","last_peered":"2024-03-23T14:57:01.551049+0000","last_clean":"2024-03-23T14:57:01.551049+0000","last_became_active":"2024-03-23T14:56:59.661554+0000","last_became_peered":"2024-03-23T14:56:59.661554+0000","last_unstale":"2024-03-23T14:57:01.551049+0000","last_undegraded":"2024-03-23T14:57:01.551049+0000","last_fullsized":"2024-03-23T14:57:01.551049+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:52:36.323039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.618010+0000","last_change":"2024-03-23T14:56:59.643866+0000","last_active":"2024-03-23T14:57:00.618010+0000","last_peered":"2024-03-23T14:57:00.618010+0000","last_clean":"2024-03-23T14:57:00.618010+0000","last_became_active":"2024-03-23T14:56:59.643577+0000","last_became_peered":"2024-03-23T14:56:59.643577+0000","last_unstale":"2024-03-23T14:57:00.618010+0000","last_undegraded":"2024-03-23T14:57:00.618010+0000","last_fullsized":"2024-03-23T14:57:00.618010+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:37:56.745366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.550754+0000","last_change":"2024-03-23T14:56:59.651798+0000","last_active":"2024-03-23T14:57:01.550754+0000","last_peered":"2024-03-23T14:57:01.550754+0000","last_clean":"2024-03-23T14:57:01.550754+0000","last_became_active":"2024-03-23T14:56:59.651534+0000","last_became_peered":"2024-03-23T14:56:59.651534+0000","last_unstale":"2024-03-23T14:57:01.550754+0000","last_undegraded":"2024-03-23T14:57:01.550754+0000","last_fullsized":"2024-03-23T14:57:01.550754+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:57:06.681361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.619373+0000","last_change":"2024-03-23T14:56:59.645160+0000","last_active":"2024-03-23T14:57:00.619373+0000","last_peered":"2024-03-23T14:57:00.619373+0000","last_clean":"2024-03-23T14:57:00.619373+0000","last_became_active":"2024-03-23T14:56:59.643402+0000","last_became_peered":"2024-03-23T14:56:59.643402+0000","last_unstale":"2024-03-23T14:57:00.619373+0000","last_undegraded":"2024-03-23T14:57:00.619373+0000","last_fullsized":"2024-03-23T14:57:00.619373+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:35:32.710122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.529573+0000","last_change":"2024-03-23T14:56:59.640718+0000","last_active":"2024-03-23T14:57:01.529573+0000","last_peered":"2024-03-23T14:57:01.529573+0000","last_clean":"2024-03-23T14:57:01.529573+0000","last_became_active":"2024-03-23T14:56:59.640386+0000","last_became_peered":"2024-03-23T14:56:59.640386+0000","last_unstale":"2024-03-23T14:57:01.529573+0000","last_undegraded":"2024-03-23T14:57:01.529573+0000","last_fullsized":"2024-03-23T14:57:01.529573+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:39:22.051918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.621420+0000","last_change":"2024-03-23T14:56:59.650572+0000","last_active":"2024-03-23T14:57:00.621420+0000","last_peered":"2024-03-23T14:57:00.621420+0000","last_clean":"2024-03-23T14:57:00.621420+0000","last_became_active":"2024-03-23T14:56:59.650260+0000","last_became_peered":"2024-03-23T14:56:59.650260+0000","last_unstale":"2024-03-23T14:57:00.621420+0000","last_undegraded":"2024-03-23T14:57:00.621420+0000","last_fullsized":"2024-03-23T14:57:00.621420+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:12:59.763661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.619035+0000","last_change":"2024-03-23T14:56:59.646883+0000","last_active":"2024-03-23T14:57:00.619035+0000","last_peered":"2024-03-23T14:57:00.619035+0000","last_clean":"2024-03-23T14:57:00.619035+0000","last_became_active":"2024-03-23T14:56:59.643427+0000","last_became_peered":"2024-03-23T14:56:59.643427+0000","last_unstale":"2024-03-23T14:57:00.619035+0000","last_undegraded":"2024-03-23T14:57:00.619035+0000","last_fullsized":"2024-03-23T14:57:00.619035+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:52:36.323039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.557036+0000","last_change":"2024-03-23T14:56:59.646524+0000","last_active":"2024-03-23T14:57:01.557036+0000","last_peered":"2024-03-23T14:57:01.557036+0000","last_clean":"2024-03-23T14:57:01.557036+0000","last_became_active":"2024-03-23T14:56:59.646037+0000","last_became_peered":"2024-03-23T14:56:59.646037+0000","last_unstale":"2024-03-23T14:57:01.557036+0000","last_undegraded":"2024-03-23T14:57:01.557036+0000","last_fullsized":"2024-03-23T14:57:01.557036+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:57:06.681361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.551413+0000","last_change":"2024-03-23T14:56:59.658015+0000","last_active":"2024-03-23T14:57:01.551413+0000","last_peered":"2024-03-23T14:57:01.551413+0000","last_clean":"2024-03-23T14:57:01.551413+0000","last_became_active":"2024-03-23T14:56:59.657623+0000","last_became_peered":"2024-03-23T14:56:59.657623+0000","last_unstale":"2024-03-23T14:57:01.551413+0000","last_undegraded":"2024-03-23T14:57:01.551413+0000","last_fullsized":"2024-03-23T14:57:01.551413+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:14:04.996547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.555261+0000","last_change":"2024-03-23T14:56:59.660146+0000","last_active":"2024-03-23T14:57:01.555261+0000","last_peered":"2024-03-23T14:57:01.555261+0000","last_clean":"2024-03-23T14:57:01.555261+0000","last_became_active":"2024-03-23T14:56:59.659765+0000","last_became_peered":"2024-03-23T14:56:59.659765+0000","last_unstale":"2024-03-23T14:57:01.555261+0000","last_undegraded":"2024-03-23T14:57:01.555261+0000","last_fullsized":"2024-03-23T14:57:01.555261+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:49:20.982742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.529104+0000","last_change":"2024-03-23T14:56:59.637937+0000","last_active":"2024-03-23T14:57:01.529104+0000","last_peered":"2024-03-23T14:57:01.529104+0000","last_clean":"2024-03-23T14:57:01.529104+0000","last_became_active":"2024-03-23T14:56:59.637465+0000","last_became_peered":"2024-03-23T14:56:59.637465+0000","last_unstale":"2024-03-23T14:57:01.529104+0000","last_undegraded":"2024-03-23T14:57:01.529104+0000","last_fullsized":"2024-03-23T14:57:01.529104+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:38:24.739838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.619136+0000","last_change":"2024-03-23T14:56:59.633433+0000","last_active":"2024-03-23T14:57:00.619136+0000","last_peered":"2024-03-23T14:57:00.619136+0000","last_clean":"2024-03-23T14:57:00.619136+0000","last_became_active":"2024-03-23T14:56:59.633171+0000","last_became_peered":"2024-03-23T14:56:59.633171+0000","last_unstale":"2024-03-23T14:57:00.619136+0000","last_undegraded":"2024-03-23T14:57:00.619136+0000","last_fullsized":"2024-03-23T14:57:00.619136+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:36:11.384485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.618779+0000","last_change":"2024-03-23T14:56:58.621210+0000","last_active":"2024-03-23T14:57:00.618779+0000","last_peered":"2024-03-23T14:57:00.618779+0000","last_clean":"2024-03-23T14:57:00.618779+0000","last_became_active":"2024-03-23T14:56:58.620750+0000","last_became_peered":"2024-03-23T14:56:58.620750+0000","last_unstale":"2024-03-23T14:57:00.618779+0000","last_undegraded":"2024-03-23T14:57:00.618779+0000","last_fullsized":"2024-03-23T14:57:00.618779+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:24:18.799175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.556969+0000","last_change":"2024-03-23T14:56:59.633767+0000","last_active":"2024-03-23T14:57:01.556969+0000","last_peered":"2024-03-23T14:57:01.556969+0000","last_clean":"2024-03-23T14:57:01.556969+0000","last_became_active":"2024-03-23T14:56:59.633390+0000","last_became_peered":"2024-03-23T14:56:59.633390+0000","last_unstale":"2024-03-23T14:57:01.556969+0000","last_undegraded":"2024-03-23T14:57:01.556969+0000","last_fullsized":"2024-03-23T14:57:01.556969+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:12:59.763661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.529138+0000","last_change":"2024-03-23T14:56:58.613611+0000","last_active":"2024-03-23T14:57:01.529138+0000","last_peered":"2024-03-23T14:57:01.529138+0000","last_clean":"2024-03-23T14:57:01.529138+0000","last_became_active":"2024-03-23T14:56:58.608339+0000","last_became_peered":"2024-03-23T14:56:58.608339+0000","last_unstale":"2024-03-23T14:57:01.529138+0000","last_undegraded":"2024-03-23T14:57:01.529138+0000","last_fullsized":"2024-03-23T14:57:01.529138+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:48:12.389105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.618094+0000","last_change":"2024-03-23T14:56:58.621716+0000","last_active":"2024-03-23T14:57:00.618094+0000","last_peered":"2024-03-23T14:57:00.618094+0000","last_clean":"2024-03-23T14:57:00.618094+0000","last_became_active":"2024-03-23T14:56:58.620675+0000","last_became_peered":"2024-03-23T14:56:58.620675+0000","last_unstale":"2024-03-23T14:57:00.618094+0000","last_undegraded":"2024-03-23T14:57:00.618094+0000","last_fullsized":"2024-03-23T14:57:00.618094+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:07:10.915534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.617954+0000","last_change":"2024-03-23T14:56:59.644784+0000","last_active":"2024-03-23T14:57:00.617954+0000","last_peered":"2024-03-23T14:57:00.617954+0000","last_clean":"2024-03-23T14:57:00.617954+0000","last_became_active":"2024-03-23T14:56:59.644312+0000","last_became_peered":"2024-03-23T14:56:59.644312+0000","last_unstale":"2024-03-23T14:57:00.617954+0000","last_undegraded":"2024-03-23T14:57:00.617954+0000","last_fullsized":"2024-03-23T14:57:00.617954+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:38:24.739838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.550888+0000","last_change":"2024-03-23T14:56:58.611793+0000","last_active":"2024-03-23T14:57:01.550888+0000","last_peered":"2024-03-23T14:57:01.550888+0000","last_clean":"2024-03-23T14:57:01.550888+0000","last_became_active":"2024-03-23T14:56:58.611131+0000","last_became_peered":"2024-03-23T14:56:58.611131+0000","last_unstale":"2024-03-23T14:57:01.550888+0000","last_undegraded":"2024-03-23T14:57:01.550888+0000","last_fullsized":"2024-03-23T14:57:01.550888+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:36:10.376200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.620461+0000","last_change":"2024-03-23T14:56:59.649256+0000","last_active":"2024-03-23T14:57:00.620461+0000","last_peered":"2024-03-23T14:57:00.620461+0000","last_clean":"2024-03-23T14:57:00.620461+0000","last_became_active":"2024-03-23T14:56:59.648957+0000","last_became_peered":"2024-03-23T14:56:59.648957+0000","last_unstale":"2024-03-23T14:57:00.620461+0000","last_undegraded":"2024-03-23T14:57:00.620461+0000","last_fullsized":"2024-03-23T14:57:00.620461+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:07:26.887157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.529168+0000","last_change":"2024-03-23T14:56:58.630337+0000","last_active":"2024-03-23T14:57:01.529168+0000","last_peered":"2024-03-23T14:57:01.529168+0000","last_clean":"2024-03-23T14:57:01.529168+0000","last_became_active":"2024-03-23T14:56:58.629899+0000","last_became_peered":"2024-03-23T14:56:58.629899+0000","last_unstale":"2024-03-23T14:57:01.529168+0000","last_undegraded":"2024-03-23T14:57:01.529168+0000","last_fullsized":"2024-03-23T14:57:01.529168+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:24:18.799175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.551507+0000","last_change":"2024-03-23T14:56:59.657632+0000","last_active":"2024-03-23T14:57:01.551507+0000","last_peered":"2024-03-23T14:57:01.551507+0000","last_clean":"2024-03-23T14:57:01.551507+0000","last_became_active":"2024-03-23T14:56:59.655136+0000","last_became_peered":"2024-03-23T14:56:59.655136+0000","last_unstale":"2024-03-23T14:57:01.551507+0000","last_undegraded":"2024-03-23T14:57:01.551507+0000","last_fullsized":"2024-03-23T14:57:01.551507+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:46:57.866240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.551534+0000","last_change":"2024-03-23T14:56:58.627359+0000","last_active":"2024-03-23T14:57:01.551534+0000","last_peered":"2024-03-23T14:57:01.551534+0000","last_clean":"2024-03-23T14:57:01.551534+0000","last_became_active":"2024-03-23T14:56:58.626614+0000","last_became_peered":"2024-03-23T14:56:58.626614+0000","last_unstale":"2024-03-23T14:57:01.551534+0000","last_undegraded":"2024-03-23T14:57:01.551534+0000","last_fullsized":"2024-03-23T14:57:01.551534+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:05:00.726517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.556310+0000","last_change":"2024-03-23T14:56:59.659806+0000","last_active":"2024-03-23T14:57:01.556310+0000","last_peered":"2024-03-23T14:57:01.556310+0000","last_clean":"2024-03-23T14:57:01.556310+0000","last_became_active":"2024-03-23T14:56:59.658983+0000","last_became_peered":"2024-03-23T14:56:59.658983+0000","last_unstale":"2024-03-23T14:57:01.556310+0000","last_undegraded":"2024-03-23T14:57:01.556310+0000","last_fullsized":"2024-03-23T14:57:01.556310+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:58:29.591659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.621558+0000","last_change":"2024-03-23T14:56:58.625877+0000","last_active":"2024-03-23T14:57:00.621558+0000","last_peered":"2024-03-23T14:57:00.621558+0000","last_clean":"2024-03-23T14:57:00.621558+0000","last_became_active":"2024-03-23T14:56:58.625410+0000","last_became_peered":"2024-03-23T14:56:58.625410+0000","last_unstale":"2024-03-23T14:57:00.621558+0000","last_undegraded":"2024-03-23T14:57:00.621558+0000","last_fullsized":"2024-03-23T14:57:00.621558+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:05:00.726517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.619847+0000","last_change":"2024-03-23T14:56:59.647811+0000","last_active":"2024-03-23T14:57:00.619847+0000","last_peered":"2024-03-23T14:57:00.619847+0000","last_clean":"2024-03-23T14:57:00.619847+0000","last_became_active":"2024-03-23T14:56:59.647138+0000","last_became_peered":"2024-03-23T14:56:59.647138+0000","last_unstale":"2024-03-23T14:57:00.619847+0000","last_undegraded":"2024-03-23T14:57:00.619847+0000","last_fullsized":"2024-03-23T14:57:00.619847+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:49:20.982742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.529463+0000","last_change":"2024-03-23T14:56:58.630507+0000","last_active":"2024-03-23T14:57:01.529463+0000","last_peered":"2024-03-23T14:57:01.529463+0000","last_clean":"2024-03-23T14:57:01.529463+0000","last_became_active":"2024-03-23T14:56:58.629991+0000","last_became_peered":"2024-03-23T14:56:58.629991+0000","last_unstale":"2024-03-23T14:57:01.529463+0000","last_undegraded":"2024-03-23T14:57:01.529463+0000","last_fullsized":"2024-03-23T14:57:01.529463+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:01:25.160372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.518629+0000","last_change":"2024-03-23T14:56:59.649839+0000","last_active":"2024-03-23T14:57:01.518629+0000","last_peered":"2024-03-23T14:57:01.518629+0000","last_clean":"2024-03-23T14:57:01.518629+0000","last_became_active":"2024-03-23T14:56:59.648333+0000","last_became_peered":"2024-03-23T14:56:59.648333+0000","last_unstale":"2024-03-23T14:57:01.518629+0000","last_undegraded":"2024-03-23T14:57:01.518629+0000","last_fullsized":"2024-03-23T14:57:01.518629+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:46:57.866240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.557119+0000","last_change":"2024-03-23T14:56:58.609669+0000","last_active":"2024-03-23T14:57:01.557119+0000","last_peered":"2024-03-23T14:57:01.557119+0000","last_clean":"2024-03-23T14:57:01.557119+0000","last_became_active":"2024-03-23T14:56:58.608906+0000","last_became_peered":"2024-03-23T14:56:58.608906+0000","last_unstale":"2024-03-23T14:57:01.557119+0000","last_undegraded":"2024-03-23T14:57:01.557119+0000","last_fullsized":"2024-03-23T14:57:01.557119+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:34:51.972168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.556854+0000","last_change":"2024-03-23T14:56:59.648070+0000","last_active":"2024-03-23T14:57:01.556854+0000","last_peered":"2024-03-23T14:57:01.556854+0000","last_clean":"2024-03-23T14:57:01.556854+0000","last_became_active":"2024-03-23T14:56:59.647803+0000","last_became_peered":"2024-03-23T14:56:59.647803+0000","last_unstale":"2024-03-23T14:57:01.556854+0000","last_undegraded":"2024-03-23T14:57:01.556854+0000","last_fullsized":"2024-03-23T14:57:01.556854+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:37:56.745366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.551552+0000","last_change":"2024-03-23T14:56:58.626829+0000","last_active":"2024-03-23T14:57:01.551552+0000","last_peered":"2024-03-23T14:57:01.551552+0000","last_clean":"2024-03-23T14:57:01.551552+0000","last_became_active":"2024-03-23T14:56:58.626375+0000","last_became_peered":"2024-03-23T14:56:58.626375+0000","last_unstale":"2024-03-23T14:57:01.551552+0000","last_undegraded":"2024-03-23T14:57:01.551552+0000","last_fullsized":"2024-03-23T14:57:01.551552+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:48:12.389105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.518496+0000","last_change":"2024-03-23T14:56:59.653570+0000","last_active":"2024-03-23T14:57:01.518496+0000","last_peered":"2024-03-23T14:57:01.518496+0000","last_clean":"2024-03-23T14:57:01.518496+0000","last_became_active":"2024-03-23T14:56:59.653284+0000","last_became_peered":"2024-03-23T14:56:59.653284+0000","last_unstale":"2024-03-23T14:57:01.518496+0000","last_undegraded":"2024-03-23T14:57:01.518496+0000","last_fullsized":"2024-03-23T14:57:01.518496+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:34:30.083232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.528879+0000","last_change":"2024-03-23T14:56:58.614950+0000","last_active":"2024-03-23T14:57:01.528879+0000","last_peered":"2024-03-23T14:57:01.528879+0000","last_clean":"2024-03-23T14:57:01.528879+0000","last_became_active":"2024-03-23T14:56:58.614441+0000","last_became_peered":"2024-03-23T14:56:58.614441+0000","last_unstale":"2024-03-23T14:57:01.528879+0000","last_undegraded":"2024-03-23T14:57:01.528879+0000","last_fullsized":"2024-03-23T14:57:01.528879+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:05:00.726517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.528950+0000","last_change":"2024-03-23T14:56:59.636998+0000","last_active":"2024-03-23T14:57:01.528950+0000","last_peered":"2024-03-23T14:57:01.528950+0000","last_clean":"2024-03-23T14:57:01.528950+0000","last_became_active":"2024-03-23T14:56:59.636444+0000","last_became_peered":"2024-03-23T14:56:59.636444+0000","last_unstale":"2024-03-23T14:57:01.528950+0000","last_undegraded":"2024-03-23T14:57:01.528950+0000","last_fullsized":"2024-03-23T14:57:01.528950+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:09:00.536071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.555118+0000","last_change":"2024-03-23T14:56:58.604409+0000","last_active":"2024-03-23T14:57:01.555118+0000","last_peered":"2024-03-23T14:57:01.555118+0000","last_clean":"2024-03-23T14:57:01.555118+0000","last_became_active":"2024-03-23T14:56:58.604175+0000","last_became_peered":"2024-03-23T14:56:58.604175+0000","last_unstale":"2024-03-23T14:57:01.555118+0000","last_undegraded":"2024-03-23T14:57:01.555118+0000","last_fullsized":"2024-03-23T14:57:01.555118+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:07:10.915534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.518264+0000","last_change":"2024-03-23T14:56:59.651120+0000","last_active":"2024-03-23T14:57:01.518264+0000","last_peered":"2024-03-23T14:57:01.518264+0000","last_clean":"2024-03-23T14:57:01.518264+0000","last_became_active":"2024-03-23T14:56:59.650822+0000","last_became_peered":"2024-03-23T14:56:59.650822+0000","last_unstale":"2024-03-23T14:57:01.518264+0000","last_undegraded":"2024-03-23T14:57:01.518264+0000","last_fullsized":"2024-03-23T14:57:01.518264+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:35:26.329408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.619806+0000","last_change":"2024-03-23T14:56:58.606372+0000","last_active":"2024-03-23T14:57:00.619806+0000","last_peered":"2024-03-23T14:57:00.619806+0000","last_clean":"2024-03-23T14:57:00.619806+0000","last_became_active":"2024-03-23T14:56:58.606011+0000","last_became_peered":"2024-03-23T14:56:58.606011+0000","last_unstale":"2024-03-23T14:57:00.619806+0000","last_undegraded":"2024-03-23T14:57:00.619806+0000","last_fullsized":"2024-03-23T14:57:00.619806+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:34:51.972168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.518669+0000","last_change":"2024-03-23T14:56:59.652521+0000","last_active":"2024-03-23T14:57:01.518669+0000","last_peered":"2024-03-23T14:57:01.518669+0000","last_clean":"2024-03-23T14:57:01.518669+0000","last_became_active":"2024-03-23T14:56:59.652270+0000","last_became_peered":"2024-03-23T14:56:59.652270+0000","last_unstale":"2024-03-23T14:57:01.518669+0000","last_undegraded":"2024-03-23T14:57:01.518669+0000","last_fullsized":"2024-03-23T14:57:01.518669+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:51:59.393587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.556006+0000","last_change":"2024-03-23T14:56:58.610985+0000","last_active":"2024-03-23T14:57:01.556006+0000","last_peered":"2024-03-23T14:57:01.556006+0000","last_clean":"2024-03-23T14:57:01.556006+0000","last_became_active":"2024-03-23T14:56:58.610717+0000","last_became_peered":"2024-03-23T14:56:58.610717+0000","last_unstale":"2024-03-23T14:57:01.556006+0000","last_undegraded":"2024-03-23T14:57:01.556006+0000","last_fullsized":"2024-03-23T14:57:01.556006+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:24:18.799175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.519128+0000","last_change":"2024-03-23T14:56:59.652993+0000","last_active":"2024-03-23T14:57:01.519128+0000","last_peered":"2024-03-23T14:57:01.519128+0000","last_clean":"2024-03-23T14:57:01.519128+0000","last_became_active":"2024-03-23T14:56:59.652718+0000","last_became_peered":"2024-03-23T14:56:59.652718+0000","last_unstale":"2024-03-23T14:57:01.519128+0000","last_undegraded":"2024-03-23T14:57:01.519128+0000","last_fullsized":"2024-03-23T14:57:01.519128+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:06:34.693789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.621685+0000","last_change":"2024-03-23T14:56:58.608418+0000","last_active":"2024-03-23T14:57:00.621685+0000","last_peered":"2024-03-23T14:57:00.621685+0000","last_clean":"2024-03-23T14:57:00.621685+0000","last_became_active":"2024-03-23T14:56:58.607992+0000","last_became_peered":"2024-03-23T14:56:58.607992+0000","last_unstale":"2024-03-23T14:57:00.621685+0000","last_undegraded":"2024-03-23T14:57:00.621685+0000","last_fullsized":"2024-03-23T14:57:00.621685+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:36:10.376200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.551352+0000","last_change":"2024-03-23T14:56:59.657882+0000","last_active":"2024-03-23T14:57:01.551352+0000","last_peered":"2024-03-23T14:57:01.551352+0000","last_clean":"2024-03-23T14:57:01.551352+0000","last_became_active":"2024-03-23T14:56:59.657004+0000","last_became_peered":"2024-03-23T14:56:59.657004+0000","last_unstale":"2024-03-23T14:57:01.551352+0000","last_undegraded":"2024-03-23T14:57:01.551352+0000","last_fullsized":"2024-03-23T14:57:01.551352+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:58:40.996298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.557066+0000","last_change":"2024-03-23T14:56:58.617074+0000","last_active":"2024-03-23T14:57:01.557066+0000","last_peered":"2024-03-23T14:57:01.557066+0000","last_clean":"2024-03-23T14:57:01.557066+0000","last_became_active":"2024-03-23T14:56:58.616732+0000","last_became_peered":"2024-03-23T14:56:58.616732+0000","last_unstale":"2024-03-23T14:57:01.557066+0000","last_undegraded":"2024-03-23T14:57:01.557066+0000","last_fullsized":"2024-03-23T14:57:01.557066+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:05:00.726517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.518726+0000","last_change":"2024-03-23T14:56:59.664625+0000","last_active":"2024-03-23T14:57:01.518726+0000","last_peered":"2024-03-23T14:57:01.518726+0000","last_clean":"2024-03-23T14:57:01.518726+0000","last_became_active":"2024-03-23T14:56:59.664268+0000","last_became_peered":"2024-03-23T14:56:59.664268+0000","last_unstale":"2024-03-23T14:57:01.518726+0000","last_undegraded":"2024-03-23T14:57:01.518726+0000","last_fullsized":"2024-03-23T14:57:01.518726+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:53:55.152886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.517881+0000","last_change":"2024-03-23T14:56:48.722063+0000","last_active":"2024-03-23T14:57:01.517881+0000","last_peered":"2024-03-23T14:57:01.517881+0000","last_clean":"2024-03-23T14:57:01.517881+0000","last_became_active":"2024-03-23T14:56:48.720079+0000","last_became_peered":"2024-03-23T14:56:48.720079+0000","last_unstale":"2024-03-23T14:57:01.517881+0000","last_undegraded":"2024-03-23T14:57:01.517881+0000","last_fullsized":"2024-03-23T14:57:01.517881+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:47.478078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:47.478078+0000","last_clean_scrub_stamp":"2024-03-23T14:56:47.478078+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:55:51.231901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.551138+0000","last_change":"2024-03-23T14:56:58.611796+0000","last_active":"2024-03-23T14:57:01.551138+0000","last_peered":"2024-03-23T14:57:01.551138+0000","last_clean":"2024-03-23T14:57:01.551138+0000","last_became_active":"2024-03-23T14:56:58.611243+0000","last_became_peered":"2024-03-23T14:56:58.611243+0000","last_unstale":"2024-03-23T14:57:01.551138+0000","last_undegraded":"2024-03-23T14:57:01.551138+0000","last_fullsized":"2024-03-23T14:57:01.551138+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:24:18.799175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.618598+0000","last_change":"2024-03-23T14:56:59.664611+0000","last_active":"2024-03-23T14:57:00.618598+0000","last_peered":"2024-03-23T14:57:00.618598+0000","last_clean":"2024-03-23T14:57:00.618598+0000","last_became_active":"2024-03-23T14:56:59.664275+0000","last_became_peered":"2024-03-23T14:56:59.664275+0000","last_unstale":"2024-03-23T14:57:00.618598+0000","last_undegraded":"2024-03-23T14:57:00.618598+0000","last_fullsized":"2024-03-23T14:57:00.618598+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:12:59.763661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.517853+0000","last_change":"2024-03-23T14:56:58.620960+0000","last_active":"2024-03-23T14:57:01.517853+0000","last_peered":"2024-03-23T14:57:01.517853+0000","last_clean":"2024-03-23T14:57:01.517853+0000","last_became_active":"2024-03-23T14:56:58.620592+0000","last_became_peered":"2024-03-23T14:56:58.620592+0000","last_unstale":"2024-03-23T14:57:01.517853+0000","last_undegraded":"2024-03-23T14:57:01.517853+0000","last_fullsized":"2024-03-23T14:57:01.517853+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:01:25.160372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.556871+0000","last_change":"2024-03-23T14:56:59.621260+0000","last_active":"2024-03-23T14:57:01.556871+0000","last_peered":"2024-03-23T14:57:01.556871+0000","last_clean":"2024-03-23T14:57:01.556871+0000","last_became_active":"2024-03-23T14:56:59.620924+0000","last_became_peered":"2024-03-23T14:56:59.620924+0000","last_unstale":"2024-03-23T14:57:01.556871+0000","last_undegraded":"2024-03-23T14:57:01.556871+0000","last_fullsized":"2024-03-23T14:57:01.556871+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:59:19.727729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.555522+0000","last_change":"2024-03-23T14:56:58.611902+0000","last_active":"2024-03-23T14:57:01.555522+0000","last_peered":"2024-03-23T14:57:01.555522+0000","last_clean":"2024-03-23T14:57:01.555522+0000","last_became_active":"2024-03-23T14:56:58.611208+0000","last_became_peered":"2024-03-23T14:56:58.611208+0000","last_unstale":"2024-03-23T14:57:01.555522+0000","last_undegraded":"2024-03-23T14:57:01.555522+0000","last_fullsized":"2024-03-23T14:57:01.555522+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:36:10.376200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.620759+0000","last_change":"2024-03-23T14:56:59.640818+0000","last_active":"2024-03-23T14:57:00.620759+0000","last_peered":"2024-03-23T14:57:00.620759+0000","last_clean":"2024-03-23T14:57:00.620759+0000","last_became_active":"2024-03-23T14:56:59.640493+0000","last_became_peered":"2024-03-23T14:56:59.640493+0000","last_unstale":"2024-03-23T14:57:00.620759+0000","last_undegraded":"2024-03-23T14:57:00.620759+0000","last_fullsized":"2024-03-23T14:57:00.620759+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:38:24.739838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.551684+0000","last_change":"2024-03-23T14:56:58.616782+0000","last_active":"2024-03-23T14:57:01.551684+0000","last_peered":"2024-03-23T14:57:01.551684+0000","last_clean":"2024-03-23T14:57:01.551684+0000","last_became_active":"2024-03-23T14:56:58.615957+0000","last_became_peered":"2024-03-23T14:56:58.615957+0000","last_unstale":"2024-03-23T14:57:01.551684+0000","last_undegraded":"2024-03-23T14:57:01.551684+0000","last_fullsized":"2024-03-23T14:57:01.551684+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:01:25.160372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.619632+0000","last_change":"2024-03-23T14:56:59.623480+0000","last_active":"2024-03-23T14:57:00.619632+0000","last_peered":"2024-03-23T14:57:00.619632+0000","last_clean":"2024-03-23T14:57:00.619632+0000","last_became_active":"2024-03-23T14:56:59.623224+0000","last_became_peered":"2024-03-23T14:56:59.623224+0000","last_unstale":"2024-03-23T14:57:00.619632+0000","last_undegraded":"2024-03-23T14:57:00.619632+0000","last_fullsized":"2024-03-23T14:57:00.619632+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:48:13.397390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.621161+0000","last_change":"2024-03-23T14:56:58.604398+0000","last_active":"2024-03-23T14:57:00.621161+0000","last_peered":"2024-03-23T14:57:00.621161+0000","last_clean":"2024-03-23T14:57:00.621161+0000","last_became_active":"2024-03-23T14:56:58.603192+0000","last_became_peered":"2024-03-23T14:56:58.603192+0000","last_unstale":"2024-03-23T14:57:00.621161+0000","last_undegraded":"2024-03-23T14:57:00.621161+0000","last_fullsized":"2024-03-23T14:57:00.621161+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:24:18.799175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.551563+0000","last_change":"2024-03-23T14:56:59.661728+0000","last_active":"2024-03-23T14:57:01.551563+0000","last_peered":"2024-03-23T14:57:01.551563+0000","last_clean":"2024-03-23T14:57:01.551563+0000","last_became_active":"2024-03-23T14:56:59.656955+0000","last_became_peered":"2024-03-23T14:56:59.656955+0000","last_unstale":"2024-03-23T14:57:01.551563+0000","last_undegraded":"2024-03-23T14:57:01.551563+0000","last_fullsized":"2024-03-23T14:57:01.551563+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:51:59.393587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.518923+0000","last_change":"2024-03-23T14:56:58.619283+0000","last_active":"2024-03-23T14:57:01.518923+0000","last_peered":"2024-03-23T14:57:01.518923+0000","last_clean":"2024-03-23T14:57:01.518923+0000","last_became_active":"2024-03-23T14:56:58.618922+0000","last_became_peered":"2024-03-23T14:56:58.618922+0000","last_unstale":"2024-03-23T14:57:01.518923+0000","last_undegraded":"2024-03-23T14:57:01.518923+0000","last_fullsized":"2024-03-23T14:57:01.518923+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:29:07.131287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.620920+0000","last_change":"2024-03-23T14:56:59.636699+0000","last_active":"2024-03-23T14:57:00.620920+0000","last_peered":"2024-03-23T14:57:00.620920+0000","last_clean":"2024-03-23T14:57:00.620920+0000","last_became_active":"2024-03-23T14:56:59.636376+0000","last_became_peered":"2024-03-23T14:56:59.636376+0000","last_unstale":"2024-03-23T14:57:00.620920+0000","last_undegraded":"2024-03-23T14:57:00.620920+0000","last_fullsized":"2024-03-23T14:57:00.620920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:37:56.745366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.622171+0000","last_change":"2024-03-23T14:56:58.627211+0000","last_active":"2024-03-23T14:57:01.622171+0000","last_peered":"2024-03-23T14:57:01.622171+0000","last_clean":"2024-03-23T14:57:01.622171+0000","last_became_active":"2024-03-23T14:56:58.626779+0000","last_became_peered":"2024-03-23T14:56:58.626779+0000","last_unstale":"2024-03-23T14:57:01.622171+0000","last_undegraded":"2024-03-23T14:57:01.622171+0000","last_fullsized":"2024-03-23T14:57:01.622171+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:01:25.160372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.619749+0000","last_change":"2024-03-23T14:56:59.643640+0000","last_active":"2024-03-23T14:57:00.619749+0000","last_peered":"2024-03-23T14:57:00.619749+0000","last_clean":"2024-03-23T14:57:00.619749+0000","last_became_active":"2024-03-23T14:56:59.643255+0000","last_became_peered":"2024-03-23T14:56:59.643255+0000","last_unstale":"2024-03-23T14:57:00.619749+0000","last_undegraded":"2024-03-23T14:57:00.619749+0000","last_fullsized":"2024-03-23T14:57:00.619749+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:35:26.329408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.529191+0000","last_change":"2024-03-23T14:56:58.623880+0000","last_active":"2024-03-23T14:57:01.529191+0000","last_peered":"2024-03-23T14:57:01.529191+0000","last_clean":"2024-03-23T14:57:01.529191+0000","last_became_active":"2024-03-23T14:56:58.623291+0000","last_became_peered":"2024-03-23T14:56:58.623291+0000","last_unstale":"2024-03-23T14:57:01.529191+0000","last_undegraded":"2024-03-23T14:57:01.529191+0000","last_fullsized":"2024-03-23T14:57:01.529191+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:36:10.376200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.556526+0000","last_change":"2024-03-23T14:56:59.634446+0000","last_active":"2024-03-23T14:57:01.556526+0000","last_peered":"2024-03-23T14:57:01.556526+0000","last_clean":"2024-03-23T14:57:01.556526+0000","last_became_active":"2024-03-23T14:56:59.633946+0000","last_became_peered":"2024-03-23T14:56:59.633946+0000","last_unstale":"2024-03-23T14:57:01.556526+0000","last_undegraded":"2024-03-23T14:57:01.556526+0000","last_fullsized":"2024-03-23T14:57:01.556526+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:09:00.536071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.517807+0000","last_change":"2024-03-23T14:56:58.618776+0000","last_active":"2024-03-23T14:57:01.517807+0000","last_peered":"2024-03-23T14:57:01.517807+0000","last_clean":"2024-03-23T14:57:01.517807+0000","last_became_active":"2024-03-23T14:56:58.618418+0000","last_became_peered":"2024-03-23T14:56:58.618418+0000","last_unstale":"2024-03-23T14:57:01.517807+0000","last_undegraded":"2024-03-23T14:57:01.517807+0000","last_fullsized":"2024-03-23T14:57:01.517807+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:48:12.389105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.619949+0000","last_change":"2024-03-23T14:56:59.633845+0000","last_active":"2024-03-23T14:57:00.619949+0000","last_peered":"2024-03-23T14:57:00.619949+0000","last_clean":"2024-03-23T14:57:00.619949+0000","last_became_active":"2024-03-23T14:56:59.633409+0000","last_became_peered":"2024-03-23T14:56:59.633409+0000","last_unstale":"2024-03-23T14:57:00.619949+0000","last_undegraded":"2024-03-23T14:57:00.619949+0000","last_fullsized":"2024-03-23T14:57:00.619949+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:29:08.139571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.620668+0000","last_change":"2024-03-23T14:56:58.616951+0000","last_active":"2024-03-23T14:57:00.620668+0000","last_peered":"2024-03-23T14:57:00.620668+0000","last_clean":"2024-03-23T14:57:00.620668+0000","last_became_active":"2024-03-23T14:56:58.616383+0000","last_became_peered":"2024-03-23T14:56:58.616383+0000","last_unstale":"2024-03-23T14:57:00.620668+0000","last_undegraded":"2024-03-23T14:57:00.620668+0000","last_fullsized":"2024-03-23T14:57:00.620668+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:48:12.389105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.617942+0000","last_change":"2024-03-23T14:56:59.645384+0000","last_active":"2024-03-23T14:57:00.617942+0000","last_peered":"2024-03-23T14:57:00.617942+0000","last_clean":"2024-03-23T14:57:00.617942+0000","last_became_active":"2024-03-23T14:56:59.643723+0000","last_became_peered":"2024-03-23T14:56:59.643723+0000","last_unstale":"2024-03-23T14:57:00.617942+0000","last_undegraded":"2024-03-23T14:57:00.617942+0000","last_fullsized":"2024-03-23T14:57:00.617942+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:09:00.536071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.620676+0000","last_change":"2024-03-23T14:56:58.622814+0000","last_active":"2024-03-23T14:57:00.620676+0000","last_peered":"2024-03-23T14:57:00.620676+0000","last_clean":"2024-03-23T14:57:00.620676+0000","last_became_active":"2024-03-23T14:56:58.621745+0000","last_became_peered":"2024-03-23T14:56:58.621745+0000","last_unstale":"2024-03-23T14:57:00.620676+0000","last_undegraded":"2024-03-23T14:57:00.620676+0000","last_fullsized":"2024-03-23T14:57:00.620676+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:49:19.974457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.618866+0000","last_change":"2024-03-23T14:56:59.645315+0000","last_active":"2024-03-23T14:57:00.618866+0000","last_peered":"2024-03-23T14:57:00.618866+0000","last_clean":"2024-03-23T14:57:00.618866+0000","last_became_active":"2024-03-23T14:56:59.644287+0000","last_became_peered":"2024-03-23T14:56:59.644287+0000","last_unstale":"2024-03-23T14:57:00.618866+0000","last_undegraded":"2024-03-23T14:57:00.618866+0000","last_fullsized":"2024-03-23T14:57:00.618866+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:57:06.681361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.518700+0000","last_change":"2024-03-23T14:56:58.625848+0000","last_active":"2024-03-23T14:57:01.518700+0000","last_peered":"2024-03-23T14:57:01.518700+0000","last_clean":"2024-03-23T14:57:01.518700+0000","last_became_active":"2024-03-23T14:56:58.625453+0000","last_became_peered":"2024-03-23T14:56:58.625453+0000","last_unstale":"2024-03-23T14:57:01.518700+0000","last_undegraded":"2024-03-23T14:57:01.518700+0000","last_fullsized":"2024-03-23T14:57:01.518700+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:06:05.434722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.529162+0000","last_change":"2024-03-23T14:56:59.635812+0000","last_active":"2024-03-23T14:57:01.529162+0000","last_peered":"2024-03-23T14:57:01.529162+0000","last_clean":"2024-03-23T14:57:01.529162+0000","last_became_active":"2024-03-23T14:56:59.635519+0000","last_became_peered":"2024-03-23T14:56:59.635519+0000","last_unstale":"2024-03-23T14:57:01.529162+0000","last_undegraded":"2024-03-23T14:57:01.529162+0000","last_fullsized":"2024-03-23T14:57:01.529162+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:37:56.745366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.551417+0000","last_change":"2024-03-23T14:56:59.662434+0000","last_active":"2024-03-23T14:57:01.551417+0000","last_peered":"2024-03-23T14:57:01.551417+0000","last_clean":"2024-03-23T14:57:01.551417+0000","last_became_active":"2024-03-23T14:56:59.661514+0000","last_became_peered":"2024-03-23T14:56:59.661514+0000","last_unstale":"2024-03-23T14:57:01.551417+0000","last_undegraded":"2024-03-23T14:57:01.551417+0000","last_fullsized":"2024-03-23T14:57:01.551417+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:34:30.083232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.518270+0000","last_change":"2024-03-23T14:56:58.629744+0000","last_active":"2024-03-23T14:57:01.518270+0000","last_peered":"2024-03-23T14:57:01.518270+0000","last_clean":"2024-03-23T14:57:01.518270+0000","last_became_active":"2024-03-23T14:56:58.629401+0000","last_became_peered":"2024-03-23T14:56:58.629401+0000","last_unstale":"2024-03-23T14:57:01.518270+0000","last_undegraded":"2024-03-23T14:57:01.518270+0000","last_fullsized":"2024-03-23T14:57:01.518270+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:13:45.673576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.551735+0000","last_change":"2024-03-23T14:56:59.658250+0000","last_active":"2024-03-23T14:57:01.551735+0000","last_peered":"2024-03-23T14:57:01.551735+0000","last_clean":"2024-03-23T14:57:01.551735+0000","last_became_active":"2024-03-23T14:56:59.657085+0000","last_became_peered":"2024-03-23T14:56:59.657085+0000","last_unstale":"2024-03-23T14:57:01.551735+0000","last_undegraded":"2024-03-23T14:57:01.551735+0000","last_fullsized":"2024-03-23T14:57:01.551735+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:53:55.152886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.518837+0000","last_change":"2024-03-23T14:56:58.624886+0000","last_active":"2024-03-23T14:57:01.518837+0000","last_peered":"2024-03-23T14:57:01.518837+0000","last_clean":"2024-03-23T14:57:01.518837+0000","last_became_active":"2024-03-23T14:56:58.624478+0000","last_became_peered":"2024-03-23T14:56:58.624478+0000","last_unstale":"2024-03-23T14:57:01.518837+0000","last_undegraded":"2024-03-23T14:57:01.518837+0000","last_fullsized":"2024-03-23T14:57:01.518837+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:49:19.974457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.555868+0000","last_change":"2024-03-23T14:56:59.660089+0000","last_active":"2024-03-23T14:57:01.555868+0000","last_peered":"2024-03-23T14:57:01.555868+0000","last_clean":"2024-03-23T14:57:01.555868+0000","last_became_active":"2024-03-23T14:56:59.659469+0000","last_became_peered":"2024-03-23T14:56:59.659469+0000","last_unstale":"2024-03-23T14:57:01.555868+0000","last_undegraded":"2024-03-23T14:57:01.555868+0000","last_fullsized":"2024-03-23T14:57:01.555868+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:37:14.115808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.619024+0000","last_change":"2024-03-23T14:56:58.608147+0000","last_active":"2024-03-23T14:57:00.619024+0000","last_peered":"2024-03-23T14:57:00.619024+0000","last_clean":"2024-03-23T14:57:00.619024+0000","last_became_active":"2024-03-23T14:56:58.603630+0000","last_became_peered":"2024-03-23T14:56:58.603630+0000","last_unstale":"2024-03-23T14:57:00.619024+0000","last_undegraded":"2024-03-23T14:57:00.619024+0000","last_fullsized":"2024-03-23T14:57:00.619024+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:08:31.251287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.618985+0000","last_change":"2024-03-23T14:56:59.645116+0000","last_active":"2024-03-23T14:57:00.618985+0000","last_peered":"2024-03-23T14:57:00.618985+0000","last_clean":"2024-03-23T14:57:00.618985+0000","last_became_active":"2024-03-23T14:56:59.644484+0000","last_became_peered":"2024-03-23T14:56:59.644484+0000","last_unstale":"2024-03-23T14:57:00.618985+0000","last_undegraded":"2024-03-23T14:57:00.618985+0000","last_fullsized":"2024-03-23T14:57:00.618985+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:39:22.051918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.556818+0000","last_change":"2024-03-23T14:56:58.619314+0000","last_active":"2024-03-23T14:57:01.556818+0000","last_peered":"2024-03-23T14:57:01.556818+0000","last_clean":"2024-03-23T14:57:01.556818+0000","last_became_active":"2024-03-23T14:56:58.618751+0000","last_became_peered":"2024-03-23T14:56:58.618751+0000","last_unstale":"2024-03-23T14:57:01.556818+0000","last_undegraded":"2024-03-23T14:57:01.556818+0000","last_fullsized":"2024-03-23T14:57:01.556818+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:16:22.644801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27800,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344040,"statfs":{"total":96636764160,"available":96608296960,"internally_reserved":0,"allocated":790528,"data_stored":655584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":780,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":798720,"data_stored":655640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-23T14:57:07.054 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-23T14:57:07.187 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.181+0000 7f23f991e640 1 Processor -- start 2024-03-23T14:57:07.187 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.181+0000 7f23f991e640 1 -- start start 2024-03-23T14:57:07.188 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.185+0000 7f23f991e640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f23f40adc90 0x7f23f40ae070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:07.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.185+0000 7f23f991e640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f23f406bf70 0x7f23f40b4a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:07.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.185+0000 7f23f991e640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23f406c9c0 con 0x7f23f40b34d0 2024-03-23T14:57:07.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.185+0000 7f23f991e640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f23f40ae810 con 0x7f23f40adc90 2024-03-23T14:57:07.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.185+0000 7f23f991e640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 -- 0x7f23f40b2ce0 con 0x7f23f406bf70 2024-03-23T14:57:07.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.185+0000 7f23f2ffd640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f23f40adc90 0x7f23f40ae070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:07.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.185+0000 7f23f2ffd640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f23f40adc90 0x7f23f40ae070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:59020/0 (socket says 172.21.15.164:59020) 2024-03-23T14:57:07.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.185+0000 7f23f2ffd640 1 -- 172.21.15.164:0/2797526830 learned_addr learned my addr 172.21.15.164:0/2797526830 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:07.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.185+0000 7f23f27fc640 1 --2- 172.21.15.164:0/2797526830 >> v2:172.21.15.173:3300/0 conn(0x7f23f406bf70 0x7f23f40b4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:07.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.185+0000 7f23f2ffd640 1 -- 172.21.15.164:0/2797526830 >> v2:172.21.15.173:3300/0 conn(0x7f23f406bf70 msgr2=0x7f23f40b4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:07.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.185+0000 7f23f2ffd640 1 --2- 172.21.15.164:0/2797526830 >> v2:172.21.15.173:3300/0 conn(0x7f23f406bf70 0x7f23f40b4a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:07.190 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.185+0000 7f23f2ffd640 1 -- 172.21.15.164:0/2797526830 >> v1:172.21.15.173:6789/0 conn(0x7f23f40b34d0 legacy=0x7f23f40b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:07.190 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.185+0000 7f23f2ffd640 1 -- 172.21.15.164:0/2797526830 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23f40af460 con 0x7f23f40adc90 2024-03-23T14:57:07.191 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.185+0000 7f23f2ffd640 1 --2- 172.21.15.164:0/2797526830 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f23f40adc90 0x7f23f40ae070 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f23f40727c0 tx=0x7f23e401c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ce0ea13d0cd3b3d server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:07.191 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.185+0000 7f23f1ffb640 1 -- 172.21.15.164:0/2797526830 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23e40ac640 con 0x7f23f40adc90 2024-03-23T14:57:07.191 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.185+0000 7f23f1ffb640 1 -- 172.21.15.164:0/2797526830 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23e40ac7a0 con 0x7f23f40adc90 2024-03-23T14:57:07.191 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.185+0000 7f23f1ffb640 1 -- 172.21.15.164:0/2797526830 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23e40acb30 con 0x7f23f40adc90 2024-03-23T14:57:07.192 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.189+0000 7f23f991e640 1 -- 172.21.15.164:0/2797526830 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f23f40adc90 msgr2=0x7f23f40ae070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:07.192 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.189+0000 7f23f991e640 1 --2- 172.21.15.164:0/2797526830 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f23f40adc90 0x7f23f40ae070 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f23f40727c0 tx=0x7f23e401c400 comp rx=0 tx=0).stop 2024-03-23T14:57:07.192 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.189+0000 7f23f991e640 1 -- 172.21.15.164:0/2797526830 shutdown_connections 2024-03-23T14:57:07.192 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.189+0000 7f23f991e640 1 --2- 172.21.15.164:0/2797526830 >> v2:172.21.15.173:3300/0 conn(0x7f23f406bf70 0x7f23f40b4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:07.193 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.189+0000 7f23f991e640 1 --2- 172.21.15.164:0/2797526830 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f23f40adc90 0x7f23f40ae070 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:07.193 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.189+0000 7f23f991e640 1 -- 172.21.15.164:0/2797526830 >> 172.21.15.164:0/2797526830 conn(0x7f23f40b8ca0 msgr2=0x7f23f40b90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:07.193 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.189+0000 7f23f991e640 1 -- 172.21.15.164:0/2797526830 shutdown_connections 2024-03-23T14:57:07.193 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.189+0000 7f23f991e640 1 -- 172.21.15.164:0/2797526830 wait complete. 2024-03-23T14:57:07.195 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.189+0000 7f23f991e640 1 Processor -- start 2024-03-23T14:57:07.195 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.189+0000 7f23f991e640 1 -- start start 2024-03-23T14:57:07.195 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.189+0000 7f23f991e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f23f40adc90 0x7f23f4147420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:07.195 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.189+0000 7f23f991e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f23f40b34d0 0x7f23f4148ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:07.195 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.189+0000 7f23f991e640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f23f4149e70 0x7f23f4149000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:07.195 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.189+0000 7f23f991e640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f23f41483e0 con 0x7f23f40b34d0 2024-03-23T14:57:07.195 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.189+0000 7f23f991e640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f23f4149790 con 0x7f23f40adc90 2024-03-23T14:57:07.195 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.189+0000 7f23f991e640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f23f406f670 con 0x7f23f4149e70 2024-03-23T14:57:07.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.189+0000 7f23f27fc640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f23f40b34d0 0x7f23f4148ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:07.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.189+0000 7f23f27fc640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f23f40b34d0 0x7f23f4148ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:59028/0 (socket says 172.21.15.164:59028) 2024-03-23T14:57:07.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.189+0000 7f23f27fc640 1 -- 172.21.15.164:0/3097020810 learned_addr learned my addr 172.21.15.164:0/3097020810 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:07.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.189+0000 7f23f37fe640 1 --2- 172.21.15.164:0/3097020810 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f23f4149e70 0x7f23f4149000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:07.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.189+0000 7f23f2ffd640 1 --2- 172.21.15.164:0/3097020810 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f23f40adc90 0x7f23f4147420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:07.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.189+0000 7f23f27fc640 1 -- 172.21.15.164:0/3097020810 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f23f4149e70 msgr2=0x7f23f4149000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:07.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.189+0000 7f23f27fc640 1 --2- 172.21.15.164:0/3097020810 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f23f4149e70 0x7f23f4149000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:07.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.189+0000 7f23f27fc640 1 -- 172.21.15.164:0/3097020810 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f23f40adc90 msgr2=0x7f23f4147420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:07.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.189+0000 7f23f27fc640 1 --2- 172.21.15.164:0/3097020810 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f23f40adc90 0x7f23f4147420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:07.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.189+0000 7f23f27fc640 1 -- 172.21.15.164:0/3097020810 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23e40094d0 con 0x7f23f40b34d0 2024-03-23T14:57:07.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.189+0000 7f23f27fc640 1 --2- 172.21.15.164:0/3097020810 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f23f40b34d0 0x7f23f4148ac0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f23e800d7d0 tx=0x7f23e800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:07.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.189+0000 7f23dbfff640 1 -- 172.21.15.164:0/3097020810 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23e8004030 con 0x7f23f40b34d0 2024-03-23T14:57:07.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.189+0000 7f23dbfff640 1 -- 172.21.15.164:0/3097020810 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23e80041b0 con 0x7f23f40b34d0 2024-03-23T14:57:07.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.189+0000 7f23f991e640 1 -- 172.21.15.164:0/3097020810 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23f406f950 con 0x7f23f40b34d0 2024-03-23T14:57:07.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.189+0000 7f23dbfff640 1 -- 172.21.15.164:0/3097020810 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23e800b6c0 con 0x7f23f40b34d0 2024-03-23T14:57:07.196 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.189+0000 7f23f991e640 1 -- 172.21.15.164:0/3097020810 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23f406f2c0 con 0x7f23f40b34d0 2024-03-23T14:57:07.197 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.189+0000 7f23f991e640 1 -- 172.21.15.164:0/3097020810 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23f4149950 con 0x7f23f40b34d0 2024-03-23T14:57:07.197 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.193+0000 7f23dbfff640 1 -- 172.21.15.164:0/3097020810 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7f23e800b820 con 0x7f23f40b34d0 2024-03-23T14:57:07.198 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.193+0000 7f23dbfff640 1 --2- 172.21.15.164:0/3097020810 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f23e00672a0 0x7f23e0069760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:07.198 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.193+0000 7f23dbfff640 1 -- 172.21.15.164:0/3097020810 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f23e80852c0 con 0x7f23f40b34d0 2024-03-23T14:57:07.198 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.193+0000 7f23f2ffd640 1 --2- 172.21.15.164:0/3097020810 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f23e00672a0 0x7f23e0069760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:07.198 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.193+0000 7f23f2ffd640 1 --2- 172.21.15.164:0/3097020810 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f23e00672a0 0x7f23e0069760 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f23e40ae000 tx=0x7f23e40a6070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:07.201 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.197+0000 7f23dbfff640 1 -- 172.21.15.164:0/3097020810 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7f23e8051f00 con 0x7f23f40b34d0 2024-03-23T14:57:07.359 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.353+0000 7f23f991e640 1 -- 172.21.15.164:0/3097020810 --> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f23f4148520 con 0x7f23e00672a0 2024-03-23T14:57:07.368 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.365+0000 7f23dbfff640 1 -- 172.21.15.164:0/3097020810 <== mgr.4107 v2:172.21.15.164:6832/2139087985 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251952 (secure 0 0 0) 0x7f23e40ac910 con 0x7f23e00672a0 2024-03-23T14:57:07.372 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-23T14:57:07.374 INFO:teuthology.orchestra.run.smithi164.stderr:dumped all 2024-03-23T14:57:07.375 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.369+0000 7f23f991e640 1 -- 172.21.15.164:0/3097020810 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f23e00672a0 msgr2=0x7f23e0069760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:07.375 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.369+0000 7f23f991e640 1 --2- 172.21.15.164:0/3097020810 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f23e00672a0 0x7f23e0069760 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f23e40ae000 tx=0x7f23e40a6070 comp rx=0 tx=0).stop 2024-03-23T14:57:07.375 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.369+0000 7f23f991e640 1 -- 172.21.15.164:0/3097020810 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f23f40b34d0 msgr2=0x7f23f4148ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:07.375 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.369+0000 7f23f991e640 1 --2- 172.21.15.164:0/3097020810 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f23f40b34d0 0x7f23f4148ac0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f23e800d7d0 tx=0x7f23e800dc90 comp rx=0 tx=0).stop 2024-03-23T14:57:07.375 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.369+0000 7f23f991e640 1 -- 172.21.15.164:0/3097020810 shutdown_connections 2024-03-23T14:57:07.376 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.369+0000 7f23f991e640 1 --2- 172.21.15.164:0/3097020810 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f23f4149e70 0x7f23f4149000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:07.376 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.369+0000 7f23f991e640 1 --2- 172.21.15.164:0/3097020810 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f23f40b34d0 0x7f23f4148ac0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:07.376 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.369+0000 7f23f991e640 1 --2- 172.21.15.164:0/3097020810 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f23e00672a0 0x7f23e0069760 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:07.376 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.369+0000 7f23f991e640 1 --2- 172.21.15.164:0/3097020810 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f23f40adc90 0x7f23f4147420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:07.376 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.369+0000 7f23f991e640 1 -- 172.21.15.164:0/3097020810 >> 172.21.15.164:0/3097020810 conn(0x7f23f40b8ca0 msgr2=0x7f23f414e3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:07.376 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.369+0000 7f23f991e640 1 -- 172.21.15.164:0/3097020810 shutdown_connections 2024-03-23T14:57:07.376 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.369+0000 7f23f991e640 1 -- 172.21.15.164:0/3097020810 wait complete. 2024-03-23T14:57:07.390 INFO:teuthology.orchestra.run.smithi164.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-03-23T14:57:07.111676+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":185,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219000,"kb_used_data":2800,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755720,"statfs":{"total":773094113280,"available":772869857280,"internally_reserved":0,"allocated":2867200,"data_stored":1740428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002728"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.518848+0000","last_change":"2024-03-23T14:56:59.652438+0000","last_active":"2024-03-23T14:57:01.518848+0000","last_peered":"2024-03-23T14:57:01.518848+0000","last_clean":"2024-03-23T14:57:01.518848+0000","last_became_active":"2024-03-23T14:56:59.652169+0000","last_became_peered":"2024-03-23T14:56:59.652169+0000","last_unstale":"2024-03-23T14:57:01.518848+0000","last_undegraded":"2024-03-23T14:57:01.518848+0000","last_fullsized":"2024-03-23T14:57:01.518848+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:58:31.542399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.621127+0000","last_change":"2024-03-23T14:56:59.632308+0000","last_active":"2024-03-23T14:57:00.621127+0000","last_peered":"2024-03-23T14:57:00.621127+0000","last_clean":"2024-03-23T14:57:00.621127+0000","last_became_active":"2024-03-23T14:56:59.631895+0000","last_became_peered":"2024-03-23T14:56:59.631895+0000","last_unstale":"2024-03-23T14:57:00.621127+0000","last_undegraded":"2024-03-23T14:57:00.621127+0000","last_fullsized":"2024-03-23T14:57:00.621127+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:59:19.727729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.555798+0000","last_change":"2024-03-23T14:56:59.636862+0000","last_active":"2024-03-23T14:57:01.555798+0000","last_peered":"2024-03-23T14:57:01.555798+0000","last_clean":"2024-03-23T14:57:01.555798+0000","last_became_active":"2024-03-23T14:56:59.636209+0000","last_became_peered":"2024-03-23T14:56:59.636209+0000","last_unstale":"2024-03-23T14:57:01.555798+0000","last_undegraded":"2024-03-23T14:57:01.555798+0000","last_fullsized":"2024-03-23T14:57:01.555798+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:11:59.565543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.551115+0000","last_change":"2024-03-23T14:56:59.662732+0000","last_active":"2024-03-23T14:57:01.551115+0000","last_peered":"2024-03-23T14:57:01.551115+0000","last_clean":"2024-03-23T14:57:01.551115+0000","last_became_active":"2024-03-23T14:56:59.661656+0000","last_became_peered":"2024-03-23T14:56:59.661656+0000","last_unstale":"2024-03-23T14:57:01.551115+0000","last_undegraded":"2024-03-23T14:57:01.551115+0000","last_fullsized":"2024-03-23T14:57:01.551115+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:35:32.710122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.518258+0000","last_change":"2024-03-23T14:56:59.651141+0000","last_active":"2024-03-23T14:57:01.518258+0000","last_peered":"2024-03-23T14:57:01.518258+0000","last_clean":"2024-03-23T14:57:01.518258+0000","last_became_active":"2024-03-23T14:56:59.650922+0000","last_became_peered":"2024-03-23T14:56:59.650922+0000","last_unstale":"2024-03-23T14:57:01.518258+0000","last_undegraded":"2024-03-23T14:57:01.518258+0000","last_fullsized":"2024-03-23T14:57:01.518258+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:14:04.996547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.551288+0000","last_change":"2024-03-23T14:56:59.662551+0000","last_active":"2024-03-23T14:57:01.551288+0000","last_peered":"2024-03-23T14:57:01.551288+0000","last_clean":"2024-03-23T14:57:01.551288+0000","last_became_active":"2024-03-23T14:56:59.661732+0000","last_became_peered":"2024-03-23T14:56:59.661732+0000","last_unstale":"2024-03-23T14:57:01.551288+0000","last_undegraded":"2024-03-23T14:57:01.551288+0000","last_fullsized":"2024-03-23T14:57:01.551288+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:35:26.329408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.518244+0000","last_change":"2024-03-23T14:56:59.646745+0000","last_active":"2024-03-23T14:57:01.518244+0000","last_peered":"2024-03-23T14:57:01.518244+0000","last_clean":"2024-03-23T14:57:01.518244+0000","last_became_active":"2024-03-23T14:56:59.646354+0000","last_became_peered":"2024-03-23T14:56:59.646354+0000","last_unstale":"2024-03-23T14:57:01.518244+0000","last_undegraded":"2024-03-23T14:57:01.518244+0000","last_fullsized":"2024-03-23T14:57:01.518244+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:58:40.996298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.617783+0000","last_change":"2024-03-23T14:56:59.659502+0000","last_active":"2024-03-23T14:57:00.617783+0000","last_peered":"2024-03-23T14:57:00.617783+0000","last_clean":"2024-03-23T14:57:00.617783+0000","last_became_active":"2024-03-23T14:56:59.659222+0000","last_became_peered":"2024-03-23T14:56:59.659222+0000","last_unstale":"2024-03-23T14:57:00.617783+0000","last_undegraded":"2024-03-23T14:57:00.617783+0000","last_fullsized":"2024-03-23T14:57:00.617783+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:07:26.887157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.519017+0000","last_change":"2024-03-23T14:56:59.659935+0000","last_active":"2024-03-23T14:57:01.519017+0000","last_peered":"2024-03-23T14:57:01.519017+0000","last_clean":"2024-03-23T14:57:01.519017+0000","last_became_active":"2024-03-23T14:56:59.659501+0000","last_became_peered":"2024-03-23T14:56:59.659501+0000","last_unstale":"2024-03-23T14:57:01.519017+0000","last_undegraded":"2024-03-23T14:57:01.519017+0000","last_fullsized":"2024-03-23T14:57:01.519017+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:21:16.376914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.529493+0000","last_change":"2024-03-23T14:56:59.659013+0000","last_active":"2024-03-23T14:57:01.529493+0000","last_peered":"2024-03-23T14:57:01.529493+0000","last_clean":"2024-03-23T14:57:01.529493+0000","last_became_active":"2024-03-23T14:56:59.658667+0000","last_became_peered":"2024-03-23T14:56:59.658667+0000","last_unstale":"2024-03-23T14:57:01.529493+0000","last_undegraded":"2024-03-23T14:57:01.529493+0000","last_fullsized":"2024-03-23T14:57:01.529493+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:12:59.763661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.529535+0000","last_change":"2024-03-23T14:56:59.663729+0000","last_active":"2024-03-23T14:57:01.529535+0000","last_peered":"2024-03-23T14:57:01.529535+0000","last_clean":"2024-03-23T14:57:01.529535+0000","last_became_active":"2024-03-23T14:56:59.663313+0000","last_became_peered":"2024-03-23T14:56:59.663313+0000","last_unstale":"2024-03-23T14:57:01.529535+0000","last_undegraded":"2024-03-23T14:57:01.529535+0000","last_fullsized":"2024-03-23T14:57:01.529535+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:57:06.681361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.619818+0000","last_change":"2024-03-23T14:56:59.640819+0000","last_active":"2024-03-23T14:57:00.619818+0000","last_peered":"2024-03-23T14:57:00.619818+0000","last_clean":"2024-03-23T14:57:00.619818+0000","last_became_active":"2024-03-23T14:56:59.640330+0000","last_became_peered":"2024-03-23T14:56:59.640330+0000","last_unstale":"2024-03-23T14:57:00.619818+0000","last_undegraded":"2024-03-23T14:57:00.619818+0000","last_fullsized":"2024-03-23T14:57:00.619818+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:01:26.168657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.618487+0000","last_change":"2024-03-23T14:56:59.645597+0000","last_active":"2024-03-23T14:57:00.618487+0000","last_peered":"2024-03-23T14:57:00.618487+0000","last_clean":"2024-03-23T14:57:00.618487+0000","last_became_active":"2024-03-23T14:56:59.644700+0000","last_became_peered":"2024-03-23T14:56:59.644700+0000","last_unstale":"2024-03-23T14:57:00.618487+0000","last_undegraded":"2024-03-23T14:57:00.618487+0000","last_fullsized":"2024-03-23T14:57:00.618487+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:59:19.727729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.556462+0000","last_change":"2024-03-23T14:56:59.636241+0000","last_active":"2024-03-23T14:57:01.556462+0000","last_peered":"2024-03-23T14:57:01.556462+0000","last_clean":"2024-03-23T14:57:01.556462+0000","last_became_active":"2024-03-23T14:56:59.635803+0000","last_became_peered":"2024-03-23T14:56:59.635803+0000","last_unstale":"2024-03-23T14:57:01.556462+0000","last_undegraded":"2024-03-23T14:57:01.556462+0000","last_fullsized":"2024-03-23T14:57:01.556462+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:07:26.887157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.620719+0000","last_change":"2024-03-23T14:56:59.642239+0000","last_active":"2024-03-23T14:57:00.620719+0000","last_peered":"2024-03-23T14:57:00.620719+0000","last_clean":"2024-03-23T14:57:00.620719+0000","last_became_active":"2024-03-23T14:56:59.641872+0000","last_became_peered":"2024-03-23T14:56:59.641872+0000","last_unstale":"2024-03-23T14:57:00.620719+0000","last_undegraded":"2024-03-23T14:57:00.620719+0000","last_fullsized":"2024-03-23T14:57:00.620719+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:09:00.536071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.551044+0000","last_change":"2024-03-23T14:56:59.634551+0000","last_active":"2024-03-23T14:57:01.551044+0000","last_peered":"2024-03-23T14:57:01.551044+0000","last_clean":"2024-03-23T14:57:01.551044+0000","last_became_active":"2024-03-23T14:56:59.634255+0000","last_became_peered":"2024-03-23T14:56:59.634255+0000","last_unstale":"2024-03-23T14:57:01.551044+0000","last_undegraded":"2024-03-23T14:57:01.551044+0000","last_fullsized":"2024-03-23T14:57:01.551044+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:39:22.051918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.552128+0000","last_change":"2024-03-23T14:56:59.657535+0000","last_active":"2024-03-23T14:57:01.552128+0000","last_peered":"2024-03-23T14:57:01.552128+0000","last_clean":"2024-03-23T14:57:01.552128+0000","last_became_active":"2024-03-23T14:56:59.656959+0000","last_became_peered":"2024-03-23T14:56:59.656959+0000","last_unstale":"2024-03-23T14:57:01.552128+0000","last_undegraded":"2024-03-23T14:57:01.552128+0000","last_fullsized":"2024-03-23T14:57:01.552128+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:58:31.542399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.555866+0000","last_change":"2024-03-23T14:56:59.639104+0000","last_active":"2024-03-23T14:57:01.555866+0000","last_peered":"2024-03-23T14:57:01.555866+0000","last_clean":"2024-03-23T14:57:01.555866+0000","last_became_active":"2024-03-23T14:56:59.638744+0000","last_became_peered":"2024-03-23T14:56:59.638744+0000","last_unstale":"2024-03-23T14:57:01.555866+0000","last_undegraded":"2024-03-23T14:57:01.555866+0000","last_fullsized":"2024-03-23T14:57:01.555866+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:29:08.139571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.518933+0000","last_change":"2024-03-23T14:56:59.647334+0000","last_active":"2024-03-23T14:57:01.518933+0000","last_peered":"2024-03-23T14:57:01.518933+0000","last_clean":"2024-03-23T14:57:01.518933+0000","last_became_active":"2024-03-23T14:56:59.646801+0000","last_became_peered":"2024-03-23T14:56:59.646801+0000","last_unstale":"2024-03-23T14:57:01.518933+0000","last_undegraded":"2024-03-23T14:57:01.518933+0000","last_fullsized":"2024-03-23T14:57:01.518933+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:02:12.733995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.529380+0000","last_change":"2024-03-23T14:56:59.639076+0000","last_active":"2024-03-23T14:57:01.529380+0000","last_peered":"2024-03-23T14:57:01.529380+0000","last_clean":"2024-03-23T14:57:01.529380+0000","last_became_active":"2024-03-23T14:56:59.638267+0000","last_became_peered":"2024-03-23T14:56:59.638267+0000","last_unstale":"2024-03-23T14:57:01.529380+0000","last_undegraded":"2024-03-23T14:57:01.529380+0000","last_fullsized":"2024-03-23T14:57:01.529380+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:59:19.727729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.556655+0000","last_change":"2024-03-23T14:56:59.647220+0000","last_active":"2024-03-23T14:57:01.556655+0000","last_peered":"2024-03-23T14:57:01.556655+0000","last_clean":"2024-03-23T14:57:01.556655+0000","last_became_active":"2024-03-23T14:56:59.646833+0000","last_became_peered":"2024-03-23T14:56:59.646833+0000","last_unstale":"2024-03-23T14:57:01.556655+0000","last_undegraded":"2024-03-23T14:57:01.556655+0000","last_fullsized":"2024-03-23T14:57:01.556655+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:38:24.739838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.551049+0000","last_change":"2024-03-23T14:56:59.662969+0000","last_active":"2024-03-23T14:57:01.551049+0000","last_peered":"2024-03-23T14:57:01.551049+0000","last_clean":"2024-03-23T14:57:01.551049+0000","last_became_active":"2024-03-23T14:56:59.661554+0000","last_became_peered":"2024-03-23T14:56:59.661554+0000","last_unstale":"2024-03-23T14:57:01.551049+0000","last_undegraded":"2024-03-23T14:57:01.551049+0000","last_fullsized":"2024-03-23T14:57:01.551049+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:52:36.323039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.618010+0000","last_change":"2024-03-23T14:56:59.643866+0000","last_active":"2024-03-23T14:57:00.618010+0000","last_peered":"2024-03-23T14:57:00.618010+0000","last_clean":"2024-03-23T14:57:00.618010+0000","last_became_active":"2024-03-23T14:56:59.643577+0000","last_became_peered":"2024-03-23T14:56:59.643577+0000","last_unstale":"2024-03-23T14:57:00.618010+0000","last_undegraded":"2024-03-23T14:57:00.618010+0000","last_fullsized":"2024-03-23T14:57:00.618010+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:37:56.745366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.550754+0000","last_change":"2024-03-23T14:56:59.651798+0000","last_active":"2024-03-23T14:57:01.550754+0000","last_peered":"2024-03-23T14:57:01.550754+0000","last_clean":"2024-03-23T14:57:01.550754+0000","last_became_active":"2024-03-23T14:56:59.651534+0000","last_became_peered":"2024-03-23T14:56:59.651534+0000","last_unstale":"2024-03-23T14:57:01.550754+0000","last_undegraded":"2024-03-23T14:57:01.550754+0000","last_fullsized":"2024-03-23T14:57:01.550754+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:57:06.681361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.619373+0000","last_change":"2024-03-23T14:56:59.645160+0000","last_active":"2024-03-23T14:57:00.619373+0000","last_peered":"2024-03-23T14:57:00.619373+0000","last_clean":"2024-03-23T14:57:00.619373+0000","last_became_active":"2024-03-23T14:56:59.643402+0000","last_became_peered":"2024-03-23T14:56:59.643402+0000","last_unstale":"2024-03-23T14:57:00.619373+0000","last_undegraded":"2024-03-23T14:57:00.619373+0000","last_fullsized":"2024-03-23T14:57:00.619373+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:35:32.710122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.529573+0000","last_change":"2024-03-23T14:56:59.640718+0000","last_active":"2024-03-23T14:57:01.529573+0000","last_peered":"2024-03-23T14:57:01.529573+0000","last_clean":"2024-03-23T14:57:01.529573+0000","last_became_active":"2024-03-23T14:56:59.640386+0000","last_became_peered":"2024-03-23T14:56:59.640386+0000","last_unstale":"2024-03-23T14:57:01.529573+0000","last_undegraded":"2024-03-23T14:57:01.529573+0000","last_fullsized":"2024-03-23T14:57:01.529573+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:39:22.051918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.621420+0000","last_change":"2024-03-23T14:56:59.650572+0000","last_active":"2024-03-23T14:57:00.621420+0000","last_peered":"2024-03-23T14:57:00.621420+0000","last_clean":"2024-03-23T14:57:00.621420+0000","last_became_active":"2024-03-23T14:56:59.650260+0000","last_became_peered":"2024-03-23T14:56:59.650260+0000","last_unstale":"2024-03-23T14:57:00.621420+0000","last_undegraded":"2024-03-23T14:57:00.621420+0000","last_fullsized":"2024-03-23T14:57:00.621420+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:12:59.763661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.619035+0000","last_change":"2024-03-23T14:56:59.646883+0000","last_active":"2024-03-23T14:57:00.619035+0000","last_peered":"2024-03-23T14:57:00.619035+0000","last_clean":"2024-03-23T14:57:00.619035+0000","last_became_active":"2024-03-23T14:56:59.643427+0000","last_became_peered":"2024-03-23T14:56:59.643427+0000","last_unstale":"2024-03-23T14:57:00.619035+0000","last_undegraded":"2024-03-23T14:57:00.619035+0000","last_fullsized":"2024-03-23T14:57:00.619035+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:52:36.323039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.557036+0000","last_change":"2024-03-23T14:56:59.646524+0000","last_active":"2024-03-23T14:57:01.557036+0000","last_peered":"2024-03-23T14:57:01.557036+0000","last_clean":"2024-03-23T14:57:01.557036+0000","last_became_active":"2024-03-23T14:56:59.646037+0000","last_became_peered":"2024-03-23T14:56:59.646037+0000","last_unstale":"2024-03-23T14:57:01.557036+0000","last_undegraded":"2024-03-23T14:57:01.557036+0000","last_fullsized":"2024-03-23T14:57:01.557036+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:57:06.681361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.551413+0000","last_change":"2024-03-23T14:56:59.658015+0000","last_active":"2024-03-23T14:57:01.551413+0000","last_peered":"2024-03-23T14:57:01.551413+0000","last_clean":"2024-03-23T14:57:01.551413+0000","last_became_active":"2024-03-23T14:56:59.657623+0000","last_became_peered":"2024-03-23T14:56:59.657623+0000","last_unstale":"2024-03-23T14:57:01.551413+0000","last_undegraded":"2024-03-23T14:57:01.551413+0000","last_fullsized":"2024-03-23T14:57:01.551413+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:14:04.996547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.555261+0000","last_change":"2024-03-23T14:56:59.660146+0000","last_active":"2024-03-23T14:57:01.555261+0000","last_peered":"2024-03-23T14:57:01.555261+0000","last_clean":"2024-03-23T14:57:01.555261+0000","last_became_active":"2024-03-23T14:56:59.659765+0000","last_became_peered":"2024-03-23T14:56:59.659765+0000","last_unstale":"2024-03-23T14:57:01.555261+0000","last_undegraded":"2024-03-23T14:57:01.555261+0000","last_fullsized":"2024-03-23T14:57:01.555261+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:49:20.982742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.529104+0000","last_change":"2024-03-23T14:56:59.637937+0000","last_active":"2024-03-23T14:57:01.529104+0000","last_peered":"2024-03-23T14:57:01.529104+0000","last_clean":"2024-03-23T14:57:01.529104+0000","last_became_active":"2024-03-23T14:56:59.637465+0000","last_became_peered":"2024-03-23T14:56:59.637465+0000","last_unstale":"2024-03-23T14:57:01.529104+0000","last_undegraded":"2024-03-23T14:57:01.529104+0000","last_fullsized":"2024-03-23T14:57:01.529104+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:38:24.739838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.619136+0000","last_change":"2024-03-23T14:56:59.633433+0000","last_active":"2024-03-23T14:57:00.619136+0000","last_peered":"2024-03-23T14:57:00.619136+0000","last_clean":"2024-03-23T14:57:00.619136+0000","last_became_active":"2024-03-23T14:56:59.633171+0000","last_became_peered":"2024-03-23T14:56:59.633171+0000","last_unstale":"2024-03-23T14:57:00.619136+0000","last_undegraded":"2024-03-23T14:57:00.619136+0000","last_fullsized":"2024-03-23T14:57:00.619136+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:36:11.384485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.618779+0000","last_change":"2024-03-23T14:56:58.621210+0000","last_active":"2024-03-23T14:57:00.618779+0000","last_peered":"2024-03-23T14:57:00.618779+0000","last_clean":"2024-03-23T14:57:00.618779+0000","last_became_active":"2024-03-23T14:56:58.620750+0000","last_became_peered":"2024-03-23T14:56:58.620750+0000","last_unstale":"2024-03-23T14:57:00.618779+0000","last_undegraded":"2024-03-23T14:57:00.618779+0000","last_fullsized":"2024-03-23T14:57:00.618779+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:24:18.799175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.556969+0000","last_change":"2024-03-23T14:56:59.633767+0000","last_active":"2024-03-23T14:57:01.556969+0000","last_peered":"2024-03-23T14:57:01.556969+0000","last_clean":"2024-03-23T14:57:01.556969+0000","last_became_active":"2024-03-23T14:56:59.633390+0000","last_became_peered":"2024-03-23T14:56:59.633390+0000","last_unstale":"2024-03-23T14:57:01.556969+0000","last_undegraded":"2024-03-23T14:57:01.556969+0000","last_fullsized":"2024-03-23T14:57:01.556969+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:12:59.763661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.529138+0000","last_change":"2024-03-23T14:56:58.613611+0000","last_active":"2024-03-23T14:57:01.529138+0000","last_peered":"2024-03-23T14:57:01.529138+0000","last_clean":"2024-03-23T14:57:01.529138+0000","last_became_active":"2024-03-23T14:56:58.608339+0000","last_became_peered":"2024-03-23T14:56:58.608339+0000","last_unstale":"2024-03-23T14:57:01.529138+0000","last_undegraded":"2024-03-23T14:57:01.529138+0000","last_fullsized":"2024-03-23T14:57:01.529138+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:48:12.389105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.618094+0000","last_change":"2024-03-23T14:56:58.621716+0000","last_active":"2024-03-23T14:57:00.618094+0000","last_peered":"2024-03-23T14:57:00.618094+0000","last_clean":"2024-03-23T14:57:00.618094+0000","last_became_active":"2024-03-23T14:56:58.620675+0000","last_became_peered":"2024-03-23T14:56:58.620675+0000","last_unstale":"2024-03-23T14:57:00.618094+0000","last_undegraded":"2024-03-23T14:57:00.618094+0000","last_fullsized":"2024-03-23T14:57:00.618094+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:07:10.915534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.617954+0000","last_change":"2024-03-23T14:56:59.644784+0000","last_active":"2024-03-23T14:57:00.617954+0000","last_peered":"2024-03-23T14:57:00.617954+0000","last_clean":"2024-03-23T14:57:00.617954+0000","last_became_active":"2024-03-23T14:56:59.644312+0000","last_became_peered":"2024-03-23T14:56:59.644312+0000","last_unstale":"2024-03-23T14:57:00.617954+0000","last_undegraded":"2024-03-23T14:57:00.617954+0000","last_fullsized":"2024-03-23T14:57:00.617954+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:38:24.739838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.550888+0000","last_change":"2024-03-23T14:56:58.611793+0000","last_active":"2024-03-23T14:57:01.550888+0000","last_peered":"2024-03-23T14:57:01.550888+0000","last_clean":"2024-03-23T14:57:01.550888+0000","last_became_active":"2024-03-23T14:56:58.611131+0000","last_became_peered":"2024-03-23T14:56:58.611131+0000","last_unstale":"2024-03-23T14:57:01.550888+0000","last_undegraded":"2024-03-23T14:57:01.550888+0000","last_fullsized":"2024-03-23T14:57:01.550888+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:36:10.376200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.620461+0000","last_change":"2024-03-23T14:56:59.649256+0000","last_active":"2024-03-23T14:57:00.620461+0000","last_peered":"2024-03-23T14:57:00.620461+0000","last_clean":"2024-03-23T14:57:00.620461+0000","last_became_active":"2024-03-23T14:56:59.648957+0000","last_became_peered":"2024-03-23T14:56:59.648957+0000","last_unstale":"2024-03-23T14:57:00.620461+0000","last_undegraded":"2024-03-23T14:57:00.620461+0000","last_fullsized":"2024-03-23T14:57:00.620461+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:07:26.887157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.529168+0000","last_change":"2024-03-23T14:56:58.630337+0000","last_active":"2024-03-23T14:57:01.529168+0000","last_peered":"2024-03-23T14:57:01.529168+0000","last_clean":"2024-03-23T14:57:01.529168+0000","last_became_active":"2024-03-23T14:56:58.629899+0000","last_became_peered":"2024-03-23T14:56:58.629899+0000","last_unstale":"2024-03-23T14:57:01.529168+0000","last_undegraded":"2024-03-23T14:57:01.529168+0000","last_fullsized":"2024-03-23T14:57:01.529168+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:24:18.799175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.551507+0000","last_change":"2024-03-23T14:56:59.657632+0000","last_active":"2024-03-23T14:57:01.551507+0000","last_peered":"2024-03-23T14:57:01.551507+0000","last_clean":"2024-03-23T14:57:01.551507+0000","last_became_active":"2024-03-23T14:56:59.655136+0000","last_became_peered":"2024-03-23T14:56:59.655136+0000","last_unstale":"2024-03-23T14:57:01.551507+0000","last_undegraded":"2024-03-23T14:57:01.551507+0000","last_fullsized":"2024-03-23T14:57:01.551507+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:46:57.866240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.551534+0000","last_change":"2024-03-23T14:56:58.627359+0000","last_active":"2024-03-23T14:57:01.551534+0000","last_peered":"2024-03-23T14:57:01.551534+0000","last_clean":"2024-03-23T14:57:01.551534+0000","last_became_active":"2024-03-23T14:56:58.626614+0000","last_became_peered":"2024-03-23T14:56:58.626614+0000","last_unstale":"2024-03-23T14:57:01.551534+0000","last_undegraded":"2024-03-23T14:57:01.551534+0000","last_fullsized":"2024-03-23T14:57:01.551534+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:05:00.726517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.556310+0000","last_change":"2024-03-23T14:56:59.659806+0000","last_active":"2024-03-23T14:57:01.556310+0000","last_peered":"2024-03-23T14:57:01.556310+0000","last_clean":"2024-03-23T14:57:01.556310+0000","last_became_active":"2024-03-23T14:56:59.658983+0000","last_became_peered":"2024-03-23T14:56:59.658983+0000","last_unstale":"2024-03-23T14:57:01.556310+0000","last_undegraded":"2024-03-23T14:57:01.556310+0000","last_fullsized":"2024-03-23T14:57:01.556310+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:58:29.591659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.621558+0000","last_change":"2024-03-23T14:56:58.625877+0000","last_active":"2024-03-23T14:57:00.621558+0000","last_peered":"2024-03-23T14:57:00.621558+0000","last_clean":"2024-03-23T14:57:00.621558+0000","last_became_active":"2024-03-23T14:56:58.625410+0000","last_became_peered":"2024-03-23T14:56:58.625410+0000","last_unstale":"2024-03-23T14:57:00.621558+0000","last_undegraded":"2024-03-23T14:57:00.621558+0000","last_fullsized":"2024-03-23T14:57:00.621558+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:05:00.726517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.619847+0000","last_change":"2024-03-23T14:56:59.647811+0000","last_active":"2024-03-23T14:57:00.619847+0000","last_peered":"2024-03-23T14:57:00.619847+0000","last_clean":"2024-03-23T14:57:00.619847+0000","last_became_active":"2024-03-23T14:56:59.647138+0000","last_became_peered":"2024-03-23T14:56:59.647138+0000","last_unstale":"2024-03-23T14:57:00.619847+0000","last_undegraded":"2024-03-23T14:57:00.619847+0000","last_fullsized":"2024-03-23T14:57:00.619847+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:49:20.982742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.529463+0000","last_change":"2024-03-23T14:56:58.630507+0000","last_active":"2024-03-23T14:57:01.529463+0000","last_peered":"2024-03-23T14:57:01.529463+0000","last_clean":"2024-03-23T14:57:01.529463+0000","last_became_active":"2024-03-23T14:56:58.629991+0000","last_became_peered":"2024-03-23T14:56:58.629991+0000","last_unstale":"2024-03-23T14:57:01.529463+0000","last_undegraded":"2024-03-23T14:57:01.529463+0000","last_fullsized":"2024-03-23T14:57:01.529463+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:01:25.160372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.518629+0000","last_change":"2024-03-23T14:56:59.649839+0000","last_active":"2024-03-23T14:57:01.518629+0000","last_peered":"2024-03-23T14:57:01.518629+0000","last_clean":"2024-03-23T14:57:01.518629+0000","last_became_active":"2024-03-23T14:56:59.648333+0000","last_became_peered":"2024-03-23T14:56:59.648333+0000","last_unstale":"2024-03-23T14:57:01.518629+0000","last_undegraded":"2024-03-23T14:57:01.518629+0000","last_fullsized":"2024-03-23T14:57:01.518629+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:46:57.866240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.557119+0000","last_change":"2024-03-23T14:56:58.609669+0000","last_active":"2024-03-23T14:57:01.557119+0000","last_peered":"2024-03-23T14:57:01.557119+0000","last_clean":"2024-03-23T14:57:01.557119+0000","last_became_active":"2024-03-23T14:56:58.608906+0000","last_became_peered":"2024-03-23T14:56:58.608906+0000","last_unstale":"2024-03-23T14:57:01.557119+0000","last_undegraded":"2024-03-23T14:57:01.557119+0000","last_fullsized":"2024-03-23T14:57:01.557119+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:34:51.972168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.556854+0000","last_change":"2024-03-23T14:56:59.648070+0000","last_active":"2024-03-23T14:57:01.556854+0000","last_peered":"2024-03-23T14:57:01.556854+0000","last_clean":"2024-03-23T14:57:01.556854+0000","last_became_active":"2024-03-23T14:56:59.647803+0000","last_became_peered":"2024-03-23T14:56:59.647803+0000","last_unstale":"2024-03-23T14:57:01.556854+0000","last_undegraded":"2024-03-23T14:57:01.556854+0000","last_fullsized":"2024-03-23T14:57:01.556854+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:37:56.745366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.551552+0000","last_change":"2024-03-23T14:56:58.626829+0000","last_active":"2024-03-23T14:57:01.551552+0000","last_peered":"2024-03-23T14:57:01.551552+0000","last_clean":"2024-03-23T14:57:01.551552+0000","last_became_active":"2024-03-23T14:56:58.626375+0000","last_became_peered":"2024-03-23T14:56:58.626375+0000","last_unstale":"2024-03-23T14:57:01.551552+0000","last_undegraded":"2024-03-23T14:57:01.551552+0000","last_fullsized":"2024-03-23T14:57:01.551552+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:48:12.389105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.518496+0000","last_change":"2024-03-23T14:56:59.653570+0000","last_active":"2024-03-23T14:57:01.518496+0000","last_peered":"2024-03-23T14:57:01.518496+0000","last_clean":"2024-03-23T14:57:01.518496+0000","last_became_active":"2024-03-23T14:56:59.653284+0000","last_became_peered":"2024-03-23T14:56:59.653284+0000","last_unstale":"2024-03-23T14:57:01.518496+0000","last_undegraded":"2024-03-23T14:57:01.518496+0000","last_fullsized":"2024-03-23T14:57:01.518496+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:34:30.083232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.528879+0000","last_change":"2024-03-23T14:56:58.614950+0000","last_active":"2024-03-23T14:57:01.528879+0000","last_peered":"2024-03-23T14:57:01.528879+0000","last_clean":"2024-03-23T14:57:01.528879+0000","last_became_active":"2024-03-23T14:56:58.614441+0000","last_became_peered":"2024-03-23T14:56:58.614441+0000","last_unstale":"2024-03-23T14:57:01.528879+0000","last_undegraded":"2024-03-23T14:57:01.528879+0000","last_fullsized":"2024-03-23T14:57:01.528879+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:05:00.726517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.528950+0000","last_change":"2024-03-23T14:56:59.636998+0000","last_active":"2024-03-23T14:57:01.528950+0000","last_peered":"2024-03-23T14:57:01.528950+0000","last_clean":"2024-03-23T14:57:01.528950+0000","last_became_active":"2024-03-23T14:56:59.636444+0000","last_became_peered":"2024-03-23T14:56:59.636444+0000","last_unstale":"2024-03-23T14:57:01.528950+0000","last_undegraded":"2024-03-23T14:57:01.528950+0000","last_fullsized":"2024-03-23T14:57:01.528950+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:09:00.536071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.555118+0000","last_change":"2024-03-23T14:56:58.604409+0000","last_active":"2024-03-23T14:57:01.555118+0000","last_peered":"2024-03-23T14:57:01.555118+0000","last_clean":"2024-03-23T14:57:01.555118+0000","last_became_active":"2024-03-23T14:56:58.604175+0000","last_became_peered":"2024-03-23T14:56:58.604175+0000","last_unstale":"2024-03-23T14:57:01.555118+0000","last_undegraded":"2024-03-23T14:57:01.555118+0000","last_fullsized":"2024-03-23T14:57:01.555118+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:07:10.915534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.518264+0000","last_change":"2024-03-23T14:56:59.651120+0000","last_active":"2024-03-23T14:57:01.518264+0000","last_peered":"2024-03-23T14:57:01.518264+0000","last_clean":"2024-03-23T14:57:01.518264+0000","last_became_active":"2024-03-23T14:56:59.650822+0000","last_became_peered":"2024-03-23T14:56:59.650822+0000","last_unstale":"2024-03-23T14:57:01.518264+0000","last_undegraded":"2024-03-23T14:57:01.518264+0000","last_fullsized":"2024-03-23T14:57:01.518264+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:35:26.329408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.619806+0000","last_change":"2024-03-23T14:56:58.606372+0000","last_active":"2024-03-23T14:57:00.619806+0000","last_peered":"2024-03-23T14:57:00.619806+0000","last_clean":"2024-03-23T14:57:00.619806+0000","last_became_active":"2024-03-23T14:56:58.606011+0000","last_became_peered":"2024-03-23T14:56:58.606011+0000","last_unstale":"2024-03-23T14:57:00.619806+0000","last_undegraded":"2024-03-23T14:57:00.619806+0000","last_fullsized":"2024-03-23T14:57:00.619806+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:34:51.972168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.518669+0000","last_change":"2024-03-23T14:56:59.652521+0000","last_active":"2024-03-23T14:57:01.518669+0000","last_peered":"2024-03-23T14:57:01.518669+0000","last_clean":"2024-03-23T14:57:01.518669+0000","last_became_active":"2024-03-23T14:56:59.652270+0000","last_became_peered":"2024-03-23T14:56:59.652270+0000","last_unstale":"2024-03-23T14:57:01.518669+0000","last_undegraded":"2024-03-23T14:57:01.518669+0000","last_fullsized":"2024-03-23T14:57:01.518669+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:51:59.393587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.556006+0000","last_change":"2024-03-23T14:56:58.610985+0000","last_active":"2024-03-23T14:57:01.556006+0000","last_peered":"2024-03-23T14:57:01.556006+0000","last_clean":"2024-03-23T14:57:01.556006+0000","last_became_active":"2024-03-23T14:56:58.610717+0000","last_became_peered":"2024-03-23T14:56:58.610717+0000","last_unstale":"2024-03-23T14:57:01.556006+0000","last_undegraded":"2024-03-23T14:57:01.556006+0000","last_fullsized":"2024-03-23T14:57:01.556006+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:24:18.799175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.519128+0000","last_change":"2024-03-23T14:56:59.652993+0000","last_active":"2024-03-23T14:57:01.519128+0000","last_peered":"2024-03-23T14:57:01.519128+0000","last_clean":"2024-03-23T14:57:01.519128+0000","last_became_active":"2024-03-23T14:56:59.652718+0000","last_became_peered":"2024-03-23T14:56:59.652718+0000","last_unstale":"2024-03-23T14:57:01.519128+0000","last_undegraded":"2024-03-23T14:57:01.519128+0000","last_fullsized":"2024-03-23T14:57:01.519128+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:06:34.693789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.621685+0000","last_change":"2024-03-23T14:56:58.608418+0000","last_active":"2024-03-23T14:57:00.621685+0000","last_peered":"2024-03-23T14:57:00.621685+0000","last_clean":"2024-03-23T14:57:00.621685+0000","last_became_active":"2024-03-23T14:56:58.607992+0000","last_became_peered":"2024-03-23T14:56:58.607992+0000","last_unstale":"2024-03-23T14:57:00.621685+0000","last_undegraded":"2024-03-23T14:57:00.621685+0000","last_fullsized":"2024-03-23T14:57:00.621685+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:36:10.376200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.551352+0000","last_change":"2024-03-23T14:56:59.657882+0000","last_active":"2024-03-23T14:57:01.551352+0000","last_peered":"2024-03-23T14:57:01.551352+0000","last_clean":"2024-03-23T14:57:01.551352+0000","last_became_active":"2024-03-23T14:56:59.657004+0000","last_became_peered":"2024-03-23T14:56:59.657004+0000","last_unstale":"2024-03-23T14:57:01.551352+0000","last_undegraded":"2024-03-23T14:57:01.551352+0000","last_fullsized":"2024-03-23T14:57:01.551352+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:58:40.996298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.557066+0000","last_change":"2024-03-23T14:56:58.617074+0000","last_active":"2024-03-23T14:57:01.557066+0000","last_peered":"2024-03-23T14:57:01.557066+0000","last_clean":"2024-03-23T14:57:01.557066+0000","last_became_active":"2024-03-23T14:56:58.616732+0000","last_became_peered":"2024-03-23T14:56:58.616732+0000","last_unstale":"2024-03-23T14:57:01.557066+0000","last_undegraded":"2024-03-23T14:57:01.557066+0000","last_fullsized":"2024-03-23T14:57:01.557066+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:05:00.726517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.518726+0000","last_change":"2024-03-23T14:56:59.664625+0000","last_active":"2024-03-23T14:57:01.518726+0000","last_peered":"2024-03-23T14:57:01.518726+0000","last_clean":"2024-03-23T14:57:01.518726+0000","last_became_active":"2024-03-23T14:56:59.664268+0000","last_became_peered":"2024-03-23T14:56:59.664268+0000","last_unstale":"2024-03-23T14:57:01.518726+0000","last_undegraded":"2024-03-23T14:57:01.518726+0000","last_fullsized":"2024-03-23T14:57:01.518726+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:53:55.152886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.517881+0000","last_change":"2024-03-23T14:56:48.722063+0000","last_active":"2024-03-23T14:57:01.517881+0000","last_peered":"2024-03-23T14:57:01.517881+0000","last_clean":"2024-03-23T14:57:01.517881+0000","last_became_active":"2024-03-23T14:56:48.720079+0000","last_became_peered":"2024-03-23T14:56:48.720079+0000","last_unstale":"2024-03-23T14:57:01.517881+0000","last_undegraded":"2024-03-23T14:57:01.517881+0000","last_fullsized":"2024-03-23T14:57:01.517881+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:47.478078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:47.478078+0000","last_clean_scrub_stamp":"2024-03-23T14:56:47.478078+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:55:51.231901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.551138+0000","last_change":"2024-03-23T14:56:58.611796+0000","last_active":"2024-03-23T14:57:01.551138+0000","last_peered":"2024-03-23T14:57:01.551138+0000","last_clean":"2024-03-23T14:57:01.551138+0000","last_became_active":"2024-03-23T14:56:58.611243+0000","last_became_peered":"2024-03-23T14:56:58.611243+0000","last_unstale":"2024-03-23T14:57:01.551138+0000","last_undegraded":"2024-03-23T14:57:01.551138+0000","last_fullsized":"2024-03-23T14:57:01.551138+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:24:18.799175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.618598+0000","last_change":"2024-03-23T14:56:59.664611+0000","last_active":"2024-03-23T14:57:00.618598+0000","last_peered":"2024-03-23T14:57:00.618598+0000","last_clean":"2024-03-23T14:57:00.618598+0000","last_became_active":"2024-03-23T14:56:59.664275+0000","last_became_peered":"2024-03-23T14:56:59.664275+0000","last_unstale":"2024-03-23T14:57:00.618598+0000","last_undegraded":"2024-03-23T14:57:00.618598+0000","last_fullsized":"2024-03-23T14:57:00.618598+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:12:59.763661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.517853+0000","last_change":"2024-03-23T14:56:58.620960+0000","last_active":"2024-03-23T14:57:01.517853+0000","last_peered":"2024-03-23T14:57:01.517853+0000","last_clean":"2024-03-23T14:57:01.517853+0000","last_became_active":"2024-03-23T14:56:58.620592+0000","last_became_peered":"2024-03-23T14:56:58.620592+0000","last_unstale":"2024-03-23T14:57:01.517853+0000","last_undegraded":"2024-03-23T14:57:01.517853+0000","last_fullsized":"2024-03-23T14:57:01.517853+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:01:25.160372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.556871+0000","last_change":"2024-03-23T14:56:59.621260+0000","last_active":"2024-03-23T14:57:01.556871+0000","last_peered":"2024-03-23T14:57:01.556871+0000","last_clean":"2024-03-23T14:57:01.556871+0000","last_became_active":"2024-03-23T14:56:59.620924+0000","last_became_peered":"2024-03-23T14:56:59.620924+0000","last_unstale":"2024-03-23T14:57:01.556871+0000","last_undegraded":"2024-03-23T14:57:01.556871+0000","last_fullsized":"2024-03-23T14:57:01.556871+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:59:19.727729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.555522+0000","last_change":"2024-03-23T14:56:58.611902+0000","last_active":"2024-03-23T14:57:01.555522+0000","last_peered":"2024-03-23T14:57:01.555522+0000","last_clean":"2024-03-23T14:57:01.555522+0000","last_became_active":"2024-03-23T14:56:58.611208+0000","last_became_peered":"2024-03-23T14:56:58.611208+0000","last_unstale":"2024-03-23T14:57:01.555522+0000","last_undegraded":"2024-03-23T14:57:01.555522+0000","last_fullsized":"2024-03-23T14:57:01.555522+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:36:10.376200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.620759+0000","last_change":"2024-03-23T14:56:59.640818+0000","last_active":"2024-03-23T14:57:00.620759+0000","last_peered":"2024-03-23T14:57:00.620759+0000","last_clean":"2024-03-23T14:57:00.620759+0000","last_became_active":"2024-03-23T14:56:59.640493+0000","last_became_peered":"2024-03-23T14:56:59.640493+0000","last_unstale":"2024-03-23T14:57:00.620759+0000","last_undegraded":"2024-03-23T14:57:00.620759+0000","last_fullsized":"2024-03-23T14:57:00.620759+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:38:24.739838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.551684+0000","last_change":"2024-03-23T14:56:58.616782+0000","last_active":"2024-03-23T14:57:01.551684+0000","last_peered":"2024-03-23T14:57:01.551684+0000","last_clean":"2024-03-23T14:57:01.551684+0000","last_became_active":"2024-03-23T14:56:58.615957+0000","last_became_peered":"2024-03-23T14:56:58.615957+0000","last_unstale":"2024-03-23T14:57:01.551684+0000","last_undegraded":"2024-03-23T14:57:01.551684+0000","last_fullsized":"2024-03-23T14:57:01.551684+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:01:25.160372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.619632+0000","last_change":"2024-03-23T14:56:59.623480+0000","last_active":"2024-03-23T14:57:00.619632+0000","last_peered":"2024-03-23T14:57:00.619632+0000","last_clean":"2024-03-23T14:57:00.619632+0000","last_became_active":"2024-03-23T14:56:59.623224+0000","last_became_peered":"2024-03-23T14:56:59.623224+0000","last_unstale":"2024-03-23T14:57:00.619632+0000","last_undegraded":"2024-03-23T14:57:00.619632+0000","last_fullsized":"2024-03-23T14:57:00.619632+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:48:13.397390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.621161+0000","last_change":"2024-03-23T14:56:58.604398+0000","last_active":"2024-03-23T14:57:00.621161+0000","last_peered":"2024-03-23T14:57:00.621161+0000","last_clean":"2024-03-23T14:57:00.621161+0000","last_became_active":"2024-03-23T14:56:58.603192+0000","last_became_peered":"2024-03-23T14:56:58.603192+0000","last_unstale":"2024-03-23T14:57:00.621161+0000","last_undegraded":"2024-03-23T14:57:00.621161+0000","last_fullsized":"2024-03-23T14:57:00.621161+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:24:18.799175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.551563+0000","last_change":"2024-03-23T14:56:59.661728+0000","last_active":"2024-03-23T14:57:01.551563+0000","last_peered":"2024-03-23T14:57:01.551563+0000","last_clean":"2024-03-23T14:57:01.551563+0000","last_became_active":"2024-03-23T14:56:59.656955+0000","last_became_peered":"2024-03-23T14:56:59.656955+0000","last_unstale":"2024-03-23T14:57:01.551563+0000","last_undegraded":"2024-03-23T14:57:01.551563+0000","last_fullsized":"2024-03-23T14:57:01.551563+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:51:59.393587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.518923+0000","last_change":"2024-03-23T14:56:58.619283+0000","last_active":"2024-03-23T14:57:01.518923+0000","last_peered":"2024-03-23T14:57:01.518923+0000","last_clean":"2024-03-23T14:57:01.518923+0000","last_became_active":"2024-03-23T14:56:58.618922+0000","last_became_peered":"2024-03-23T14:56:58.618922+0000","last_unstale":"2024-03-23T14:57:01.518923+0000","last_undegraded":"2024-03-23T14:57:01.518923+0000","last_fullsized":"2024-03-23T14:57:01.518923+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:29:07.131287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.620920+0000","last_change":"2024-03-23T14:56:59.636699+0000","last_active":"2024-03-23T14:57:00.620920+0000","last_peered":"2024-03-23T14:57:00.620920+0000","last_clean":"2024-03-23T14:57:00.620920+0000","last_became_active":"2024-03-23T14:56:59.636376+0000","last_became_peered":"2024-03-23T14:56:59.636376+0000","last_unstale":"2024-03-23T14:57:00.620920+0000","last_undegraded":"2024-03-23T14:57:00.620920+0000","last_fullsized":"2024-03-23T14:57:00.620920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:37:56.745366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.622171+0000","last_change":"2024-03-23T14:56:58.627211+0000","last_active":"2024-03-23T14:57:01.622171+0000","last_peered":"2024-03-23T14:57:01.622171+0000","last_clean":"2024-03-23T14:57:01.622171+0000","last_became_active":"2024-03-23T14:56:58.626779+0000","last_became_peered":"2024-03-23T14:56:58.626779+0000","last_unstale":"2024-03-23T14:57:01.622171+0000","last_undegraded":"2024-03-23T14:57:01.622171+0000","last_fullsized":"2024-03-23T14:57:01.622171+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:01:25.160372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.619749+0000","last_change":"2024-03-23T14:56:59.643640+0000","last_active":"2024-03-23T14:57:00.619749+0000","last_peered":"2024-03-23T14:57:00.619749+0000","last_clean":"2024-03-23T14:57:00.619749+0000","last_became_active":"2024-03-23T14:56:59.643255+0000","last_became_peered":"2024-03-23T14:56:59.643255+0000","last_unstale":"2024-03-23T14:57:00.619749+0000","last_undegraded":"2024-03-23T14:57:00.619749+0000","last_fullsized":"2024-03-23T14:57:00.619749+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:35:26.329408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.529191+0000","last_change":"2024-03-23T14:56:58.623880+0000","last_active":"2024-03-23T14:57:01.529191+0000","last_peered":"2024-03-23T14:57:01.529191+0000","last_clean":"2024-03-23T14:57:01.529191+0000","last_became_active":"2024-03-23T14:56:58.623291+0000","last_became_peered":"2024-03-23T14:56:58.623291+0000","last_unstale":"2024-03-23T14:57:01.529191+0000","last_undegraded":"2024-03-23T14:57:01.529191+0000","last_fullsized":"2024-03-23T14:57:01.529191+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:36:10.376200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.556526+0000","last_change":"2024-03-23T14:56:59.634446+0000","last_active":"2024-03-23T14:57:01.556526+0000","last_peered":"2024-03-23T14:57:01.556526+0000","last_clean":"2024-03-23T14:57:01.556526+0000","last_became_active":"2024-03-23T14:56:59.633946+0000","last_became_peered":"2024-03-23T14:56:59.633946+0000","last_unstale":"2024-03-23T14:57:01.556526+0000","last_undegraded":"2024-03-23T14:57:01.556526+0000","last_fullsized":"2024-03-23T14:57:01.556526+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:09:00.536071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.517807+0000","last_change":"2024-03-23T14:56:58.618776+0000","last_active":"2024-03-23T14:57:01.517807+0000","last_peered":"2024-03-23T14:57:01.517807+0000","last_clean":"2024-03-23T14:57:01.517807+0000","last_became_active":"2024-03-23T14:56:58.618418+0000","last_became_peered":"2024-03-23T14:56:58.618418+0000","last_unstale":"2024-03-23T14:57:01.517807+0000","last_undegraded":"2024-03-23T14:57:01.517807+0000","last_fullsized":"2024-03-23T14:57:01.517807+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:48:12.389105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.619949+0000","last_change":"2024-03-23T14:56:59.633845+0000","last_active":"2024-03-23T14:57:00.619949+0000","last_peered":"2024-03-23T14:57:00.619949+0000","last_clean":"2024-03-23T14:57:00.619949+0000","last_became_active":"2024-03-23T14:56:59.633409+0000","last_became_peered":"2024-03-23T14:56:59.633409+0000","last_unstale":"2024-03-23T14:57:00.619949+0000","last_undegraded":"2024-03-23T14:57:00.619949+0000","last_fullsized":"2024-03-23T14:57:00.619949+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:29:08.139571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.620668+0000","last_change":"2024-03-23T14:56:58.616951+0000","last_active":"2024-03-23T14:57:00.620668+0000","last_peered":"2024-03-23T14:57:00.620668+0000","last_clean":"2024-03-23T14:57:00.620668+0000","last_became_active":"2024-03-23T14:56:58.616383+0000","last_became_peered":"2024-03-23T14:56:58.616383+0000","last_unstale":"2024-03-23T14:57:00.620668+0000","last_undegraded":"2024-03-23T14:57:00.620668+0000","last_fullsized":"2024-03-23T14:57:00.620668+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:48:12.389105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.617942+0000","last_change":"2024-03-23T14:56:59.645384+0000","last_active":"2024-03-23T14:57:00.617942+0000","last_peered":"2024-03-23T14:57:00.617942+0000","last_clean":"2024-03-23T14:57:00.617942+0000","last_became_active":"2024-03-23T14:56:59.643723+0000","last_became_peered":"2024-03-23T14:56:59.643723+0000","last_unstale":"2024-03-23T14:57:00.617942+0000","last_undegraded":"2024-03-23T14:57:00.617942+0000","last_fullsized":"2024-03-23T14:57:00.617942+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:09:00.536071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.620676+0000","last_change":"2024-03-23T14:56:58.622814+0000","last_active":"2024-03-23T14:57:00.620676+0000","last_peered":"2024-03-23T14:57:00.620676+0000","last_clean":"2024-03-23T14:57:00.620676+0000","last_became_active":"2024-03-23T14:56:58.621745+0000","last_became_peered":"2024-03-23T14:56:58.621745+0000","last_unstale":"2024-03-23T14:57:00.620676+0000","last_undegraded":"2024-03-23T14:57:00.620676+0000","last_fullsized":"2024-03-23T14:57:00.620676+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:49:19.974457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.618866+0000","last_change":"2024-03-23T14:56:59.645315+0000","last_active":"2024-03-23T14:57:00.618866+0000","last_peered":"2024-03-23T14:57:00.618866+0000","last_clean":"2024-03-23T14:57:00.618866+0000","last_became_active":"2024-03-23T14:56:59.644287+0000","last_became_peered":"2024-03-23T14:56:59.644287+0000","last_unstale":"2024-03-23T14:57:00.618866+0000","last_undegraded":"2024-03-23T14:57:00.618866+0000","last_fullsized":"2024-03-23T14:57:00.618866+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:57:06.681361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.518700+0000","last_change":"2024-03-23T14:56:58.625848+0000","last_active":"2024-03-23T14:57:01.518700+0000","last_peered":"2024-03-23T14:57:01.518700+0000","last_clean":"2024-03-23T14:57:01.518700+0000","last_became_active":"2024-03-23T14:56:58.625453+0000","last_became_peered":"2024-03-23T14:56:58.625453+0000","last_unstale":"2024-03-23T14:57:01.518700+0000","last_undegraded":"2024-03-23T14:57:01.518700+0000","last_fullsized":"2024-03-23T14:57:01.518700+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:06:05.434722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.529162+0000","last_change":"2024-03-23T14:56:59.635812+0000","last_active":"2024-03-23T14:57:01.529162+0000","last_peered":"2024-03-23T14:57:01.529162+0000","last_clean":"2024-03-23T14:57:01.529162+0000","last_became_active":"2024-03-23T14:56:59.635519+0000","last_became_peered":"2024-03-23T14:56:59.635519+0000","last_unstale":"2024-03-23T14:57:01.529162+0000","last_undegraded":"2024-03-23T14:57:01.529162+0000","last_fullsized":"2024-03-23T14:57:01.529162+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:37:56.745366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.551417+0000","last_change":"2024-03-23T14:56:59.662434+0000","last_active":"2024-03-23T14:57:01.551417+0000","last_peered":"2024-03-23T14:57:01.551417+0000","last_clean":"2024-03-23T14:57:01.551417+0000","last_became_active":"2024-03-23T14:56:59.661514+0000","last_became_peered":"2024-03-23T14:56:59.661514+0000","last_unstale":"2024-03-23T14:57:01.551417+0000","last_undegraded":"2024-03-23T14:57:01.551417+0000","last_fullsized":"2024-03-23T14:57:01.551417+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:34:30.083232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.518270+0000","last_change":"2024-03-23T14:56:58.629744+0000","last_active":"2024-03-23T14:57:01.518270+0000","last_peered":"2024-03-23T14:57:01.518270+0000","last_clean":"2024-03-23T14:57:01.518270+0000","last_became_active":"2024-03-23T14:56:58.629401+0000","last_became_peered":"2024-03-23T14:56:58.629401+0000","last_unstale":"2024-03-23T14:57:01.518270+0000","last_undegraded":"2024-03-23T14:57:01.518270+0000","last_fullsized":"2024-03-23T14:57:01.518270+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:13:45.673576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.551735+0000","last_change":"2024-03-23T14:56:59.658250+0000","last_active":"2024-03-23T14:57:01.551735+0000","last_peered":"2024-03-23T14:57:01.551735+0000","last_clean":"2024-03-23T14:57:01.551735+0000","last_became_active":"2024-03-23T14:56:59.657085+0000","last_became_peered":"2024-03-23T14:56:59.657085+0000","last_unstale":"2024-03-23T14:57:01.551735+0000","last_undegraded":"2024-03-23T14:57:01.551735+0000","last_fullsized":"2024-03-23T14:57:01.551735+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:53:55.152886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.518837+0000","last_change":"2024-03-23T14:56:58.624886+0000","last_active":"2024-03-23T14:57:01.518837+0000","last_peered":"2024-03-23T14:57:01.518837+0000","last_clean":"2024-03-23T14:57:01.518837+0000","last_became_active":"2024-03-23T14:56:58.624478+0000","last_became_peered":"2024-03-23T14:56:58.624478+0000","last_unstale":"2024-03-23T14:57:01.518837+0000","last_undegraded":"2024-03-23T14:57:01.518837+0000","last_fullsized":"2024-03-23T14:57:01.518837+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:49:19.974457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.555868+0000","last_change":"2024-03-23T14:56:59.660089+0000","last_active":"2024-03-23T14:57:01.555868+0000","last_peered":"2024-03-23T14:57:01.555868+0000","last_clean":"2024-03-23T14:57:01.555868+0000","last_became_active":"2024-03-23T14:56:59.659469+0000","last_became_peered":"2024-03-23T14:56:59.659469+0000","last_unstale":"2024-03-23T14:57:01.555868+0000","last_undegraded":"2024-03-23T14:57:01.555868+0000","last_fullsized":"2024-03-23T14:57:01.555868+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:37:14.115808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.619024+0000","last_change":"2024-03-23T14:56:58.608147+0000","last_active":"2024-03-23T14:57:00.619024+0000","last_peered":"2024-03-23T14:57:00.619024+0000","last_clean":"2024-03-23T14:57:00.619024+0000","last_became_active":"2024-03-23T14:56:58.603630+0000","last_became_peered":"2024-03-23T14:56:58.603630+0000","last_unstale":"2024-03-23T14:57:00.619024+0000","last_undegraded":"2024-03-23T14:57:00.619024+0000","last_fullsized":"2024-03-23T14:57:00.619024+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:08:31.251287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.618985+0000","last_change":"2024-03-23T14:56:59.645116+0000","last_active":"2024-03-23T14:57:00.618985+0000","last_peered":"2024-03-23T14:57:00.618985+0000","last_clean":"2024-03-23T14:57:00.618985+0000","last_became_active":"2024-03-23T14:56:59.644484+0000","last_became_peered":"2024-03-23T14:56:59.644484+0000","last_unstale":"2024-03-23T14:57:00.618985+0000","last_undegraded":"2024-03-23T14:57:00.618985+0000","last_fullsized":"2024-03-23T14:57:00.618985+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:39:22.051918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.556818+0000","last_change":"2024-03-23T14:56:58.619314+0000","last_active":"2024-03-23T14:57:01.556818+0000","last_peered":"2024-03-23T14:57:01.556818+0000","last_clean":"2024-03-23T14:57:01.556818+0000","last_became_active":"2024-03-23T14:56:58.618751+0000","last_became_peered":"2024-03-23T14:56:58.618751+0000","last_unstale":"2024-03-23T14:57:01.556818+0000","last_undegraded":"2024-03-23T14:57:01.556818+0000","last_fullsized":"2024-03-23T14:57:01.556818+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:16:22.644801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27800,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344040,"statfs":{"total":96636764160,"available":96608296960,"internally_reserved":0,"allocated":790528,"data_stored":655584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":780,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":798720,"data_stored":655640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-23T14:57:07.396 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-23T14:57:07.396 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-03-23T14:57:07.396 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-03-23T14:57:07.397 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-03-23T14:57:07.397 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-03-23T14:57:07.560 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.557+0000 7fa6cb071640 1 Processor -- start 2024-03-23T14:57:07.561 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.557+0000 7fa6cb071640 1 -- start start 2024-03-23T14:57:07.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.557+0000 7fa6cb071640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fa6c4141640 0x7fa6c4152c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:07.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.557+0000 7fa6cb071640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6c40b4300 con 0x7fa6c40b4a70 2024-03-23T14:57:07.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.557+0000 7fa6cb071640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6c40b45e0 con 0x7fa6c40b56f0 2024-03-23T14:57:07.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.557+0000 7fa6cb071640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 -- 0x7fa6c40b5fa0 con 0x7fa6c4141640 2024-03-23T14:57:07.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.557+0000 7fa6bbfff640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fa6c40b4a70 0x7fa6c4140f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:42400/0 (socket says 172.21.15.164:42400) 2024-03-23T14:57:07.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.557+0000 7fa6bbfff640 1 -- 172.21.15.164:0/208639450 learned_addr learned my addr 172.21.15.164:0/208639450 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-03-23T14:57:07.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.557+0000 7fa6c960e640 1 --2- 172.21.15.164:0/208639450 >> v2:172.21.15.173:3300/0 conn(0x7fa6c4141640 0x7fa6c4152c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:07.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.557+0000 7fa6bb7fe640 1 -- 172.21.15.164:0/208639450 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 782181907 0 0) 0x7fa6c40b4300 con 0x7fa6c40b4a70 2024-03-23T14:57:07.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.557+0000 7fa6bb7fe640 1 -- 172.21.15.164:0/208639450 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa6ac003530 con 0x7fa6c40b4a70 2024-03-23T14:57:07.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.557+0000 7fa6bb7fe640 1 -- 172.21.15.164:0/208639450 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2509694746 0 0) 0x7fa6ac003530 con 0x7fa6c40b4a70 2024-03-23T14:57:07.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.557+0000 7fa6bb7fe640 1 -- 172.21.15.164:0/208639450 >> v2:172.21.15.173:3300/0 conn(0x7fa6c4141640 msgr2=0x7fa6c4152c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:07.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.557+0000 7fa6bb7fe640 1 --2- 172.21.15.164:0/208639450 >> v2:172.21.15.173:3300/0 conn(0x7fa6c4141640 0x7fa6c4152c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:07.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.557+0000 7fa6bb7fe640 1 -- 172.21.15.164:0/208639450 >> v1:172.21.15.173:6789/0 conn(0x7fa6c40b56f0 legacy=0x7fa6c414a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:07.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.557+0000 7fa6bb7fe640 1 -- 172.21.15.164:0/208639450 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6c40bd7e0 con 0x7fa6c40b4a70 2024-03-23T14:57:07.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.557+0000 7fa6bb7fe640 1 -- 172.21.15.164:0/208639450 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2526390548 0 0) 0x7fa6b4001680 con 0x7fa6c40b4a70 2024-03-23T14:57:07.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.557+0000 7fa6bb7fe640 1 -- 172.21.15.164:0/208639450 >> v1:172.21.15.164:6789/0 conn(0x7fa6c40b4a70 legacy=0x7fa6c4140f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:07.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.557+0000 7fa6bb7fe640 1 --2- 172.21.15.164:0/208639450 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa6ac003980 0x7fa6ac003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:07.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.557+0000 7fa6bb7fe640 1 --2- 172.21.15.164:0/208639450 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa6ac0047d0 0x7fa6ac004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:07.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.557+0000 7fa6bb7fe640 1 --2- 172.21.15.164:0/208639450 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa6ac005190 0x7fa6ac00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:07.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.557+0000 7fa6bb7fe640 1 -- 172.21.15.164:0/208639450 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa6ac00c000 con 0x7fa6ac005190 2024-03-23T14:57:07.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.557+0000 7fa6bb7fe640 1 -- 172.21.15.164:0/208639450 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa6ac00c170 con 0x7fa6ac0047d0 2024-03-23T14:57:07.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.557+0000 7fa6bb7fe640 1 -- 172.21.15.164:0/208639450 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fa6ac00c330 con 0x7fa6ac003980 2024-03-23T14:57:07.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.557+0000 7fa6c960e640 1 --2- 172.21.15.164:0/208639450 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa6ac005190 0x7fa6ac00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:07.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.557+0000 7fa6bbfff640 1 --2- 172.21.15.164:0/208639450 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa6ac0047d0 0x7fa6ac004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:07.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.557+0000 7fa6c8e0d640 1 --2- 172.21.15.164:0/208639450 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa6ac003980 0x7fa6ac003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:07.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.557+0000 7fa6c960e640 1 -- 172.21.15.164:0/208639450 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa6ac003980 msgr2=0x7fa6ac003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:07.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.557+0000 7fa6c960e640 1 --2- 172.21.15.164:0/208639450 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa6ac003980 0x7fa6ac003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:07.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.557+0000 7fa6c960e640 1 -- 172.21.15.164:0/208639450 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa6ac0047d0 msgr2=0x7fa6ac004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:07.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.557+0000 7fa6c960e640 1 --2- 172.21.15.164:0/208639450 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa6ac0047d0 0x7fa6ac004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:07.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.557+0000 7fa6c960e640 1 -- 172.21.15.164:0/208639450 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6ac003530 con 0x7fa6ac005190 2024-03-23T14:57:07.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.557+0000 7fa6bbfff640 1 --2- 172.21.15.164:0/208639450 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa6ac0047d0 0x7fa6ac004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T14:57:07.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.561+0000 7fa6c960e640 1 -- 172.21.15.164:0/208639450 reap_dead start 2024-03-23T14:57:07.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.561+0000 7fa6c960e640 1 --2- 172.21.15.164:0/208639450 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa6ac005190 0x7fa6ac00b990 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc009fc0 tx=0x7fa6bc01ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37da60a86496e6cc server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:07.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.561+0000 7fa6bb7fe640 1 -- 172.21.15.164:0/208639450 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6bc0a5cd0 con 0x7fa6ac005190 2024-03-23T14:57:07.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.561+0000 7fa6bb7fe640 1 -- 172.21.15.164:0/208639450 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6bc00f040 con 0x7fa6ac005190 2024-03-23T14:57:07.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.561+0000 7fa6bb7fe640 1 -- 172.21.15.164:0/208639450 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6bc003be0 con 0x7fa6ac005190 2024-03-23T14:57:07.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.561+0000 7fa6cb071640 1 -- 172.21.15.164:0/208639450 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa6ac005190 msgr2=0x7fa6ac00b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:07.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.561+0000 7fa6cb071640 1 --2- 172.21.15.164:0/208639450 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa6ac005190 0x7fa6ac00b990 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc009fc0 tx=0x7fa6bc01ccc0 comp rx=0 tx=0).stop 2024-03-23T14:57:07.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.561+0000 7fa6cb071640 1 -- 172.21.15.164:0/208639450 shutdown_connections 2024-03-23T14:57:07.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.561+0000 7fa6cb071640 1 --2- 172.21.15.164:0/208639450 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa6ac005190 0x7fa6ac00b990 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:07.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.561+0000 7fa6cb071640 1 -- 172.21.15.164:0/208639450 >> 172.21.15.164:0/208639450 conn(0x7fa6c402c3c0 msgr2=0x7fa6c402c7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:07.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.561+0000 7fa6cb071640 1 -- 172.21.15.164:0/208639450 shutdown_connections 2024-03-23T14:57:07.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.561+0000 7fa6cb071640 1 -- 172.21.15.164:0/208639450 wait complete. 2024-03-23T14:57:07.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.561+0000 7fa6cb071640 1 Processor -- start 2024-03-23T14:57:07.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.561+0000 7fa6cb071640 1 -- start start 2024-03-23T14:57:07.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.561+0000 7fa6cb071640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa6ac005190 0x7fa6c4144440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:07.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.561+0000 7fa6cb071640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa6c4144980 0x7fa6c414aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:07.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.561+0000 7fa6cb071640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa6c414b3f0 0x7fa6c41dfeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:07.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.561+0000 7fa6cb071640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa6c4149af0 con 0x7fa6ac005190 2024-03-23T14:57:07.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.561+0000 7fa6cb071640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa6c4149700 con 0x7fa6c4144980 2024-03-23T14:57:07.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.561+0000 7fa6cb071640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fa6c41dfa60 con 0x7fa6c414b3f0 2024-03-23T14:57:07.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.561+0000 7fa6c8e0d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa6ac005190 0x7fa6c4144440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:07.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.561+0000 7fa6c8e0d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa6ac005190 0x7fa6c4144440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:59040/0 (socket says 172.21.15.164:59040) 2024-03-23T14:57:07.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.561+0000 7fa6c8e0d640 1 -- 172.21.15.164:0/609437188 learned_addr learned my addr 172.21.15.164:0/609437188 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:07.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.561+0000 7fa6c8e0d640 1 -- 172.21.15.164:0/609437188 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa6c414b3f0 msgr2=0x7fa6c41dfeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:07.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.561+0000 7fa6c960e640 1 --2- 172.21.15.164:0/609437188 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa6c414b3f0 0x7fa6c41dfeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:07.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.561+0000 7fa6bbfff640 1 --2- 172.21.15.164:0/609437188 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa6c4144980 0x7fa6c414aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:07.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.561+0000 7fa6c8e0d640 1 --2- 172.21.15.164:0/609437188 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa6c414b3f0 0x7fa6c41dfeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:07.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.561+0000 7fa6c8e0d640 1 -- 172.21.15.164:0/609437188 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa6c4144980 msgr2=0x7fa6c414aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:07.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.561+0000 7fa6c8e0d640 1 --2- 172.21.15.164:0/609437188 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa6c4144980 0x7fa6c414aeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:07.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.561+0000 7fa6c8e0d640 1 -- 172.21.15.164:0/609437188 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6c41434e0 con 0x7fa6ac005190 2024-03-23T14:57:07.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.561+0000 7fa6c8e0d640 1 --2- 172.21.15.164:0/609437188 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa6ac005190 0x7fa6c4144440 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fa6c000e4b0 tx=0x7fa6c000e970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:07.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.565+0000 7fa6b97fa640 1 -- 172.21.15.164:0/609437188 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6c00052e0 con 0x7fa6ac005190 2024-03-23T14:57:07.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.565+0000 7fa6b97fa640 1 -- 172.21.15.164:0/609437188 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6c0005440 con 0x7fa6ac005190 2024-03-23T14:57:07.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.565+0000 7fa6cb071640 1 -- 172.21.15.164:0/609437188 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6c4149140 con 0x7fa6ac005190 2024-03-23T14:57:07.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.565+0000 7fa6b97fa640 1 -- 172.21.15.164:0/609437188 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6c0003840 con 0x7fa6ac005190 2024-03-23T14:57:07.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.565+0000 7fa6cb071640 1 -- 172.21.15.164:0/609437188 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6c414a750 con 0x7fa6ac005190 2024-03-23T14:57:07.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.565+0000 7fa6cb071640 1 -- 172.21.15.164:0/609437188 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6c4148fb0 con 0x7fa6ac005190 2024-03-23T14:57:07.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.565+0000 7fa6b97fa640 1 -- 172.21.15.164:0/609437188 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7fa6c00039a0 con 0x7fa6ac005190 2024-03-23T14:57:07.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.565+0000 7fa6b97fa640 1 --2- 172.21.15.164:0/609437188 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fa6b00672a0 0x7fa6b0069760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:07.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.565+0000 7fa6b97fa640 1 -- 172.21.15.164:0/609437188 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa6c0084ce0 con 0x7fa6ac005190 2024-03-23T14:57:07.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.565+0000 7fa6bbfff640 1 --2- 172.21.15.164:0/609437188 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fa6b00672a0 0x7fa6b0069760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:07.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.565+0000 7fa6bbfff640 1 --2- 172.21.15.164:0/609437188 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fa6b00672a0 0x7fa6b0069760 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa6b4001210 tx=0x7fa6b4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:07.583 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.569+0000 7fa6b97fa640 1 -- 172.21.15.164:0/609437188 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7fa6c0051920 con 0x7fa6ac005190 2024-03-23T14:57:07.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.765+0000 7fa6cb071640 1 -- 172.21.15.164:0/609437188 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fa6c4148fb0 con 0x7fa6ac005190 2024-03-23T14:57:07.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.773+0000 7fa6b97fa640 1 -- 172.21.15.164:0/609437188 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+165740 (secure 0 0 0) 0x7fa6c0054fe0 con 0x7fa6ac005190 2024-03-23T14:57:07.779 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-23T14:57:07.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.777+0000 7fa6cb071640 1 -- 172.21.15.164:0/609437188 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fa6b00672a0 msgr2=0x7fa6b0069760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:07.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.777+0000 7fa6cb071640 1 --2- 172.21.15.164:0/609437188 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fa6b00672a0 0x7fa6b0069760 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa6b4001210 tx=0x7fa6b4009040 comp rx=0 tx=0).stop 2024-03-23T14:57:07.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.777+0000 7fa6cb071640 1 -- 172.21.15.164:0/609437188 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa6ac005190 msgr2=0x7fa6c4144440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:07.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.777+0000 7fa6cb071640 1 --2- 172.21.15.164:0/609437188 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa6ac005190 0x7fa6c4144440 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fa6c000e4b0 tx=0x7fa6c000e970 comp rx=0 tx=0).stop 2024-03-23T14:57:07.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.777+0000 7fa6cb071640 1 -- 172.21.15.164:0/609437188 shutdown_connections 2024-03-23T14:57:07.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.777+0000 7fa6cb071640 1 --2- 172.21.15.164:0/609437188 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa6c414b3f0 0x7fa6c41dfeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:07.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.777+0000 7fa6cb071640 1 --2- 172.21.15.164:0/609437188 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fa6b00672a0 0x7fa6b0069760 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:07.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.777+0000 7fa6cb071640 1 --2- 172.21.15.164:0/609437188 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa6c4144980 0x7fa6c414aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:07.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.777+0000 7fa6cb071640 1 --2- 172.21.15.164:0/609437188 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa6ac005190 0x7fa6c4144440 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:07.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.777+0000 7fa6cb071640 1 -- 172.21.15.164:0/609437188 >> 172.21.15.164:0/609437188 conn(0x7fa6c402c3c0 msgr2=0x7fa6c40b7d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:07.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.777+0000 7fa6cb071640 1 -- 172.21.15.164:0/609437188 shutdown_connections 2024-03-23T14:57:07.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.777+0000 7fa6cb071640 1 -- 172.21.15.164:0/609437188 wait complete. 2024-03-23T14:57:07.796 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":5,"flags":0,"active_gid":4107,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":2139087985},{"type":"v1","addr":"172.21.15.164:6833","nonce":2139087985}]},"active_addr":"172.21.15.164:6833/2139087985","active_change":"2024-03-23T14:56:39.097865+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4110,"name":"x","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.164:0","nonce":3627620542}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.164:0","nonce":3200185440}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.164:0","nonce":3239881732}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.164:0","nonce":3336279675}]}]} 2024-03-23T14:57:07.799 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-03-23T14:57:07.799 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-03-23T14:57:07.799 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T14:57:07.919 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.913+0000 7f0540517640 1 Processor -- start 2024-03-23T14:57:07.919 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.913+0000 7f0540517640 1 -- start start 2024-03-23T14:57:07.919 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.913+0000 7f0540517640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f05380b0230 0x7f05380b34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:07.919 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.913+0000 7f0540517640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f0538148cf0 0x7f0538149180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:07.920 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.913+0000 7f0540517640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05380b51d0 con 0x7f05380b1c70 2024-03-23T14:57:07.920 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.913+0000 7f0540517640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f05380b0610 con 0x7f0538148cf0 2024-03-23T14:57:07.920 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.913+0000 7f0540517640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 -- 0x7f05380b1500 con 0x7f05380b0230 2024-03-23T14:57:07.920 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.913+0000 7f053dab2640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f0538148cf0 0x7f0538149180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:07.920 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.913+0000 7f053dab2640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f0538148cf0 0x7f0538149180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:59056/0 (socket says 172.21.15.164:59056) 2024-03-23T14:57:07.920 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.913+0000 7f053dab2640 1 -- 172.21.15.164:0/402580309 learned_addr learned my addr 172.21.15.164:0/402580309 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:07.920 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.913+0000 7f053e2b3640 1 --2- 172.21.15.164:0/402580309 >> v2:172.21.15.173:3300/0 conn(0x7f05380b0230 0x7f05380b34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:07.920 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.913+0000 7f053dab2640 1 -- 172.21.15.164:0/402580309 >> v2:172.21.15.173:3300/0 conn(0x7f05380b0230 msgr2=0x7f05380b34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:07.920 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.913+0000 7f053dab2640 1 --2- 172.21.15.164:0/402580309 >> v2:172.21.15.173:3300/0 conn(0x7f05380b0230 0x7f05380b34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:07.920 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.913+0000 7f053dab2640 1 -- 172.21.15.164:0/402580309 >> v1:172.21.15.173:6789/0 conn(0x7f05380b1c70 legacy=0x7f05380b20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:07.920 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.913+0000 7f053dab2640 1 -- 172.21.15.164:0/402580309 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f053806d250 con 0x7f0538148cf0 2024-03-23T14:57:07.920 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.913+0000 7f053e2b3640 1 --2- 172.21.15.164:0/402580309 >> v2:172.21.15.173:3300/0 conn(0x7f05380b0230 0x7f05380b34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T14:57:07.922 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.917+0000 7f053dab2640 1 --2- 172.21.15.164:0/402580309 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0538148cf0 0x7f0538149180 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f0528009870 tx=0x7f052801c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3538e19b7de48357 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:07.922 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.917+0000 7f053d2b1640 1 -- 172.21.15.164:0/402580309 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05280ad640 con 0x7f0538148cf0 2024-03-23T14:57:07.922 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.917+0000 7f053d2b1640 1 -- 172.21.15.164:0/402580309 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05280ad7a0 con 0x7f0538148cf0 2024-03-23T14:57:07.922 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.917+0000 7f053d2b1640 1 -- 172.21.15.164:0/402580309 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05280adb30 con 0x7f0538148cf0 2024-03-23T14:57:07.923 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.917+0000 7f0540517640 1 -- 172.21.15.164:0/402580309 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0538148cf0 msgr2=0x7f0538149180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:07.923 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.917+0000 7f0540517640 1 --2- 172.21.15.164:0/402580309 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0538148cf0 0x7f0538149180 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f0528009870 tx=0x7f052801c390 comp rx=0 tx=0).stop 2024-03-23T14:57:07.923 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.917+0000 7f0540517640 1 -- 172.21.15.164:0/402580309 shutdown_connections 2024-03-23T14:57:07.923 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.917+0000 7f0540517640 1 --2- 172.21.15.164:0/402580309 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0538148cf0 0x7f0538149180 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:07.923 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.917+0000 7f0540517640 1 --2- 172.21.15.164:0/402580309 >> v2:172.21.15.173:3300/0 conn(0x7f05380b0230 0x7f05380b34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:07.923 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.917+0000 7f0540517640 1 -- 172.21.15.164:0/402580309 >> 172.21.15.164:0/402580309 conn(0x7f05380b8ca0 msgr2=0x7f05380b90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:07.923 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.917+0000 7f0540517640 1 -- 172.21.15.164:0/402580309 shutdown_connections 2024-03-23T14:57:07.923 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.917+0000 7f0540517640 1 -- 172.21.15.164:0/402580309 wait complete. 2024-03-23T14:57:07.923 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.917+0000 7f0540517640 1 Processor -- start 2024-03-23T14:57:07.923 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.917+0000 7f0540517640 1 -- start start 2024-03-23T14:57:07.925 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.921+0000 7f0540517640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f05380b0230 0x7f0538145100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:07.925 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.921+0000 7f0540517640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f05380b1c70 0x7f05381dd6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:07.926 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.921+0000 7f0540517640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0538148cf0 0x7f053809cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:07.926 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.921+0000 7f0540517640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f053809b290 con 0x7f05380b1c70 2024-03-23T14:57:07.926 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.921+0000 7f0540517640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f05381de6c0 con 0x7f05380b0230 2024-03-23T14:57:07.926 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.921+0000 7f0540517640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f05381ddc20 con 0x7f0538148cf0 2024-03-23T14:57:07.926 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.921+0000 7f053dab2640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f05380b1c70 0x7f05381dd6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:07.926 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.921+0000 7f053dab2640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f05380b1c70 0x7f05381dd6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:59064/0 (socket says 172.21.15.164:59064) 2024-03-23T14:57:07.926 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.921+0000 7f053dab2640 1 -- 172.21.15.164:0/3393261235 learned_addr learned my addr 172.21.15.164:0/3393261235 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:07.926 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.921+0000 7f053e2b3640 1 --2- 172.21.15.164:0/3393261235 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f05380b0230 0x7f0538145100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:07.926 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.921+0000 7f053dab2640 1 -- 172.21.15.164:0/3393261235 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0538148cf0 msgr2=0x7f053809cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:07.926 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.921+0000 7f053eab4640 1 --2- 172.21.15.164:0/3393261235 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0538148cf0 0x7f053809cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:07.926 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.921+0000 7f053dab2640 1 --2- 172.21.15.164:0/3393261235 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0538148cf0 0x7f053809cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:07.926 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.921+0000 7f053dab2640 1 -- 172.21.15.164:0/3393261235 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f05380b0230 msgr2=0x7f0538145100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:07.926 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.921+0000 7f053dab2640 1 --2- 172.21.15.164:0/3393261235 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f05380b0230 0x7f0538145100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:07.926 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.921+0000 7f053dab2640 1 -- 172.21.15.164:0/3393261235 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05280094d0 con 0x7f05380b1c70 2024-03-23T14:57:07.926 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.921+0000 7f053e2b3640 1 --2- 172.21.15.164:0/3393261235 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f05380b0230 0x7f0538145100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T14:57:07.927 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.921+0000 7f053dab2640 1 --2- 172.21.15.164:0/3393261235 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f05380b1c70 0x7f05381dd6e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f05280b0040 tx=0x7f0528019130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:07.927 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.921+0000 7f052effd640 1 -- 172.21.15.164:0/3393261235 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05280bc070 con 0x7f05380b1c70 2024-03-23T14:57:07.927 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.921+0000 7f052effd640 1 -- 172.21.15.164:0/3393261235 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05280a6040 con 0x7f05380b1c70 2024-03-23T14:57:07.927 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.921+0000 7f0540517640 1 -- 172.21.15.164:0/3393261235 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f053813d510 con 0x7f05380b1c70 2024-03-23T14:57:07.927 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.921+0000 7f052effd640 1 -- 172.21.15.164:0/3393261235 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05280b7870 con 0x7f05380b1c70 2024-03-23T14:57:07.927 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.921+0000 7f0540517640 1 -- 172.21.15.164:0/3393261235 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f053813dad0 con 0x7f05380b1c70 2024-03-23T14:57:07.927 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.921+0000 7f0540517640 1 -- 172.21.15.164:0/3393261235 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05380b3990 con 0x7f05380b1c70 2024-03-23T14:57:07.927 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.921+0000 7f052effd640 1 -- 172.21.15.164:0/3393261235 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7f05280b79d0 con 0x7f05380b1c70 2024-03-23T14:57:07.928 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.921+0000 7f052effd640 1 --2- 172.21.15.164:0/3393261235 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f051c0672a0 0x7f051c069760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:07.928 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.921+0000 7f052effd640 1 -- 172.21.15.164:0/3393261235 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f0528120210 con 0x7f05380b1c70 2024-03-23T14:57:07.928 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.921+0000 7f053e2b3640 1 --2- 172.21.15.164:0/3393261235 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f051c0672a0 0x7f051c069760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:07.928 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.921+0000 7f053e2b3640 1 --2- 172.21.15.164:0/3393261235 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f051c0672a0 0x7f051c069760 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0534008890 tx=0x7f0534008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:07.930 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:07.925+0000 7f052effd640 1 -- 172.21.15.164:0/3393261235 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7f05280ecf30 con 0x7f05380b1c70 2024-03-23T14:57:08.079 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.073+0000 7f0540517640 1 -- 172.21.15.164:0/3393261235 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f05380690a0 con 0x7f05380b1c70 2024-03-23T14:57:08.082 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.077+0000 7f052effd640 1 -- 172.21.15.164:0/3393261235 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16629 (secure 0 0 0) 0x7f05280f05f0 con 0x7f05380b1c70 2024-03-23T14:57:08.082 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-23T14:57:08.082 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":20,"fsid":"45c7911b-b69d-4e68-907b-fbfeb780aa3b","created":"2024-03-23T14:56:34.367633+0000","modified":"2024-03-23T14:57:00.611265+0000","last_up_change":"2024-03-23T14:56:46.297403+0000","last_in_change":"2024-03-23T14:56:36.629884+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T14:56:47.186157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-23T14:56:56.994057+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-23T14:56:57.953862+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"80abdc62-bd0d-4e60-a554-7ed6fd67cf54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":984716690},{"type":"v1","addr":"172.21.15.164:6801","nonce":984716690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":984716690},{"type":"v1","addr":"172.21.15.164:6803","nonce":984716690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":984716690},{"type":"v1","addr":"172.21.15.164:6807","nonce":984716690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":984716690},{"type":"v1","addr":"172.21.15.164:6805","nonce":984716690}]},"public_addr":"172.21.15.164:6801/984716690","cluster_addr":"172.21.15.164:6803/984716690","heartbeat_back_addr":"172.21.15.164:6807/984716690","heartbeat_front_addr":"172.21.15.164:6805/984716690","state":["exists","up"]},{"osd":1,"uuid":"f9001b06-be4d-4f42-a0e6-bcef2afa0d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":1349657239},{"type":"v1","addr":"172.21.15.164:6817","nonce":1349657239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":1349657239},{"type":"v1","addr":"172.21.15.164:6819","nonce":1349657239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":1349657239},{"type":"v1","addr":"172.21.15.164:6823","nonce":1349657239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":1349657239},{"type":"v1","addr":"172.21.15.164:6821","nonce":1349657239}]},"public_addr":"172.21.15.164:6817/1349657239","cluster_addr":"172.21.15.164:6819/1349657239","heartbeat_back_addr":"172.21.15.164:6823/1349657239","heartbeat_front_addr":"172.21.15.164:6821/1349657239","state":["exists","up"]},{"osd":2,"uuid":"fa0aeb9f-8b54-4831-a188-3e361c7980d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":887861252},{"type":"v1","addr":"172.21.15.164:6809","nonce":887861252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":887861252},{"type":"v1","addr":"172.21.15.164:6811","nonce":887861252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":887861252},{"type":"v1","addr":"172.21.15.164:6815","nonce":887861252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":887861252},{"type":"v1","addr":"172.21.15.164:6813","nonce":887861252}]},"public_addr":"172.21.15.164:6809/887861252","cluster_addr":"172.21.15.164:6811/887861252","heartbeat_back_addr":"172.21.15.164:6815/887861252","heartbeat_front_addr":"172.21.15.164:6813/887861252","state":["exists","up"]},{"osd":3,"uuid":"7e18c957-3091-46d5-8e17-8f3354c1726c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":812540271},{"type":"v1","addr":"172.21.15.164:6825","nonce":812540271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":812540271},{"type":"v1","addr":"172.21.15.164:6827","nonce":812540271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":812540271},{"type":"v1","addr":"172.21.15.164:6831","nonce":812540271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":812540271},{"type":"v1","addr":"172.21.15.164:6829","nonce":812540271}]},"public_addr":"172.21.15.164:6825/812540271","cluster_addr":"172.21.15.164:6827/812540271","heartbeat_back_addr":"172.21.15.164:6831/812540271","heartbeat_front_addr":"172.21.15.164:6829/812540271","state":["exists","up"]},{"osd":4,"uuid":"8e83f315-42bf-4471-9ce8-d81948da0f9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":2656857421},{"type":"v1","addr":"172.21.15.173:6825","nonce":2656857421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2656857421},{"type":"v1","addr":"172.21.15.173:6827","nonce":2656857421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2656857421},{"type":"v1","addr":"172.21.15.173:6831","nonce":2656857421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":2656857421},{"type":"v1","addr":"172.21.15.173:6829","nonce":2656857421}]},"public_addr":"172.21.15.173:6825/2656857421","cluster_addr":"172.21.15.173:6827/2656857421","heartbeat_back_addr":"172.21.15.173:6831/2656857421","heartbeat_front_addr":"172.21.15.173:6829/2656857421","state":["exists","up"]},{"osd":5,"uuid":"a230c872-71f9-40ab-8999-defd0162d4ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":3676483185},{"type":"v1","addr":"172.21.15.173:6801","nonce":3676483185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":3676483185},{"type":"v1","addr":"172.21.15.173:6803","nonce":3676483185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":3676483185},{"type":"v1","addr":"172.21.15.173:6807","nonce":3676483185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":3676483185},{"type":"v1","addr":"172.21.15.173:6805","nonce":3676483185}]},"public_addr":"172.21.15.173:6801/3676483185","cluster_addr":"172.21.15.173:6803/3676483185","heartbeat_back_addr":"172.21.15.173:6807/3676483185","heartbeat_front_addr":"172.21.15.173:6805/3676483185","state":["exists","up"]},{"osd":6,"uuid":"6193bfd7-8432-4f7f-a117-a3fcee21af99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":3631684087},{"type":"v1","addr":"172.21.15.173:6809","nonce":3631684087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":3631684087},{"type":"v1","addr":"172.21.15.173:6811","nonce":3631684087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":3631684087},{"type":"v1","addr":"172.21.15.173:6815","nonce":3631684087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":3631684087},{"type":"v1","addr":"172.21.15.173:6813","nonce":3631684087}]},"public_addr":"172.21.15.173:6809/3631684087","cluster_addr":"172.21.15.173:6811/3631684087","heartbeat_back_addr":"172.21.15.173:6815/3631684087","heartbeat_front_addr":"172.21.15.173:6813/3631684087","state":["exists","up"]},{"osd":7,"uuid":"b0f1b80a-afc4-4538-83b9-f081d2dcb8cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":3261114948},{"type":"v1","addr":"172.21.15.173:6817","nonce":3261114948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":3261114948},{"type":"v1","addr":"172.21.15.173:6819","nonce":3261114948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":3261114948},{"type":"v1","addr":"172.21.15.173:6823","nonce":3261114948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":3261114948},{"type":"v1","addr":"172.21.15.173:6821","nonce":3261114948}]},"public_addr":"172.21.15.173:6817/3261114948","cluster_addr":"172.21.15.173:6819/3261114948","heartbeat_back_addr":"172.21.15.173:6823/3261114948","heartbeat_front_addr":"172.21.15.173:6821/3261114948","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.188390+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.159932+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.191101+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.181776+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.224365+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.177515+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.153631+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.139972+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T14:57:08.083 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.077+0000 7f0540517640 1 -- 172.21.15.164:0/3393261235 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f051c0672a0 msgr2=0x7f051c069760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:08.083 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.077+0000 7f0540517640 1 --2- 172.21.15.164:0/3393261235 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f051c0672a0 0x7f051c069760 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0534008890 tx=0x7f0534008040 comp rx=0 tx=0).stop 2024-03-23T14:57:08.084 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.077+0000 7f0540517640 1 -- 172.21.15.164:0/3393261235 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f05380b1c70 msgr2=0x7f05381dd6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:08.084 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.077+0000 7f0540517640 1 --2- 172.21.15.164:0/3393261235 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f05380b1c70 0x7f05381dd6e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f05280b0040 tx=0x7f0528019130 comp rx=0 tx=0).stop 2024-03-23T14:57:08.084 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.077+0000 7f0540517640 1 -- 172.21.15.164:0/3393261235 shutdown_connections 2024-03-23T14:57:08.084 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.077+0000 7f0540517640 1 --2- 172.21.15.164:0/3393261235 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0538148cf0 0x7f053809cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:08.084 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.077+0000 7f0540517640 1 --2- 172.21.15.164:0/3393261235 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f05380b1c70 0x7f05381dd6e0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:08.084 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.077+0000 7f0540517640 1 --2- 172.21.15.164:0/3393261235 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f051c0672a0 0x7f051c069760 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:08.084 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.077+0000 7f0540517640 1 --2- 172.21.15.164:0/3393261235 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f05380b0230 0x7f0538145100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:08.084 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.077+0000 7f0540517640 1 -- 172.21.15.164:0/3393261235 >> 172.21.15.164:0/3393261235 conn(0x7f05380b8ca0 msgr2=0x7f05380b5500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:08.084 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.077+0000 7f0540517640 1 -- 172.21.15.164:0/3393261235 shutdown_connections 2024-03-23T14:57:08.084 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.077+0000 7f0540517640 1 -- 172.21.15.164:0/3393261235 wait complete. 2024-03-23T14:57:08.099 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-03-23T14:57:08.099 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T14:57:08.208 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.205+0000 7f180a137640 1 Processor -- start 2024-03-23T14:57:08.208 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.205+0000 7f180a137640 1 -- start start 2024-03-23T14:57:08.210 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.205+0000 7f180a137640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f180409def0 0x7f18040b45e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:08.210 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.205+0000 7f180a137640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f18040add50 0x7f180406bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:08.210 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.205+0000 7f180a137640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f180406d650 con 0x7f18040b4b20 2024-03-23T14:57:08.210 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.205+0000 7f180a137640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f18040af4a0 con 0x7f18040add50 2024-03-23T14:57:08.210 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.205+0000 7f180a137640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 -- 0x7f180409e2d0 con 0x7f180409def0 2024-03-23T14:57:08.210 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.205+0000 7f1802ffd640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f18040add50 0x7f180406bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:08.210 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.205+0000 7f1803fff640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f18040b4b20 0x7f180406c4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:42412/0 (socket says 172.21.15.164:42412) 2024-03-23T14:57:08.210 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.205+0000 7f1803fff640 1 -- 172.21.15.164:0/2695993442 learned_addr learned my addr 172.21.15.164:0/2695993442 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-03-23T14:57:08.210 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.205+0000 7f18037fe640 1 --2- 172.21.15.164:0/2695993442 >> v2:172.21.15.173:3300/0 conn(0x7f180409def0 0x7f18040b45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:08.211 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.205+0000 7f1802ffd640 1 -- 172.21.15.164:0/2695993442 >> v2:172.21.15.173:3300/0 conn(0x7f180409def0 msgr2=0x7f18040b45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:08.211 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.205+0000 7f1802ffd640 1 --2- 172.21.15.164:0/2695993442 >> v2:172.21.15.173:3300/0 conn(0x7f180409def0 0x7f18040b45e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:08.211 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.205+0000 7f1802ffd640 1 -- 172.21.15.164:0/2695993442 >> v1:172.21.15.164:6789/0 conn(0x7f18040b4b20 legacy=0x7f180406c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:08.211 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.205+0000 7f1802ffd640 1 -- 172.21.15.164:0/2695993442 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18040aeca0 con 0x7f18040add50 2024-03-23T14:57:08.211 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.205+0000 7f1802ffd640 1 --2- 172.21.15.164:0/2695993442 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f18040add50 0x7f180406bf70 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f180406da30 tx=0x7f17f801c360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dc8415d15582aeb server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:08.212 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.205+0000 7f18027fc640 1 -- 172.21.15.164:0/2695993442 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f80ab600 con 0x7f18040add50 2024-03-23T14:57:08.212 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.209+0000 7f18027fc640 1 -- 172.21.15.164:0/2695993442 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17f80a7050 con 0x7f18040add50 2024-03-23T14:57:08.212 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.209+0000 7f18027fc640 1 -- 172.21.15.164:0/2695993442 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f80ab980 con 0x7f18040add50 2024-03-23T14:57:08.212 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.209+0000 7f180a137640 1 -- 172.21.15.164:0/2695993442 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f18040add50 msgr2=0x7f180406bf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:08.212 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.209+0000 7f180a137640 1 --2- 172.21.15.164:0/2695993442 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f18040add50 0x7f180406bf70 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f180406da30 tx=0x7f17f801c360 comp rx=0 tx=0).stop 2024-03-23T14:57:08.213 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.209+0000 7f180a137640 1 -- 172.21.15.164:0/2695993442 shutdown_connections 2024-03-23T14:57:08.213 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.209+0000 7f180a137640 1 --2- 172.21.15.164:0/2695993442 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f18040add50 0x7f180406bf70 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:08.213 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.209+0000 7f180a137640 1 --2- 172.21.15.164:0/2695993442 >> v2:172.21.15.173:3300/0 conn(0x7f180409def0 0x7f18040b45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:08.213 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.209+0000 7f180a137640 1 -- 172.21.15.164:0/2695993442 >> 172.21.15.164:0/2695993442 conn(0x7f18040b8ca0 msgr2=0x7f18040b90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:08.213 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.209+0000 7f180a137640 1 -- 172.21.15.164:0/2695993442 shutdown_connections 2024-03-23T14:57:08.213 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.209+0000 7f180a137640 1 -- 172.21.15.164:0/2695993442 wait complete. 2024-03-23T14:57:08.215 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.209+0000 7f180a137640 1 Processor -- start 2024-03-23T14:57:08.215 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.209+0000 7f180a137640 1 -- start start 2024-03-23T14:57:08.215 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.209+0000 7f180a137640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f180409def0 0x7f18040b0c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:08.215 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.209+0000 7f180a137640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f18040add50 0x7f18040b7a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:08.215 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.209+0000 7f180a137640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f18040b4b20 0x7f1804143090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:08.216 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.209+0000 7f180a137640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f18040b26b0 con 0x7f18040add50 2024-03-23T14:57:08.216 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.209+0000 7f180a137640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f18040b2220 con 0x7f180409def0 2024-03-23T14:57:08.216 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.209+0000 7f180a137640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f18040b1f40 con 0x7f18040b4b20 2024-03-23T14:57:08.216 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.209+0000 7f1802ffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f18040add50 0x7f18040b7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:08.216 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.209+0000 7f1802ffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f18040add50 0x7f18040b7a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:59084/0 (socket says 172.21.15.164:59084) 2024-03-23T14:57:08.216 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.209+0000 7f1802ffd640 1 -- 172.21.15.164:0/94546831 learned_addr learned my addr 172.21.15.164:0/94546831 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:08.216 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.209+0000 7f1803fff640 1 --2- 172.21.15.164:0/94546831 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f18040b4b20 0x7f1804143090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:08.216 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.209+0000 7f18037fe640 1 --2- 172.21.15.164:0/94546831 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f180409def0 0x7f18040b0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:08.216 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.209+0000 7f1802ffd640 1 -- 172.21.15.164:0/94546831 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f18040b4b20 msgr2=0x7f1804143090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:08.216 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.209+0000 7f1802ffd640 1 --2- 172.21.15.164:0/94546831 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f18040b4b20 0x7f1804143090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:08.216 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.209+0000 7f1802ffd640 1 -- 172.21.15.164:0/94546831 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f180409def0 msgr2=0x7f18040b0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:08.216 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.209+0000 7f1802ffd640 1 --2- 172.21.15.164:0/94546831 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f180409def0 0x7f18040b0c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:08.216 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.209+0000 7f1802ffd640 1 -- 172.21.15.164:0/94546831 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17f8009420 con 0x7f18040add50 2024-03-23T14:57:08.216 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.209+0000 7f1802ffd640 1 --2- 172.21.15.164:0/94546831 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f18040add50 0x7f18040b7a50 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f17f80ae040 tx=0x7f17f8019100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:08.217 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.209+0000 7f17ebfff640 1 -- 172.21.15.164:0/94546831 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f80b4430 con 0x7f18040add50 2024-03-23T14:57:08.217 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.209+0000 7f17ebfff640 1 -- 172.21.15.164:0/94546831 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17f80b45b0 con 0x7f18040add50 2024-03-23T14:57:08.217 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.209+0000 7f180a137640 1 -- 172.21.15.164:0/94546831 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18040b8650 con 0x7f18040add50 2024-03-23T14:57:08.217 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.209+0000 7f17ebfff640 1 -- 172.21.15.164:0/94546831 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f80b4960 con 0x7f18040add50 2024-03-23T14:57:08.217 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.209+0000 7f180a137640 1 -- 172.21.15.164:0/94546831 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18040b28e0 con 0x7f18040add50 2024-03-23T14:57:08.217 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.209+0000 7f180a137640 1 -- 172.21.15.164:0/94546831 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f180414e680 con 0x7f18040add50 2024-03-23T14:57:08.218 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.213+0000 7f17ebfff640 1 -- 172.21.15.164:0/94546831 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7f17f80b2040 con 0x7f18040add50 2024-03-23T14:57:08.218 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.213+0000 7f17ebfff640 1 --2- 172.21.15.164:0/94546831 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f17f00672a0 0x7f17f0069760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:08.218 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.213+0000 7f18037fe640 1 --2- 172.21.15.164:0/94546831 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f17f00672a0 0x7f17f0069760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:08.218 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.213+0000 7f17ebfff640 1 -- 172.21.15.164:0/94546831 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f17f811eea0 con 0x7f18040add50 2024-03-23T14:57:08.218 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.213+0000 7f18037fe640 1 --2- 172.21.15.164:0/94546831 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f17f00672a0 0x7f17f0069760 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f17f40096f0 tx=0x7f17f4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:08.220 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.217+0000 7f17ebfff640 1 -- 172.21.15.164:0/94546831 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7f180414e680 con 0x7f18040add50 2024-03-23T14:57:08.367 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.361+0000 7f180a137640 1 -- 172.21.15.164:0/94546831 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f180414e680 con 0x7f18040add50 2024-03-23T14:57:08.368 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.365+0000 7f17ebfff640 1 -- 172.21.15.164:0/94546831 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16629 (secure 0 0 0) 0x7f180414e680 con 0x7f18040add50 2024-03-23T14:57:08.368 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-23T14:57:08.368 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":20,"fsid":"45c7911b-b69d-4e68-907b-fbfeb780aa3b","created":"2024-03-23T14:56:34.367633+0000","modified":"2024-03-23T14:57:00.611265+0000","last_up_change":"2024-03-23T14:56:46.297403+0000","last_in_change":"2024-03-23T14:56:36.629884+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T14:56:47.186157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-23T14:56:56.994057+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-23T14:56:57.953862+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"80abdc62-bd0d-4e60-a554-7ed6fd67cf54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":984716690},{"type":"v1","addr":"172.21.15.164:6801","nonce":984716690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":984716690},{"type":"v1","addr":"172.21.15.164:6803","nonce":984716690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":984716690},{"type":"v1","addr":"172.21.15.164:6807","nonce":984716690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":984716690},{"type":"v1","addr":"172.21.15.164:6805","nonce":984716690}]},"public_addr":"172.21.15.164:6801/984716690","cluster_addr":"172.21.15.164:6803/984716690","heartbeat_back_addr":"172.21.15.164:6807/984716690","heartbeat_front_addr":"172.21.15.164:6805/984716690","state":["exists","up"]},{"osd":1,"uuid":"f9001b06-be4d-4f42-a0e6-bcef2afa0d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":1349657239},{"type":"v1","addr":"172.21.15.164:6817","nonce":1349657239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":1349657239},{"type":"v1","addr":"172.21.15.164:6819","nonce":1349657239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":1349657239},{"type":"v1","addr":"172.21.15.164:6823","nonce":1349657239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":1349657239},{"type":"v1","addr":"172.21.15.164:6821","nonce":1349657239}]},"public_addr":"172.21.15.164:6817/1349657239","cluster_addr":"172.21.15.164:6819/1349657239","heartbeat_back_addr":"172.21.15.164:6823/1349657239","heartbeat_front_addr":"172.21.15.164:6821/1349657239","state":["exists","up"]},{"osd":2,"uuid":"fa0aeb9f-8b54-4831-a188-3e361c7980d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":887861252},{"type":"v1","addr":"172.21.15.164:6809","nonce":887861252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":887861252},{"type":"v1","addr":"172.21.15.164:6811","nonce":887861252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":887861252},{"type":"v1","addr":"172.21.15.164:6815","nonce":887861252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":887861252},{"type":"v1","addr":"172.21.15.164:6813","nonce":887861252}]},"public_addr":"172.21.15.164:6809/887861252","cluster_addr":"172.21.15.164:6811/887861252","heartbeat_back_addr":"172.21.15.164:6815/887861252","heartbeat_front_addr":"172.21.15.164:6813/887861252","state":["exists","up"]},{"osd":3,"uuid":"7e18c957-3091-46d5-8e17-8f3354c1726c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":812540271},{"type":"v1","addr":"172.21.15.164:6825","nonce":812540271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":812540271},{"type":"v1","addr":"172.21.15.164:6827","nonce":812540271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":812540271},{"type":"v1","addr":"172.21.15.164:6831","nonce":812540271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":812540271},{"type":"v1","addr":"172.21.15.164:6829","nonce":812540271}]},"public_addr":"172.21.15.164:6825/812540271","cluster_addr":"172.21.15.164:6827/812540271","heartbeat_back_addr":"172.21.15.164:6831/812540271","heartbeat_front_addr":"172.21.15.164:6829/812540271","state":["exists","up"]},{"osd":4,"uuid":"8e83f315-42bf-4471-9ce8-d81948da0f9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":2656857421},{"type":"v1","addr":"172.21.15.173:6825","nonce":2656857421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2656857421},{"type":"v1","addr":"172.21.15.173:6827","nonce":2656857421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2656857421},{"type":"v1","addr":"172.21.15.173:6831","nonce":2656857421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":2656857421},{"type":"v1","addr":"172.21.15.173:6829","nonce":2656857421}]},"public_addr":"172.21.15.173:6825/2656857421","cluster_addr":"172.21.15.173:6827/2656857421","heartbeat_back_addr":"172.21.15.173:6831/2656857421","heartbeat_front_addr":"172.21.15.173:6829/2656857421","state":["exists","up"]},{"osd":5,"uuid":"a230c872-71f9-40ab-8999-defd0162d4ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":3676483185},{"type":"v1","addr":"172.21.15.173:6801","nonce":3676483185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":3676483185},{"type":"v1","addr":"172.21.15.173:6803","nonce":3676483185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":3676483185},{"type":"v1","addr":"172.21.15.173:6807","nonce":3676483185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":3676483185},{"type":"v1","addr":"172.21.15.173:6805","nonce":3676483185}]},"public_addr":"172.21.15.173:6801/3676483185","cluster_addr":"172.21.15.173:6803/3676483185","heartbeat_back_addr":"172.21.15.173:6807/3676483185","heartbeat_front_addr":"172.21.15.173:6805/3676483185","state":["exists","up"]},{"osd":6,"uuid":"6193bfd7-8432-4f7f-a117-a3fcee21af99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":3631684087},{"type":"v1","addr":"172.21.15.173:6809","nonce":3631684087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":3631684087},{"type":"v1","addr":"172.21.15.173:6811","nonce":3631684087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":3631684087},{"type":"v1","addr":"172.21.15.173:6815","nonce":3631684087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":3631684087},{"type":"v1","addr":"172.21.15.173:6813","nonce":3631684087}]},"public_addr":"172.21.15.173:6809/3631684087","cluster_addr":"172.21.15.173:6811/3631684087","heartbeat_back_addr":"172.21.15.173:6815/3631684087","heartbeat_front_addr":"172.21.15.173:6813/3631684087","state":["exists","up"]},{"osd":7,"uuid":"b0f1b80a-afc4-4538-83b9-f081d2dcb8cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":3261114948},{"type":"v1","addr":"172.21.15.173:6817","nonce":3261114948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":3261114948},{"type":"v1","addr":"172.21.15.173:6819","nonce":3261114948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":3261114948},{"type":"v1","addr":"172.21.15.173:6823","nonce":3261114948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":3261114948},{"type":"v1","addr":"172.21.15.173:6821","nonce":3261114948}]},"public_addr":"172.21.15.173:6817/3261114948","cluster_addr":"172.21.15.173:6819/3261114948","heartbeat_back_addr":"172.21.15.173:6823/3261114948","heartbeat_front_addr":"172.21.15.173:6821/3261114948","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.188390+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.159932+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.191101+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.181776+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.224365+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.177515+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.153631+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.139972+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T14:57:08.370 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.365+0000 7f180a137640 1 -- 172.21.15.164:0/94546831 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f17f00672a0 msgr2=0x7f17f0069760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:08.370 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.365+0000 7f180a137640 1 --2- 172.21.15.164:0/94546831 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f17f00672a0 0x7f17f0069760 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f17f40096f0 tx=0x7f17f4008040 comp rx=0 tx=0).stop 2024-03-23T14:57:08.370 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.365+0000 7f180a137640 1 -- 172.21.15.164:0/94546831 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f18040add50 msgr2=0x7f18040b7a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:08.371 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.365+0000 7f180a137640 1 --2- 172.21.15.164:0/94546831 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f18040add50 0x7f18040b7a50 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f17f80ae040 tx=0x7f17f8019100 comp rx=0 tx=0).stop 2024-03-23T14:57:08.371 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.365+0000 7f180a137640 1 -- 172.21.15.164:0/94546831 shutdown_connections 2024-03-23T14:57:08.371 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.365+0000 7f180a137640 1 --2- 172.21.15.164:0/94546831 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f18040b4b20 0x7f1804143090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:08.371 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.365+0000 7f180a137640 1 --2- 172.21.15.164:0/94546831 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f18040add50 0x7f18040b7a50 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:08.371 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.365+0000 7f180a137640 1 --2- 172.21.15.164:0/94546831 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f17f00672a0 0x7f17f0069760 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:08.371 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.365+0000 7f180a137640 1 --2- 172.21.15.164:0/94546831 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f180409def0 0x7f18040b0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:08.371 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.365+0000 7f180a137640 1 -- 172.21.15.164:0/94546831 >> 172.21.15.164:0/94546831 conn(0x7f18040b8ca0 msgr2=0x7f18040b3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:08.371 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.365+0000 7f180a137640 1 -- 172.21.15.164:0/94546831 shutdown_connections 2024-03-23T14:57:08.371 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.365+0000 7f180a137640 1 -- 172.21.15.164:0/94546831 wait complete. 2024-03-23T14:57:08.386 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-03-23T14:57:08.387 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-03-23T14:57:08.387 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-03-23T14:57:08.388 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-03-23T14:57:08.388 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-03-23T14:57:08.389 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-03-23T14:57:08.389 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-03-23T14:57:08.389 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-03-23T14:57:08.565 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.561+0000 7f9ba13ef640 1 Processor -- start 2024-03-23T14:57:08.565 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.561+0000 7f9ba13ef640 1 -- start start 2024-03-23T14:57:08.566 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.561+0000 7f9ba13ef640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9b940c8600 0x7f9b940d61c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:08.566 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.561+0000 7f9ba13ef640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b94036e00 con 0x7f9b94006eb0 2024-03-23T14:57:08.566 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.561+0000 7f9ba13ef640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b94029fb0 con 0x7f9b94036540 2024-03-23T14:57:08.566 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.561+0000 7f9ba13ef640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f9b94035c70 con 0x7f9b940c8600 2024-03-23T14:57:08.566 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.561+0000 7f9b9bfff640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f9b94006eb0 0x7f9b94033920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:42416/0 (socket says 172.21.15.164:42416) 2024-03-23T14:57:08.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.561+0000 7f9b9bfff640 1 -- 172.21.15.164:0/420243123 learned_addr learned my addr 172.21.15.164:0/420243123 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-03-23T14:57:08.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.561+0000 7fb37ffff640 1 Processor -- start 2024-03-23T14:57:08.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.561+0000 7f9ba0bee640 1 --2- 172.21.15.164:0/420243123 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9b940c8600 0x7f9b940d61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:08.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.561+0000 7fb37ffff640 1 -- start start 2024-03-23T14:57:08.568 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.561+0000 7f9b9affd640 1 -- 172.21.15.164:0/420243123 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2127043408 0 0) 0x7f9b94036e00 con 0x7f9b94006eb0 2024-03-23T14:57:08.568 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.561+0000 7f9b9affd640 1 -- 172.21.15.164:0/420243123 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9b8c003530 con 0x7f9b94006eb0 2024-03-23T14:57:08.568 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.561+0000 7f9ba0bee640 1 -- 172.21.15.164:0/420243123 >> v1:172.21.15.173:6789/0 conn(0x7f9b94036540 legacy=0x7f9b94035560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:08.568 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.561+0000 7f9ba0bee640 1 -- 172.21.15.164:0/420243123 >> v1:172.21.15.164:6789/0 conn(0x7f9b94006eb0 legacy=0x7f9b94033920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:08.568 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.561+0000 7f9ba0bee640 1 -- 172.21.15.164:0/420243123 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b940369a0 con 0x7f9b940c8600 2024-03-23T14:57:08.569 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.565+0000 7fd281317640 1 Processor -- start 2024-03-23T14:57:08.569 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.565+0000 7fd281317640 1 -- start start 2024-03-23T14:57:08.569 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.565+0000 7f0a71df8640 1 Processor -- start 2024-03-23T14:57:08.569 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.565+0000 7f0a71df8640 1 -- start start 2024-03-23T14:57:08.569 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.565+0000 7f1b2251d640 1 Processor -- start 2024-03-23T14:57:08.570 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.565+0000 7fb37ffff640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fb378039ff0 0x7fb3780c2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:08.570 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.565+0000 7fb37ffff640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fb3780c2d60 0x7fb3780d33a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:08.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.565+0000 7fb37ffff640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb378037200 con 0x7fb378038e80 2024-03-23T14:57:08.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.565+0000 7fb37ffff640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fb3780347f0 con 0x7fb378039ff0 2024-03-23T14:57:08.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.565+0000 7fb37ffff640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 -- 0x7fb378037970 con 0x7fb3780c2d60 2024-03-23T14:57:08.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.565+0000 7fb37e7fc640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fb378039ff0 0x7fb3780c2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:08.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.565+0000 7fb37e7fc640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fb378039ff0 0x7fb3780c2820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:59094/0 (socket says 172.21.15.164:59094) 2024-03-23T14:57:08.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.565+0000 7fb37e7fc640 1 -- 172.21.15.164:0/1864950431 learned_addr learned my addr 172.21.15.164:0/1864950431 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:08.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.565+0000 7fb37effd640 1 --1- >> v1:172.21.15.173:6789/0 conn(0x7fb378038e80 0x7fb3780cc010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.173:6789/0 says I am v1:172.21.15.164:35116/0 (socket says 172.21.15.164:35116) 2024-03-23T14:57:08.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.565+0000 7fb37f7fe640 1 --2- 172.21.15.164:0/1864950431 >> v2:172.21.15.173:3300/0 conn(0x7fb3780c2d60 0x7fb3780d33a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:08.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.565+0000 7fb37dffb640 1 -- 172.21.15.164:0/1864950431 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1170382282 0 0) 0x7fb378037200 con 0x7fb378038e80 2024-03-23T14:57:08.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.565+0000 7fb37dffb640 1 -- 172.21.15.164:0/1864950431 --> v1:172.21.15.173:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb368003530 con 0x7fb378038e80 2024-03-23T14:57:08.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.565+0000 7f0a71df8640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f0a6c0679e0 0x7f0a6c0b4170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:08.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.565+0000 7f0a71df8640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f0a6c1486e0 0x7f0a6c14fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:08.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.565+0000 7f0a71df8640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a6c0b3ce0 con 0x7f0a6c09bb60 2024-03-23T14:57:08.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.565+0000 7f0a71df8640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f0a6c148b90 con 0x7f0a6c0679e0 2024-03-23T14:57:08.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.565+0000 7f0a71df8640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 -- 0x7f0a6c0b52d0 con 0x7f0a6c1486e0 2024-03-23T14:57:08.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.565+0000 7f1b2251d640 1 -- start start 2024-03-23T14:57:08.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.565+0000 7f0a70df6640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f0a6c0679e0 0x7f0a6c0b4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:08.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.565+0000 7f0a70df6640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f0a6c0679e0 0x7f0a6c0b4170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:59104/0 (socket says 172.21.15.164:59104) 2024-03-23T14:57:08.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.565+0000 7f0a70df6640 1 -- 172.21.15.164:0/3446453390 learned_addr learned my addr 172.21.15.164:0/3446453390 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:08.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.565+0000 7f0a715f7640 1 --2- 172.21.15.164:0/3446453390 >> v2:172.21.15.173:3300/0 conn(0x7f0a6c1486e0 0x7f0a6c14fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:08.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.565+0000 7fb37dffb640 1 -- 172.21.15.164:0/1864950431 <== mon.1 v1:172.21.15.173:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2989297504 0 0) 0x7fb368003530 con 0x7fb378038e80 2024-03-23T14:57:08.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.565+0000 7fb37dffb640 1 -- 172.21.15.164:0/1864950431 >> v2:172.21.15.173:3300/0 conn(0x7fb3780c2d60 msgr2=0x7fb3780d33a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:08.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.565+0000 7fb37dffb640 1 --2- 172.21.15.164:0/1864950431 >> v2:172.21.15.173:3300/0 conn(0x7fb3780c2d60 0x7fb3780d33a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:08.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.565+0000 7fb37dffb640 1 -- 172.21.15.164:0/1864950431 >> v2:172.21.15.164:3300/0 conn(0x7fb378039ff0 msgr2=0x7fb3780c2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:08.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.565+0000 7fb37dffb640 1 --2- 172.21.15.164:0/1864950431 >> v2:172.21.15.164:3300/0 conn(0x7fb378039ff0 0x7fb3780c2820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:08.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.565+0000 7fb37dffb640 1 -- 172.21.15.164:0/1864950431 --> v1:172.21.15.173:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb378039260 con 0x7fb378038e80 2024-03-23T14:57:08.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.565+0000 7fb37dffb640 1 -- 172.21.15.164:0/1864950431 <== mon.1 v1:172.21.15.173:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2526390548 0 0) 0x7fb370001680 con 0x7fb378038e80 2024-03-23T14:57:08.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.565+0000 7fb37dffb640 1 -- 172.21.15.164:0/1864950431 >> v1:172.21.15.173:6789/0 conn(0x7fb378038e80 legacy=0x7fb3780cc010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:08.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.565+0000 7fb37dffb640 1 --2- 172.21.15.164:0/1864950431 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb3680039b0 0x7fb368003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:08.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.565+0000 7fd281317640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd27c0b2870 0x7fd27c0afb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:08.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.565+0000 7fd281317640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd27c072b20 con 0x7fd27c06d590 2024-03-23T14:57:08.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.565+0000 7fd281317640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd27c143410 con 0x7fd27c146770 2024-03-23T14:57:08.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.565+0000 7fd281317640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fd27c13da20 con 0x7fd27c0b2870 2024-03-23T14:57:08.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.565+0000 7fb37dffb640 1 --2- 172.21.15.164:0/1864950431 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb368004310 0x7fb3680047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:08.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.565+0000 7fb37dffb640 1 --2- 172.21.15.164:0/1864950431 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb368004e40 0x7fb36800ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:08.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.565+0000 7fb37dffb640 1 -- 172.21.15.164:0/1864950431 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb378037970 con 0x7fb368004310 2024-03-23T14:57:08.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.565+0000 7fb37dffb640 1 -- 172.21.15.164:0/1864950431 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fb368005690 con 0x7fb3680039b0 2024-03-23T14:57:08.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.565+0000 7fb37dffb640 1 -- 172.21.15.164:0/1864950431 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fb368010560 con 0x7fb368004e40 2024-03-23T14:57:08.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.565+0000 7fb37e7fc640 1 --2- 172.21.15.164:0/1864950431 >> v2:172.21.15.164:3300/0 conn(0x7fb378039ff0 0x7fb3780c2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T14:57:08.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.565+0000 7f1b2251d640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f1b1c0b5720 0x7f1b1c0b7520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:08.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.565+0000 7f1b2251d640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1b1c141fa0 0x7f1b1c142430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:08.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.565+0000 7f1b2251d640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b1c0b6430 con 0x7f1b1c0727b0 2024-03-23T14:57:08.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.565+0000 7f1b2251d640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 -- 0x7f1b1c0b5b00 con 0x7f1b1c0b5720 2024-03-23T14:57:08.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.565+0000 7f1b2251d640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f1b1c0b6ba0 con 0x7f1b1c141fa0 2024-03-23T14:57:08.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.565+0000 7fd9b1100640 1 Processor -- start 2024-03-23T14:57:08.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.565+0000 7f0a70df6640 1 -- 172.21.15.164:0/3446453390 >> v2:172.21.15.173:3300/0 conn(0x7f0a6c1486e0 msgr2=0x7f0a6c14fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:08.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.565+0000 7f0a70df6640 1 --2- 172.21.15.164:0/3446453390 >> v2:172.21.15.173:3300/0 conn(0x7f0a6c1486e0 0x7f0a6c14fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:08.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.565+0000 7f0a70df6640 1 -- 172.21.15.164:0/3446453390 >> v1:172.21.15.173:6789/0 conn(0x7f0a6c09bb60 legacy=0x7f0a6c072a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:08.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.565+0000 7f0a6b7fe640 1 -- 172.21.15.164:0/3446453390 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 387303756 0 0) 0x7f0a6c0b3ce0 con 0x7f0a6c09bb60 2024-03-23T14:57:08.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.565+0000 7fd9b1100640 1 -- start start 2024-03-23T14:57:08.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.565+0000 7f1b20aba640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1b1c141fa0 0x7f1b1c142430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:08.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.565+0000 7f1b20aba640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1b1c141fa0 0x7f1b1c142430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.164:60472/0 (socket says 172.21.15.164:60472) 2024-03-23T14:57:08.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.565+0000 7f1b20aba640 1 -- 172.21.15.164:0/2592802474 learned_addr learned my addr 172.21.15.164:0/2592802474 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:08.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.565+0000 7f1b1bfff640 1 --2- 172.21.15.164:0/2592802474 >> v2:172.21.15.173:3300/0 conn(0x7f1b1c0b5720 0x7f1b1c0b7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:08.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.565+0000 7fb37f7fe640 1 --2- 172.21.15.164:0/1864950431 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb368004e40 0x7fb36800ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:08.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.565+0000 7fb37e7fc640 1 --2- 172.21.15.164:0/1864950431 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb368004310 0x7fb3680047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:08.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.565+0000 7fb37effd640 1 --2- 172.21.15.164:0/1864950431 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb3680039b0 0x7fb368003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:08.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.565+0000 7f9ba0bee640 1 --2- 172.21.15.164:0/420243123 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9b940c8600 0x7f9b940d61c0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f9b94036b90 tx=0x7f9b8801ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aba50683100ba81f server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:08.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.565+0000 7fb37e7fc640 1 -- 172.21.15.164:0/1864950431 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb368004e40 msgr2=0x7fb36800ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:08.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.565+0000 7f9b9affd640 1 -- 172.21.15.164:0/420243123 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b880a89b0 con 0x7f9b940c8600 2024-03-23T14:57:08.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.565+0000 7fd280b16640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd27c0b2870 0x7fd27c0afb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:08.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.565+0000 7fd280b16640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd27c0b2870 0x7fd27c0afb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.164:60468/0 (socket says 172.21.15.164:60468) 2024-03-23T14:57:08.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.565+0000 7fd280b16640 1 -- 172.21.15.164:0/670045958 learned_addr learned my addr 172.21.15.164:0/670045958 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:08.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.565+0000 7fd280b16640 1 -- 172.21.15.164:0/670045958 >> v1:172.21.15.173:6789/0 conn(0x7fd27c146770 legacy=0x7fd27c143bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:08.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.565+0000 7fd280b16640 1 -- 172.21.15.164:0/670045958 >> v1:172.21.15.164:6789/0 conn(0x7fd27c06d590 legacy=0x7fd27c0b8610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:08.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.565+0000 7fd280b16640 1 -- 172.21.15.164:0/670045958 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd27c1436b0 con 0x7fd27c0b2870 2024-03-23T14:57:08.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.565+0000 7f0a70df6640 1 -- 172.21.15.164:0/3446453390 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a6c067dc0 con 0x7f0a6c0679e0 2024-03-23T14:57:08.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.565+0000 7f1b20aba640 1 -- 172.21.15.164:0/2592802474 >> v2:172.21.15.173:3300/0 conn(0x7f1b1c0b5720 msgr2=0x7f1b1c0b7520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:08.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.565+0000 7f1b20aba640 1 --2- 172.21.15.164:0/2592802474 >> v2:172.21.15.173:3300/0 conn(0x7f1b1c0b5720 0x7f1b1c0b7520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:08.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.565+0000 7f1b20aba640 1 -- 172.21.15.164:0/2592802474 >> v1:172.21.15.173:6789/0 conn(0x7f1b1c0727b0 legacy=0x7f1b1c144fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:08.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.565+0000 7f1b20aba640 1 -- 172.21.15.164:0/2592802474 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b1c0b7b70 con 0x7f1b1c141fa0 2024-03-23T14:57:08.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.565+0000 7f9b9affd640 1 -- 172.21.15.164:0/420243123 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b880a8b10 con 0x7f9b940c8600 2024-03-23T14:57:08.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.565+0000 7f9b9affd640 1 -- 172.21.15.164:0/420243123 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b880a8e80 con 0x7f9b940c8600 2024-03-23T14:57:08.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.565+0000 7fd9b1100640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd9ac0b0440 0x7fd9ac0723e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:08.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.565+0000 7fd9b1100640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fd9ac072920 0x7fd9ac155e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:08.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.565+0000 7fd9b1100640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9ac0b2af0 con 0x7fd9ac0b2710 2024-03-23T14:57:08.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.565+0000 7fd9b1100640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 -- 0x7fd9ac0b4a00 con 0x7fd9ac072920 2024-03-23T14:57:08.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.565+0000 7fd9b1100640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fd9ac0b0900 con 0x7fd9ac0b0440 2024-03-23T14:57:08.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.569+0000 7fd9ab7fe640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd9ac0b0440 0x7fd9ac0723e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:08.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.569+0000 7fd9ab7fe640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd9ac0b0440 0x7fd9ac0723e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.164:60496/0 (socket says 172.21.15.164:60496) 2024-03-23T14:57:08.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.569+0000 7fd9ab7fe640 1 -- 172.21.15.164:0/1978710794 learned_addr learned my addr 172.21.15.164:0/1978710794 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:08.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.569+0000 7fd9b08ff640 1 --2- 172.21.15.164:0/1978710794 >> v2:172.21.15.173:3300/0 conn(0x7fd9ac072920 0x7fd9ac155e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:08.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.565+0000 7fb37e7fc640 1 --2- 172.21.15.164:0/1864950431 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb368004e40 0x7fb36800ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:08.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.565+0000 7fb37e7fc640 1 -- 172.21.15.164:0/1864950431 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb3680039b0 msgr2=0x7fb368003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:08.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.565+0000 7fb37e7fc640 1 --2- 172.21.15.164:0/1864950431 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb3680039b0 0x7fb368003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:08.583 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.565+0000 7fb37e7fc640 1 -- 172.21.15.164:0/1864950431 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb378039260 con 0x7fb368004310 2024-03-23T14:57:08.583 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.565+0000 7fb37f7fe640 1 --2- 172.21.15.164:0/1864950431 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb368004e40 0x7fb36800ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T14:57:08.583 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.565+0000 7fb37f7fe640 1 -- 172.21.15.164:0/1864950431 reap_dead start 2024-03-23T14:57:08.583 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.569+0000 7f9ba13ef640 1 -- 172.21.15.164:0/420243123 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9b940c8600 msgr2=0x7f9b940d61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:08.583 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.569+0000 7f9ba13ef640 1 --2- 172.21.15.164:0/420243123 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9b940c8600 0x7f9b940d61c0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f9b94036b90 tx=0x7f9b8801ecb0 comp rx=0 tx=0).stop 2024-03-23T14:57:08.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.569+0000 7fdb8ffff640 1 Processor -- start 2024-03-23T14:57:08.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.569+0000 7fdb8ffff640 1 -- start start 2024-03-23T14:57:08.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.569+0000 7f3cdbfaa640 1 Processor -- start 2024-03-23T14:57:08.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.569+0000 7f9ba13ef640 1 -- 172.21.15.164:0/420243123 shutdown_connections 2024-03-23T14:57:08.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.569+0000 7f9ba13ef640 1 --2- 172.21.15.164:0/420243123 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9b940c8600 0x7f9b940d61c0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:08.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.569+0000 7f9ba13ef640 1 -- 172.21.15.164:0/420243123 >> 172.21.15.164:0/420243123 conn(0x7f9b9403a860 msgr2=0x7f9b9403ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:08.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.569+0000 7f0a70df6640 1 --2- 172.21.15.164:0/3446453390 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0a6c0679e0 0x7f0a6c0b4170 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f0a6c0b0480 tx=0x7f0a6001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1ccb74bc6fbf59a server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:08.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.569+0000 7fdb8ffff640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdb900b8870 0x7fdb900b3080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:08.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.569+0000 7fdb8ffff640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb900ae3c0 con 0x7fdb9014fc70 2024-03-23T14:57:08.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.569+0000 7fdb8ffff640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb900ae6a0 con 0x7fdb900b1650 2024-03-23T14:57:08.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.569+0000 7fdb8ffff640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fdb900b8c50 con 0x7fdb900b8870 2024-03-23T14:57:08.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.569+0000 7f0a6b7fe640 1 -- 172.21.15.164:0/3446453390 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a600a9400 con 0x7f0a6c0679e0 2024-03-23T14:57:08.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.569+0000 7f3cdbfaa640 1 -- start start 2024-03-23T14:57:08.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.569+0000 7f0a6b7fe640 1 -- 172.21.15.164:0/3446453390 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a600a9560 con 0x7f0a6c0679e0 2024-03-23T14:57:08.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.569+0000 7f0a6b7fe640 1 -- 172.21.15.164:0/3446453390 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a600a98f0 con 0x7f0a6c0679e0 2024-03-23T14:57:08.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.569+0000 7fdb8effd640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdb900b8870 0x7fdb900b3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:08.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.569+0000 7fdb8effd640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdb900b8870 0x7fdb900b3080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.164:60500/0 (socket says 172.21.15.164:60500) 2024-03-23T14:57:08.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.569+0000 7fdb8effd640 1 -- 172.21.15.164:0/2153498436 learned_addr learned my addr 172.21.15.164:0/2153498436 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:08.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.569+0000 7fd9ab7fe640 1 -- 172.21.15.164:0/1978710794 >> v2:172.21.15.173:3300/0 conn(0x7fd9ac072920 msgr2=0x7fd9ac155e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:08.588 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.569+0000 7fd9ab7fe640 1 --2- 172.21.15.164:0/1978710794 >> v2:172.21.15.173:3300/0 conn(0x7fd9ac072920 0x7fd9ac155e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:08.588 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.569+0000 7fd9ab7fe640 1 -- 172.21.15.164:0/1978710794 >> v1:172.21.15.173:6789/0 conn(0x7fd9ac0b2710 legacy=0x7fd9ac0b8680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:08.588 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.569+0000 7fd9ab7fe640 1 -- 172.21.15.164:0/1978710794 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9ac0b0be0 con 0x7fd9ac0b0440 2024-03-23T14:57:08.588 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.569+0000 7f3cdbfaa640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3cd40b6630 0x7f3cd4070ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:08.588 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.569+0000 7f3cdbfaa640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f3cd4072a30 0x7f3cd41411d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:08.588 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.569+0000 7f3cdbfaa640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3cd40b0970 con 0x7f3cd4141710 2024-03-23T14:57:08.588 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.569+0000 7f3cdbfaa640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 -- 0x7f3cd40b6a10 con 0x7f3cd4072a30 2024-03-23T14:57:08.588 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.569+0000 7f3cdbfaa640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f3cd40b5980 con 0x7f3cd40b6630 2024-03-23T14:57:08.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.569+0000 7f9ba13ef640 1 -- 172.21.15.164:0/420243123 shutdown_connections 2024-03-23T14:57:08.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.569+0000 7fd280b16640 1 --2- 172.21.15.164:0/670045958 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd27c0b2870 0x7fd27c0afb80 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd26c009e80 tx=0x7fd26c027a10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7960cc3e9e810bf6 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:08.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.569+0000 7fdb8dffb640 1 -- 172.21.15.164:0/2153498436 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 35556023 0 0) 0x7fdb900ae3c0 con 0x7fdb9014fc70 2024-03-23T14:57:08.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.569+0000 7fdb8dffb640 1 -- 172.21.15.164:0/2153498436 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdb78003530 con 0x7fdb9014fc70 2024-03-23T14:57:08.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.569+0000 7f3cd9545640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f3cd4072a30 0x7f3cd41411d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:08.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.569+0000 7f3cd9545640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f3cd4072a30 0x7f3cd41411d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.164:39456/0 (socket says 172.21.15.164:39456) 2024-03-23T14:57:08.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.569+0000 7f3cd9545640 1 -- 172.21.15.164:0/3811463969 learned_addr learned my addr 172.21.15.164:0/3811463969 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:08.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.569+0000 7f3cd9d46640 1 --2- 172.21.15.164:0/3811463969 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3cd40b6630 0x7f3cd4070ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:08.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.569+0000 7fb37e7fc640 1 --2- 172.21.15.164:0/1864950431 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb368004310 0x7fb3680047d0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fb380050a70 tx=0x7fb380098610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a972832714399946 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:08.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.569+0000 7fd27affd640 1 -- 172.21.15.164:0/670045958 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd26c0b1620 con 0x7fd27c0b2870 2024-03-23T14:57:08.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.569+0000 7fb37dffb640 1 -- 172.21.15.164:0/1864950431 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb38012a070 con 0x7fb368004310 2024-03-23T14:57:08.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.569+0000 7f0a71df8640 1 -- 172.21.15.164:0/3446453390 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0a6c0679e0 msgr2=0x7f0a6c0b4170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:08.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.569+0000 7f0a71df8640 1 --2- 172.21.15.164:0/3446453390 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0a6c0679e0 0x7f0a6c0b4170 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f0a6c0b0480 tx=0x7f0a6001c520 comp rx=0 tx=0).stop 2024-03-23T14:57:08.592 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.569+0000 7fdb8dffb640 1 -- 172.21.15.164:0/2153498436 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 842196014 0 0) 0x7fdb78003530 con 0x7fdb9014fc70 2024-03-23T14:57:08.592 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.569+0000 7fdb8dffb640 1 -- 172.21.15.164:0/2153498436 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdb900b8870 msgr2=0x7fdb900b3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:08.592 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.569+0000 7fdb8dffb640 1 --2- 172.21.15.164:0/2153498436 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdb900b8870 0x7fdb900b3080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:08.592 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.569+0000 7fdb8dffb640 1 -- 172.21.15.164:0/2153498436 >> v1:172.21.15.173:6789/0 conn(0x7fdb900b1650 legacy=0x7fdb900b1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:08.592 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.569+0000 7fdb8dffb640 1 -- 172.21.15.164:0/2153498436 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb900aeb30 con 0x7fdb9014fc70 2024-03-23T14:57:08.592 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.569+0000 7f1b20aba640 1 --2- 172.21.15.164:0/2592802474 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1b1c141fa0 0x7f1b1c142430 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f1b100098a0 tx=0x7f1b1001c4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3138a90bb8e2759c server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:08.592 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.569+0000 7fdb8dffb640 1 -- 172.21.15.164:0/2153498436 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2526390548 0 0) 0x7fdb88001a00 con 0x7fdb9014fc70 2024-03-23T14:57:08.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.569+0000 7f0a71df8640 1 -- 172.21.15.164:0/3446453390 shutdown_connections 2024-03-23T14:57:08.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.569+0000 7f0a71df8640 1 --2- 172.21.15.164:0/3446453390 >> v2:172.21.15.173:3300/0 conn(0x7f0a6c1486e0 0x7f0a6c14fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:08.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.569+0000 7f0a71df8640 1 --2- 172.21.15.164:0/3446453390 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0a6c0679e0 0x7f0a6c0b4170 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:08.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.569+0000 7f0a71df8640 1 -- 172.21.15.164:0/3446453390 >> 172.21.15.164:0/3446453390 conn(0x7f0a6c06b960 msgr2=0x7f0a6c0b90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:08.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.569+0000 7fb37dffb640 1 -- 172.21.15.164:0/1864950431 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3801264f0 con 0x7fb368004310 2024-03-23T14:57:08.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.569+0000 7fb37dffb640 1 -- 172.21.15.164:0/1864950431 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb380126740 con 0x7fb368004310 2024-03-23T14:57:08.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.569+0000 7fd27affd640 1 -- 172.21.15.164:0/670045958 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd26c0b1780 con 0x7fd27c0b2870 2024-03-23T14:57:08.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.569+0000 7fd27affd640 1 -- 172.21.15.164:0/670045958 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd26c0b1b10 con 0x7fd27c0b2870 2024-03-23T14:57:08.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.569+0000 7fdb8dffb640 1 -- 172.21.15.164:0/2153498436 >> v1:172.21.15.164:6789/0 conn(0x7fdb9014fc70 legacy=0x7fdb90152060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:08.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.569+0000 7fdb8dffb640 1 --2- 172.21.15.164:0/2153498436 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdb780039b0 0x7fdb78003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:08.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.569+0000 7fdb8dffb640 1 --2- 172.21.15.164:0/2153498436 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdb900b8870 0x7fdb780047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:08.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.569+0000 7fdb8dffb640 1 --2- 172.21.15.164:0/2153498436 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdb78004d10 0x7fdb78005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:08.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.569+0000 7fdb8dffb640 1 -- 172.21.15.164:0/2153498436 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fdb900b8c50 con 0x7fdb78004d10 2024-03-23T14:57:08.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.569+0000 7fdb8dffb640 1 -- 172.21.15.164:0/2153498436 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fdb7800bb20 con 0x7fdb780039b0 2024-03-23T14:57:08.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.569+0000 7fdb8dffb640 1 -- 172.21.15.164:0/2153498436 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fdb7800bce0 con 0x7fdb900b8870 2024-03-23T14:57:08.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.569+0000 7fdb8f7fe640 1 --2- 172.21.15.164:0/2153498436 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdb78004d10 0x7fdb78005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:08.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.569+0000 7fdb8e7fc640 1 --2- 172.21.15.164:0/2153498436 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdb900b8870 0x7fdb780047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:08.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.573+0000 7fdb8effd640 1 --2- 172.21.15.164:0/2153498436 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdb780039b0 0x7fdb78003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:08.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.569+0000 7f3cd9545640 1 -- 172.21.15.164:0/3811463969 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3cd40b6630 msgr2=0x7f3cd4070ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:08.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.569+0000 7f3cd9545640 1 --2- 172.21.15.164:0/3811463969 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3cd40b6630 0x7f3cd4070ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:08.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.569+0000 7f3cd9545640 1 -- 172.21.15.164:0/3811463969 >> v1:172.21.15.173:6789/0 conn(0x7f3cd4141710 legacy=0x7f3cd40b43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:08.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.569+0000 7f3cd9545640 1 -- 172.21.15.164:0/3811463969 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cd4070580 con 0x7f3cd4072a30 2024-03-23T14:57:08.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.569+0000 7f9ba13ef640 1 -- 172.21.15.164:0/420243123 wait complete. 2024-03-23T14:57:08.596 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.573+0000 7fdb8f7fe640 1 -- 172.21.15.164:0/2153498436 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdb900b8870 msgr2=0x7fdb780047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:08.596 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.573+0000 7fdb8f7fe640 1 --2- 172.21.15.164:0/2153498436 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdb900b8870 0x7fdb780047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:08.596 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.573+0000 7fdb8f7fe640 1 -- 172.21.15.164:0/2153498436 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdb780039b0 msgr2=0x7fdb78003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:08.596 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.573+0000 7fdb8f7fe640 1 --2- 172.21.15.164:0/2153498436 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdb780039b0 0x7fdb78003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:08.596 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.573+0000 7fdb8f7fe640 1 -- 172.21.15.164:0/2153498436 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb78003530 con 0x7fdb78004d10 2024-03-23T14:57:08.596 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.573+0000 7fdb8e7fc640 1 --2- 172.21.15.164:0/2153498436 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdb900b8870 0x7fdb780047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T14:57:08.596 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.573+0000 7fb37ffff640 1 -- 172.21.15.164:0/1864950431 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb368004310 msgr2=0x7fb3680047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:08.596 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.573+0000 7fb37ffff640 1 --2- 172.21.15.164:0/1864950431 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb368004310 0x7fb3680047d0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fb380050a70 tx=0x7fb380098610 comp rx=0 tx=0).stop 2024-03-23T14:57:08.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.573+0000 7fd9ab7fe640 1 --2- 172.21.15.164:0/1978710794 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd9ac0b0440 0x7fd9ac0723e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fd99c009e50 tx=0x7fd99c01c980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c35ed8dd02671fc6 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:08.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.573+0000 7f0a71df8640 1 -- 172.21.15.164:0/3446453390 shutdown_connections 2024-03-23T14:57:08.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.573+0000 7f9ba13ef640 1 Processor -- start 2024-03-23T14:57:08.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.573+0000 7fb37ffff640 1 -- 172.21.15.164:0/1864950431 shutdown_connections 2024-03-23T14:57:08.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.573+0000 7fb37ffff640 1 --2- 172.21.15.164:0/1864950431 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb368004310 0x7fb3680047d0 secure :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fb380050a70 tx=0x7fb380098610 comp rx=0 tx=0).stop 2024-03-23T14:57:08.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.573+0000 7fb37ffff640 1 -- 172.21.15.164:0/1864950431 >> 172.21.15.164:0/1864950431 conn(0x7fb37803a860 msgr2=0x7fb37803ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:08.599 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.573+0000 7fd9aaffd640 1 -- 172.21.15.164:0/1978710794 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd99c0a5690 con 0x7fd9ac0b0440 2024-03-23T14:57:08.599 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.573+0000 7fb37ffff640 1 -- 172.21.15.164:0/1864950431 shutdown_connections 2024-03-23T14:57:08.599 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.573+0000 7fd9aaffd640 1 -- 172.21.15.164:0/1978710794 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd99c0a57f0 con 0x7fd9ac0b0440 2024-03-23T14:57:08.599 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.573+0000 7fd9aaffd640 1 -- 172.21.15.164:0/1978710794 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd99c0a5b80 con 0x7fd9ac0b0440 2024-03-23T14:57:08.599 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.573+0000 7fd281317640 1 -- 172.21.15.164:0/670045958 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd27c0b2870 msgr2=0x7fd27c0afb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:08.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.573+0000 7fd281317640 1 --2- 172.21.15.164:0/670045958 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd27c0b2870 0x7fd27c0afb80 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd26c009e80 tx=0x7fd26c027a10 comp rx=0 tx=0).stop 2024-03-23T14:57:08.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.573+0000 7f1b1affd640 1 -- 172.21.15.164:0/2592802474 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b100aa430 con 0x7f1b1c141fa0 2024-03-23T14:57:08.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.573+0000 7f1b1affd640 1 -- 172.21.15.164:0/2592802474 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b100aa590 con 0x7f1b1c141fa0 2024-03-23T14:57:08.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.573+0000 7f1b1affd640 1 -- 172.21.15.164:0/2592802474 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b100aa720 con 0x7f1b1c141fa0 2024-03-23T14:57:08.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.573+0000 7f1b2251d640 1 -- 172.21.15.164:0/2592802474 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1b1c141fa0 msgr2=0x7f1b1c142430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:08.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.573+0000 7f1b2251d640 1 --2- 172.21.15.164:0/2592802474 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1b1c141fa0 0x7f1b1c142430 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f1b100098a0 tx=0x7f1b1001c4b0 comp rx=0 tx=0).stop 2024-03-23T14:57:08.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.573+0000 7f9ba13ef640 1 -- start start 2024-03-23T14:57:08.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.573+0000 7f9ba13ef640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9b94006eb0 0x7f9b940b7150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:08.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.573+0000 7f9ba13ef640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9b94036540 0x7f9b940b30a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:08.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.573+0000 7f9ba13ef640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9b940c8600 0x7f9b940c5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:08.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.573+0000 7f9ba13ef640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9b941659b0 con 0x7f9b94036540 2024-03-23T14:57:08.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.573+0000 7f9ba13ef640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f9b941650c0 con 0x7f9b940c8600 2024-03-23T14:57:08.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.573+0000 7f9ba13ef640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f9b94165360 con 0x7f9b94006eb0 2024-03-23T14:57:08.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.573+0000 7f1b2251d640 1 -- 172.21.15.164:0/2592802474 shutdown_connections 2024-03-23T14:57:08.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.573+0000 7f1b2251d640 1 --2- 172.21.15.164:0/2592802474 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1b1c141fa0 0x7f1b1c142430 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:08.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.573+0000 7f1b2251d640 1 --2- 172.21.15.164:0/2592802474 >> v2:172.21.15.173:3300/0 conn(0x7f1b1c0b5720 0x7f1b1c0b7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:08.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.573+0000 7f1b2251d640 1 -- 172.21.15.164:0/2592802474 >> 172.21.15.164:0/2592802474 conn(0x7f1b1c02bfc0 msgr2=0x7f1b1c02c3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:08.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.573+0000 7f9b9bfff640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9b94006eb0 0x7f9b940b7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:08.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.573+0000 7f9b9bfff640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9b94006eb0 0x7f9b940b7150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.164:60520/0 (socket says 172.21.15.164:60520) 2024-03-23T14:57:08.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.573+0000 7f9b9bfff640 1 -- 172.21.15.164:0/479814664 learned_addr learned my addr 172.21.15.164:0/479814664 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:08.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.573+0000 7f9ba0bee640 1 --2- 172.21.15.164:0/479814664 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9b940c8600 0x7f9b940c5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:08.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.573+0000 7fd9b1100640 1 -- 172.21.15.164:0/1978710794 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd9ac0b0440 msgr2=0x7fd9ac0723e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:08.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.573+0000 7fd9b1100640 1 --2- 172.21.15.164:0/1978710794 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd9ac0b0440 0x7fd9ac0723e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fd99c009e50 tx=0x7fd99c01c980 comp rx=0 tx=0).stop 2024-03-23T14:57:08.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.573+0000 7f9b9b7fe640 1 --2- 172.21.15.164:0/479814664 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9b94036540 0x7f9b940b30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:08.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.573+0000 7fd9b1100640 1 -- 172.21.15.164:0/1978710794 shutdown_connections 2024-03-23T14:57:08.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.573+0000 7fd9b1100640 1 --2- 172.21.15.164:0/1978710794 >> v2:172.21.15.173:3300/0 conn(0x7fd9ac072920 0x7fd9ac155e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:08.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.573+0000 7fd9b1100640 1 --2- 172.21.15.164:0/1978710794 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd9ac0b0440 0x7fd9ac0723e0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:08.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.573+0000 7fd9b1100640 1 -- 172.21.15.164:0/1978710794 >> 172.21.15.164:0/1978710794 conn(0x7fd9ac06bc80 msgr2=0x7fd9ac06c090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:08.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.573+0000 7f1b2251d640 1 -- 172.21.15.164:0/2592802474 shutdown_connections 2024-03-23T14:57:08.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.573+0000 7f1b2251d640 1 -- 172.21.15.164:0/2592802474 wait complete. 2024-03-23T14:57:08.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.573+0000 7f9b9bfff640 1 -- 172.21.15.164:0/479814664 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9b940c8600 msgr2=0x7f9b940c5830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:08.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.573+0000 7f9b9bfff640 1 --2- 172.21.15.164:0/479814664 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9b940c8600 0x7f9b940c5830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:08.605 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.573+0000 7f9b9bfff640 1 -- 172.21.15.164:0/479814664 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9b94036540 msgr2=0x7f9b940b30a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:08.605 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.573+0000 7f9b9bfff640 1 --2- 172.21.15.164:0/479814664 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9b94036540 0x7f9b940b30a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:08.605 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.573+0000 7f9b9bfff640 1 -- 172.21.15.164:0/479814664 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b88009420 con 0x7f9b94006eb0 2024-03-23T14:57:08.605 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.573+0000 7fd9b1100640 1 -- 172.21.15.164:0/1978710794 shutdown_connections 2024-03-23T14:57:08.605 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.573+0000 7f0a71df8640 1 -- 172.21.15.164:0/3446453390 wait complete. 2024-03-23T14:57:08.605 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.573+0000 7f0a71df8640 1 Processor -- start 2024-03-23T14:57:08.606 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.573+0000 7fb37ffff640 1 -- 172.21.15.164:0/1864950431 wait complete. 2024-03-23T14:57:08.606 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.573+0000 7fb37ffff640 1 Processor -- start 2024-03-23T14:57:08.606 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.573+0000 7f3cd9545640 1 --2- 172.21.15.164:0/3811463969 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3cd4072a30 0x7f3cd41411d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3ccc009e50 tx=0x7f3ccc01c930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=28177b9c1b0b3b84 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:08.606 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.573+0000 7f9b9bfff640 1 --2- 172.21.15.164:0/479814664 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9b94006eb0 0x7f9b940b7150 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f9b9000c4d0 tx=0x7f9b9000c990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:08.607 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.573+0000 7f0a71df8640 1 -- start start 2024-03-23T14:57:08.607 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.573+0000 7fd281317640 1 -- 172.21.15.164:0/670045958 shutdown_connections 2024-03-23T14:57:08.607 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.573+0000 7fd281317640 1 --2- 172.21.15.164:0/670045958 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd27c0b2870 0x7fd27c0afb80 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:08.607 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.573+0000 7fd281317640 1 -- 172.21.15.164:0/670045958 >> 172.21.15.164:0/670045958 conn(0x7fd27c06bc80 msgr2=0x7fd27c06c090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:08.607 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.573+0000 7f3cd8d44640 1 -- 172.21.15.164:0/3811463969 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ccc0a55e0 con 0x7f3cd4072a30 2024-03-23T14:57:08.608 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.573+0000 7fd9b1100640 1 -- 172.21.15.164:0/1978710794 wait complete. 2024-03-23T14:57:08.608 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.573+0000 7f0a71df8640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0a6c0679e0 0x7f0a6c1ef7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:08.608 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.573+0000 7f0a71df8640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0a6c09bb60 0x7f0a6c0b1a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:08.608 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.573+0000 7f0a71df8640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0a6c1486e0 0x7f0a6c1451d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:08.608 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.573+0000 7f0a71df8640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0a6c0b0a70 con 0x7f0a6c1486e0 2024-03-23T14:57:08.608 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.573+0000 7f0a71df8640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f0a6c0b0cc0 con 0x7f0a6c0679e0 2024-03-23T14:57:08.608 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.573+0000 7f0a71df8640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f0a6c145f30 con 0x7f0a6c09bb60 2024-03-23T14:57:08.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.573+0000 7f1b2251d640 1 Processor -- start 2024-03-23T14:57:08.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.573+0000 7f9b98ff9640 1 -- 172.21.15.164:0/479814664 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b90005cf0 con 0x7f9b94006eb0 2024-03-23T14:57:08.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.573+0000 7f9b98ff9640 1 -- 172.21.15.164:0/479814664 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b90005e50 con 0x7f9b94006eb0 2024-03-23T14:57:08.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.573+0000 7f9b98ff9640 1 -- 172.21.15.164:0/479814664 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b90007270 con 0x7f9b94006eb0 2024-03-23T14:57:08.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.573+0000 7fdb8f7fe640 1 --2- 172.21.15.164:0/2153498436 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdb78004d10 0x7fdb78005190 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fdb8800c490 tx=0x7fdb8801ce40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9bb88bc073075b server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:08.610 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.573+0000 7f0a715f7640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0a6c1486e0 0x7f0a6c1451d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:08.610 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.573+0000 7f0a715f7640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0a6c1486e0 0x7f0a6c1451d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:59140/0 (socket says 172.21.15.164:59140) 2024-03-23T14:57:08.610 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.573+0000 7f0a715f7640 1 -- 172.21.15.164:0/1915291815 learned_addr learned my addr 172.21.15.164:0/1915291815 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:08.610 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.573+0000 7f0a6bfff640 1 --2- 172.21.15.164:0/1915291815 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0a6c09bb60 0x7f0a6c0b1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:08.610 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.573+0000 7f3cd8d44640 1 -- 172.21.15.164:0/3811463969 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ccc0a5740 con 0x7f3cd4072a30 2024-03-23T14:57:08.610 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.573+0000 7f3cd8d44640 1 -- 172.21.15.164:0/3811463969 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ccc0a5ad0 con 0x7f3cd4072a30 2024-03-23T14:57:08.610 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.577+0000 7f3cdbfaa640 1 -- 172.21.15.164:0/3811463969 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3cd4072a30 msgr2=0x7f3cd41411d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:08.610 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.577+0000 7f3cdbfaa640 1 --2- 172.21.15.164:0/3811463969 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3cd4072a30 0x7f3cd41411d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3ccc009e50 tx=0x7f3ccc01c930 comp rx=0 tx=0).stop 2024-03-23T14:57:08.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.577+0000 7f0a70df6640 1 --2- 172.21.15.164:0/1915291815 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0a6c0679e0 0x7f0a6c1ef7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:08.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.577+0000 7fd9b1100640 1 Processor -- start 2024-03-23T14:57:08.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.573+0000 7fb37ffff640 1 -- start start 2024-03-23T14:57:08.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.573+0000 7fb37ffff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb3780c84b0 0x7fb378155fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:08.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.573+0000 7fb37ffff640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb3780be770 0x7fb3781687c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:08.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.573+0000 7fb37ffff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb378168d00 0x7fb3780c4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:08.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.573+0000 7fb37ffff640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb3780c3850 con 0x7fb378168d00 2024-03-23T14:57:08.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.573+0000 7fb37ffff640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fb3780c4570 con 0x7fb3780c84b0 2024-03-23T14:57:08.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.573+0000 7fb37ffff640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fb3780c6040 con 0x7fb3780be770 2024-03-23T14:57:08.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.577+0000 7fb37effd640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb3780c84b0 0x7fb378155fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:08.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.577+0000 7fb37effd640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb3780c84b0 0x7fb378155fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.164:39496/0 (socket says 172.21.15.164:39496) 2024-03-23T14:57:08.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.577+0000 7fb37effd640 1 -- 172.21.15.164:0/57348807 learned_addr learned my addr 172.21.15.164:0/57348807 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:08.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.577+0000 7fb37e7fc640 1 --2- 172.21.15.164:0/57348807 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb3780be770 0x7fb3781687c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:08.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.577+0000 7f3cdbfaa640 1 -- 172.21.15.164:0/3811463969 shutdown_connections 2024-03-23T14:57:08.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.577+0000 7f3cdbfaa640 1 --2- 172.21.15.164:0/3811463969 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3cd4072a30 0x7f3cd41411d0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:08.613 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.577+0000 7f3cdbfaa640 1 --2- 172.21.15.164:0/3811463969 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3cd40b6630 0x7f3cd4070ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:08.613 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.577+0000 7f3cdbfaa640 1 -- 172.21.15.164:0/3811463969 >> 172.21.15.164:0/3811463969 conn(0x7f3cd406d4d0 msgr2=0x7f3cd406b1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:08.613 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.577+0000 7f9ba13ef640 1 -- 172.21.15.164:0/479814664 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b94037410 con 0x7f9b94006eb0 2024-03-23T14:57:08.613 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.577+0000 7f9ba13ef640 1 -- 172.21.15.164:0/479814664 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b94165e40 con 0x7f9b94006eb0 2024-03-23T14:57:08.613 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.577+0000 7f0a715f7640 1 -- 172.21.15.164:0/1915291815 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0a6c09bb60 msgr2=0x7f0a6c0b1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:08.614 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.577+0000 7f0a715f7640 1 --2- 172.21.15.164:0/1915291815 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0a6c09bb60 0x7f0a6c0b1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:08.614 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.577+0000 7f0a715f7640 1 -- 172.21.15.164:0/1915291815 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0a6c0679e0 msgr2=0x7f0a6c1ef7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:08.614 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.577+0000 7f0a715f7640 1 --2- 172.21.15.164:0/1915291815 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0a6c0679e0 0x7f0a6c1ef7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:08.614 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.577+0000 7f0a715f7640 1 -- 172.21.15.164:0/1915291815 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a600094d0 con 0x7f0a6c1486e0 2024-03-23T14:57:08.614 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.573+0000 7f1b2251d640 1 -- start start 2024-03-23T14:57:08.614 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.577+0000 7f1b2251d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b1c0727b0 0x7f1b1c14eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:08.614 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.577+0000 7f1b2251d640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1b1c0b5720 0x7f1b1c13d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:08.614 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.577+0000 7f1b2251d640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1b1c141fa0 0x7f1b1c1e3f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:08.614 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.577+0000 7f1b2251d640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1b1c13f810 con 0x7f1b1c0727b0 2024-03-23T14:57:08.615 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.577+0000 7f1b2251d640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f1b1c1e4450 con 0x7f1b1c0b5720 2024-03-23T14:57:08.615 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.577+0000 7f1b2251d640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f1b1c140270 con 0x7f1b1c141fa0 2024-03-23T14:57:08.615 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.577+0000 7fb37f7fe640 1 --2- 172.21.15.164:0/57348807 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb378168d00 0x7fb3780c4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:08.615 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.577+0000 7fd9b1100640 1 -- start start 2024-03-23T14:57:08.615 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.577+0000 7fd9b1100640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd9ac072920 0x7fd9ac145020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:08.616 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.577+0000 7fb37effd640 1 -- 172.21.15.164:0/57348807 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb3780be770 msgr2=0x7fb3781687c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:08.616 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.577+0000 7fb37effd640 1 --2- 172.21.15.164:0/57348807 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb3780be770 0x7fb3781687c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:08.616 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.577+0000 7fb37effd640 1 -- 172.21.15.164:0/57348807 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb378168d00 msgr2=0x7fb3780c4f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:08.616 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.577+0000 7fb37effd640 1 --2- 172.21.15.164:0/57348807 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb378168d00 0x7fb3780c4f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:08.616 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.577+0000 7fb37effd640 1 -- 172.21.15.164:0/57348807 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3780c8890 con 0x7fb3780c84b0 2024-03-23T14:57:08.616 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.577+0000 7fd9b1100640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd9ac0b0440 0x7fd9ac1469a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:08.616 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.577+0000 7fdb8dffb640 1 -- 172.21.15.164:0/2153498436 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb880a5d40 con 0x7fdb78004d10 2024-03-23T14:57:08.617 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.577+0000 7f0a715f7640 1 --2- 172.21.15.164:0/1915291815 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0a6c1486e0 0x7f0a6c1451d0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f0a64007e60 tx=0x7f0a6400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:08.617 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.577+0000 7fd9b1100640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd9ac0b2710 0x7fd9ac146ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:08.617 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.577+0000 7fdb8dffb640 1 -- 172.21.15.164:0/2153498436 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb880ad440 con 0x7fdb78004d10 2024-03-23T14:57:08.617 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.577+0000 7fd9b1100640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd9ac148f60 con 0x7fd9ac0b0440 2024-03-23T14:57:08.618 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.577+0000 7fdb8dffb640 1 -- 172.21.15.164:0/2153498436 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb880a5d40 con 0x7fdb78004d10 2024-03-23T14:57:08.618 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.577+0000 7fd9b1100640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd9ac1482a0 con 0x7fd9ac0b2710 2024-03-23T14:57:08.618 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.577+0000 7fd9b1100640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fd9ac071f60 con 0x7fd9ac072920 2024-03-23T14:57:08.618 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.577+0000 7fd9abfff640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd9ac072920 0x7fd9ac145020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:08.618 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.577+0000 7fd9abfff640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd9ac072920 0x7fd9ac145020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.164:60548/0 (socket says 172.21.15.164:60548) 2024-03-23T14:57:08.618 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.577+0000 7fd9abfff640 1 -- 172.21.15.164:0/1641416357 learned_addr learned my addr 172.21.15.164:0/1641416357 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:08.619 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.577+0000 7f9ba13ef640 1 -- 172.21.15.164:0/479814664 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9b94039510 con 0x7f9b94006eb0 2024-03-23T14:57:08.619 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.577+0000 7fb37effd640 1 --2- 172.21.15.164:0/57348807 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb3780c84b0 0x7fb378155fb0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb370004a50 tx=0x7fb370004f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:08.619 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.577+0000 7f1b1bfff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b1c0727b0 0x7f1b1c14eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:08.619 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.577+0000 7f1b1b7fe640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1b1c0b5720 0x7f1b1c13d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:08.619 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.577+0000 7f1b1bfff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b1c0727b0 0x7f1b1c14eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:59162/0 (socket says 172.21.15.164:59162) 2024-03-23T14:57:08.620 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.577+0000 7f1b1bfff640 1 -- 172.21.15.164:0/4081819543 learned_addr learned my addr 172.21.15.164:0/4081819543 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:08.620 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.577+0000 7fd9ab7fe640 1 --2- 172.21.15.164:0/1641416357 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd9ac0b0440 0x7fd9ac1469a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:08.620 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.577+0000 7fd281317640 1 -- 172.21.15.164:0/670045958 shutdown_connections 2024-03-23T14:57:08.620 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.577+0000 7fd9b08ff640 1 --2- 172.21.15.164:0/1641416357 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd9ac0b2710 0x7fd9ac146ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:08.621 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.577+0000 7f0a697fa640 1 -- 172.21.15.164:0/1915291815 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a64014420 con 0x7f0a6c1486e0 2024-03-23T14:57:08.621 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.577+0000 7f0a697fa640 1 -- 172.21.15.164:0/1915291815 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a640145a0 con 0x7f0a6c1486e0 2024-03-23T14:57:08.621 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.577+0000 7f0a71df8640 1 -- 172.21.15.164:0/1915291815 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a6c149680 con 0x7f0a6c1486e0 2024-03-23T14:57:08.621 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.577+0000 7f0a697fa640 1 -- 172.21.15.164:0/1915291815 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a64014950 con 0x7f0a6c1486e0 2024-03-23T14:57:08.621 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.577+0000 7f0a71df8640 1 -- 172.21.15.164:0/1915291815 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a6c0b2f40 con 0x7f0a6c1486e0 2024-03-23T14:57:08.622 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.577+0000 7f1b1bfff640 1 -- 172.21.15.164:0/4081819543 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1b1c141fa0 msgr2=0x7f1b1c1e3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:08.622 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.577+0000 7f1b1bfff640 1 --2- 172.21.15.164:0/4081819543 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1b1c141fa0 0x7f1b1c1e3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:08.622 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.577+0000 7f1b1bfff640 1 -- 172.21.15.164:0/4081819543 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1b1c0b5720 msgr2=0x7f1b1c13d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:08.622 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.577+0000 7f1b1bfff640 1 --2- 172.21.15.164:0/4081819543 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1b1c0b5720 0x7f1b1c13d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:08.622 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.577+0000 7f1b1bfff640 1 -- 172.21.15.164:0/4081819543 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b100094d0 con 0x7f1b1c0727b0 2024-03-23T14:57:08.622 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.577+0000 7fd9ab7fe640 1 -- 172.21.15.164:0/1641416357 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd9ac072920 msgr2=0x7fd9ac145020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:08.622 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.577+0000 7fd9ab7fe640 1 --2- 172.21.15.164:0/1641416357 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd9ac072920 0x7fd9ac145020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:08.622 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.577+0000 7fd9ab7fe640 1 -- 172.21.15.164:0/1641416357 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd9ac0b2710 msgr2=0x7fd9ac146ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:08.622 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.577+0000 7fd9ab7fe640 1 --2- 172.21.15.164:0/1641416357 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd9ac0b2710 0x7fd9ac146ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:08.623 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.577+0000 7fd9ab7fe640 1 -- 172.21.15.164:0/1641416357 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd99c009b30 con 0x7fd9ac0b0440 2024-03-23T14:57:08.623 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.577+0000 7fdb8ffff640 1 -- 172.21.15.164:0/2153498436 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdb78004d10 msgr2=0x7fdb78005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:08.623 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.577+0000 7fdb8ffff640 1 --2- 172.21.15.164:0/2153498436 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdb78004d10 0x7fdb78005190 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fdb8800c490 tx=0x7fdb8801ce40 comp rx=0 tx=0).stop 2024-03-23T14:57:08.623 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.577+0000 7f3cdbfaa640 1 -- 172.21.15.164:0/3811463969 shutdown_connections 2024-03-23T14:57:08.623 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.577+0000 7fb367fff640 1 -- 172.21.15.164:0/57348807 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb37000f8c0 con 0x7fb3780c84b0 2024-03-23T14:57:08.624 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.577+0000 7fb367fff640 1 -- 172.21.15.164:0/57348807 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb37000fa40 con 0x7fb3780c84b0 2024-03-23T14:57:08.624 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.577+0000 7fdb8f7fe640 1 -- 172.21.15.164:0/2153498436 reap_dead start 2024-03-23T14:57:08.624 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.577+0000 7f1b1bfff640 1 --2- 172.21.15.164:0/4081819543 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b1c0727b0 0x7f1b1c14eda0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f1b1400efc0 tx=0x7f1b1400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:08.624 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.577+0000 7fb367fff640 1 -- 172.21.15.164:0/57348807 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb37000fdf0 con 0x7fb3780c84b0 2024-03-23T14:57:08.624 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.577+0000 7fb37ffff640 1 -- 172.21.15.164:0/57348807 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3780c62e0 con 0x7fb3780c84b0 2024-03-23T14:57:08.625 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.577+0000 7fb37ffff640 1 -- 172.21.15.164:0/57348807 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3780c47c0 con 0x7fb3780c84b0 2024-03-23T14:57:08.625 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.577+0000 7fdb8ffff640 1 -- 172.21.15.164:0/2153498436 shutdown_connections 2024-03-23T14:57:08.625 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.577+0000 7fdb8ffff640 1 -- 172.21.15.164:0/2153498436 >> 172.21.15.164:0/2153498436 conn(0x7fdb9006bc80 msgr2=0x7fdb9006c090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:08.625 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.577+0000 7fd9ab7fe640 1 --2- 172.21.15.164:0/1641416357 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd9ac0b0440 0x7fd9ac1469a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fd99c0ab000 tx=0x7fd99c019720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:08.625 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.577+0000 7fb37ffff640 1 -- 172.21.15.164:0/57348807 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb378036e40 con 0x7fb3780c84b0 2024-03-23T14:57:08.626 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.577+0000 7f3cdbfaa640 1 -- 172.21.15.164:0/3811463969 wait complete. 2024-03-23T14:57:08.626 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.577+0000 7fd9a8ff9640 1 -- 172.21.15.164:0/1641416357 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd99c0a5690 con 0x7fd9ac0b0440 2024-03-23T14:57:08.626 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.577+0000 7fd281317640 1 -- 172.21.15.164:0/670045958 wait complete. 2024-03-23T14:57:08.627 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.577+0000 7f3cdbfaa640 1 Processor -- start 2024-03-23T14:57:08.627 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.577+0000 7f9b98ff9640 1 -- 172.21.15.164:0/479814664 <== mon.2 v2:172.21.15.173:3301/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7f9b900073d0 con 0x7f9b94006eb0 2024-03-23T14:57:08.627 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.577+0000 7f9b98ff9640 1 --2- 172.21.15.164:0/479814664 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f9b80067370 0x7f9b80069830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:08.627 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.577+0000 7f9b98ff9640 1 -- 172.21.15.164:0/479814664 <== mon.2 v2:172.21.15.173:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f9b90084d00 con 0x7f9b94006eb0 2024-03-23T14:57:08.627 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.577+0000 7f9b98ff9640 1 --2- 172.21.15.164:0/479814664 >> [v2:172.21.15.164:6824/812540271,v1:172.21.15.164:6825/812540271] conn(0x7f9b8006f5f0 0x7f9b80071a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:08.627 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.577+0000 7f9b98ff9640 1 -- 172.21.15.164:0/479814664 --> [v2:172.21.15.164:6824/812540271,v1:172.21.15.164:6825/812540271] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9b800720c0 con 0x7f9b8006f5f0 2024-03-23T14:57:08.628 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.577+0000 7f9b98ff9640 1 -- 172.21.15.164:0/479814664 <== mon.2 v2:172.21.15.173:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f9b900850b0 con 0x7f9b94006eb0 2024-03-23T14:57:08.628 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.577+0000 7f3cdbfaa640 1 -- start start 2024-03-23T14:57:08.628 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.577+0000 7fd9b1100640 1 -- 172.21.15.164:0/1641416357 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9ac148c80 con 0x7fd9ac0b0440 2024-03-23T14:57:08.628 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.577+0000 7fd9b1100640 1 -- 172.21.15.164:0/1641416357 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9ac155d60 con 0x7fd9ac0b0440 2024-03-23T14:57:08.628 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.577+0000 7fdb8ffff640 1 -- 172.21.15.164:0/2153498436 shutdown_connections 2024-03-23T14:57:08.628 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.577+0000 7fdb8ffff640 1 -- 172.21.15.164:0/2153498436 wait complete. 2024-03-23T14:57:08.629 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.577+0000 7fd281317640 1 Processor -- start 2024-03-23T14:57:08.629 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.577+0000 7f1b18ff9640 1 -- 172.21.15.164:0/4081819543 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b14014480 con 0x7f1b1c0727b0 2024-03-23T14:57:08.629 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.577+0000 7f1b18ff9640 1 -- 172.21.15.164:0/4081819543 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b1400f040 con 0x7f1b1c0727b0 2024-03-23T14:57:08.629 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.577+0000 7f1b18ff9640 1 -- 172.21.15.164:0/4081819543 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b14014800 con 0x7f1b1c0727b0 2024-03-23T14:57:08.629 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.577+0000 7f1b2251d640 1 -- 172.21.15.164:0/4081819543 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b1c141440 con 0x7f1b1c0727b0 2024-03-23T14:57:08.629 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.577+0000 7f1b2251d640 1 -- 172.21.15.164:0/4081819543 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b1c140550 con 0x7f1b1c0727b0 2024-03-23T14:57:08.630 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.581+0000 7fd9a8ff9640 1 -- 172.21.15.164:0/1641416357 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd99c0b8670 con 0x7fd9ac0b0440 2024-03-23T14:57:08.630 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.581+0000 7f1b2251d640 1 -- 172.21.15.164:0/4081819543 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1afc001040 con 0x7f1b1c0727b0 2024-03-23T14:57:08.631 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.581+0000 7f3cdbfaa640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3cd40b6630 0x7f3cd40b11c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:08.631 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.581+0000 7f3cdbfaa640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3cd4141710 0x7f3cd40b3110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:08.631 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.581+0000 7f3cdbfaa640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3cd40b3650 0x7f3cd41ef920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:08.631 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.581+0000 7f3cdbfaa640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd41ef490 con 0x7f3cd40b6630 2024-03-23T14:57:08.631 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.581+0000 7f3cdbfaa640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd40b3b20 con 0x7f3cd40b3650 2024-03-23T14:57:08.631 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.581+0000 7f3cdbfaa640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f3cd40b3e00 con 0x7f3cd4141710 2024-03-23T14:57:08.631 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.581+0000 7f3cd9d46640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3cd40b6630 0x7f3cd40b11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:08.631 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.581+0000 7f3cd9d46640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3cd40b6630 0x7f3cd40b11c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:59178/0 (socket says 172.21.15.164:59178) 2024-03-23T14:57:08.631 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.581+0000 7f3cd9d46640 1 -- 172.21.15.164:0/2713863714 learned_addr learned my addr 172.21.15.164:0/2713863714 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:08.631 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.581+0000 7f3cda547640 1 --2- 172.21.15.164:0/2713863714 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3cd40b3650 0x7f3cd41ef920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:08.632 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.581+0000 7fd9a8ff9640 1 -- 172.21.15.164:0/1641416357 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd99c0b8a00 con 0x7fd9ac0b0440 2024-03-23T14:57:08.632 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.581+0000 7f3cd9545640 1 --2- 172.21.15.164:0/2713863714 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3cd4141710 0x7f3cd40b3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:08.632 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.581+0000 7f9b9b7fe640 1 --2- 172.21.15.164:0/479814664 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f9b80067370 0x7f9b80069830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:08.632 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.581+0000 7f9b9b7fe640 1 --2- 172.21.15.164:0/479814664 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f9b80067370 0x7f9b80069830 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c050790 tx=0x7f9b9c069c70 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:08.633 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.581+0000 7fdb8ffff640 1 Processor -- start 2024-03-23T14:57:08.633 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.581+0000 7fdb8ffff640 1 -- start start 2024-03-23T14:57:08.633 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.581+0000 7f9ba0bee640 1 --2- 172.21.15.164:0/479814664 >> [v2:172.21.15.164:6824/812540271,v1:172.21.15.164:6825/812540271] conn(0x7f9b8006f5f0 0x7f9b80071a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:08.633 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.581+0000 7fdb8ffff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdb900715d0 0x7fdb901dff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:08.633 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.581+0000 7f9ba0bee640 1 --2- 172.21.15.164:0/479814664 >> [v2:172.21.15.164:6824/812540271,v1:172.21.15.164:6825/812540271] conn(0x7f9b8006f5f0 0x7f9b80071a10 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:08.634 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.581+0000 7fdb8ffff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdb901e04b0 0x7fdb9013e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:08.634 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.581+0000 7fdb8ffff640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdb9013eda0 0x7fdb901dc620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:08.634 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.581+0000 7fdb8ffff640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fdb901dfb40 con 0x7fdb901e04b0 2024-03-23T14:57:08.634 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.581+0000 7fdb8ffff640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fdb90148bb0 con 0x7fdb900715d0 2024-03-23T14:57:08.634 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.581+0000 7fdb8ffff640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fdb901e0930 con 0x7fdb9013eda0 2024-03-23T14:57:08.634 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.581+0000 7fd9a8ff9640 1 -- 172.21.15.164:0/1641416357 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7fd99c0b8b60 con 0x7fd9ac0b0440 2024-03-23T14:57:08.635 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.581+0000 7f3cd9d46640 1 -- 172.21.15.164:0/2713863714 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3cd4141710 msgr2=0x7f3cd40b3110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:08.635 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.581+0000 7f3cd9d46640 1 --2- 172.21.15.164:0/2713863714 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3cd4141710 0x7f3cd40b3110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:08.635 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.581+0000 7f3cd9d46640 1 -- 172.21.15.164:0/2713863714 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3cd40b3650 msgr2=0x7f3cd41ef920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:08.635 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.581+0000 7f3cd9d46640 1 --2- 172.21.15.164:0/2713863714 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3cd40b3650 0x7f3cd41ef920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:08.635 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.581+0000 7f3cd9d46640 1 -- 172.21.15.164:0/2713863714 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ccc009b30 con 0x7f3cd40b6630 2024-03-23T14:57:08.635 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.581+0000 7f3cd9d46640 1 --2- 172.21.15.164:0/2713863714 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3cd40b6630 0x7f3cd40b11c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f3cc400d800 tx=0x7f3cc400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:08.635 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.581+0000 7fd281317640 1 -- start start 2024-03-23T14:57:08.636 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.581+0000 7fdb8f7fe640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdb9013eda0 0x7fdb901dc620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:08.636 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.581+0000 7fdb8f7fe640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdb9013eda0 0x7fdb901dc620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.164:60584/0 (socket says 172.21.15.164:60584) 2024-03-23T14:57:08.636 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.581+0000 7fdb8f7fe640 1 -- 172.21.15.164:0/2676194799 learned_addr learned my addr 172.21.15.164:0/2676194799 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:08.636 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.581+0000 7fdb8e7fc640 1 --2- 172.21.15.164:0/2676194799 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdb901e04b0 0x7fdb9013e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:08.636 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.581+0000 7f1b18ff9640 1 -- 172.21.15.164:0/4081819543 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7f1b14014960 con 0x7f1b1c0727b0 2024-03-23T14:57:08.636 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.581+0000 7f1b18ff9640 1 --2- 172.21.15.164:0/4081819543 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f1b08067370 0x7f1b08069830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:08.636 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.581+0000 7f1b18ff9640 1 -- 172.21.15.164:0/4081819543 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f1b14086640 con 0x7f1b1c0727b0 2024-03-23T14:57:08.636 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.581+0000 7f1b1b7fe640 1 --2- 172.21.15.164:0/4081819543 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f1b08067370 0x7f1b08069830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:08.636 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.581+0000 7f1b18ff9640 1 --2- 172.21.15.164:0/4081819543 >> [v2:172.21.15.164:6816/1349657239,v1:172.21.15.164:6817/1349657239] conn(0x7f1b0806f570 0x7f1b08071990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:08.636 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.581+0000 7f1b18ff9640 1 -- 172.21.15.164:0/4081819543 --> [v2:172.21.15.164:6816/1349657239,v1:172.21.15.164:6817/1349657239] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1b08072040 con 0x7f1b0806f570 2024-03-23T14:57:08.637 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.581+0000 7fd9a8ff9640 1 --2- 172.21.15.164:0/1641416357 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fd9980672f0 0x7fd9980697b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:08.637 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.581+0000 7f1b18ff9640 1 -- 172.21.15.164:0/4081819543 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f1b14053260 con 0x7f1b1c0727b0 2024-03-23T14:57:08.637 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.581+0000 7fd9abfff640 1 --2- 172.21.15.164:0/1641416357 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fd9980672f0 0x7fd9980697b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:08.637 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.581+0000 7fb367fff640 1 -- 172.21.15.164:0/57348807 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7fb37001d460 con 0x7fb3780c84b0 2024-03-23T14:57:08.638 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.581+0000 7fd281317640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd27c06d590 0x7fd27c13e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:08.638 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.581+0000 7fd281317640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd27c0b2870 0x7fd27c140660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:08.638 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.581+0000 7fd281317640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd27c146770 0x7fd27c1dba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:08.638 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.581+0000 7fd281317640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd27c13f990 con 0x7fd27c06d590 2024-03-23T14:57:08.638 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.581+0000 7fd281317640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd27c13dea0 con 0x7fd27c0b2870 2024-03-23T14:57:08.638 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.581+0000 7fd281317640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fd27c070800 con 0x7fd27c146770 2024-03-23T14:57:08.638 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.581+0000 7fd9b1100640 1 -- 172.21.15.164:0/1641416357 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd98c001040 con 0x7fd9ac0b0440 2024-03-23T14:57:08.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.581+0000 7fd27bfff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd27c06d590 0x7fd27c13e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:08.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.581+0000 7fd27bfff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd27c06d590 0x7fd27c13e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:59194/0 (socket says 172.21.15.164:59194) 2024-03-23T14:57:08.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.581+0000 7fd27bfff640 1 -- 172.21.15.164:0/3058052105 learned_addr learned my addr 172.21.15.164:0/3058052105 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:08.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.585+0000 7fd27bfff640 1 -- 172.21.15.164:0/3058052105 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd27c146770 msgr2=0x7fd27c1dba00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-23T14:57:08.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.585+0000 7fd27bfff640 1 --2- 172.21.15.164:0/3058052105 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd27c146770 0x7fd27c1dba00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:08.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.585+0000 7fd27bfff640 1 -- 172.21.15.164:0/3058052105 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd27c0b2870 msgr2=0x7fd27c140660 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-23T14:57:08.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.585+0000 7fd27bfff640 1 --2- 172.21.15.164:0/3058052105 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd27c0b2870 0x7fd27c140660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:08.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.585+0000 7fd27bfff640 1 -- 172.21.15.164:0/3058052105 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd26c009b30 con 0x7fd27c06d590 2024-03-23T14:57:08.640 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.585+0000 7fd27bfff640 1 --2- 172.21.15.164:0/3058052105 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd27c06d590 0x7fd27c13e680 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd27400b410 tx=0x7fd27400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:08.640 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.585+0000 7fd278ff9640 1 -- 172.21.15.164:0/3058052105 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd274004020 con 0x7fd27c06d590 2024-03-23T14:57:08.640 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.585+0000 7fd278ff9640 1 -- 172.21.15.164:0/3058052105 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd274004180 con 0x7fd27c06d590 2024-03-23T14:57:08.640 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.585+0000 7fd278ff9640 1 -- 172.21.15.164:0/3058052105 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd274011630 con 0x7fd27c06d590 2024-03-23T14:57:08.640 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.585+0000 7f1b1b7fe640 1 --2- 172.21.15.164:0/4081819543 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f1b08067370 0x7f1b08069830 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f1b1c0b4cc0 tx=0x7f1b0c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:08.640 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.585+0000 7f1b20aba640 1 --2- 172.21.15.164:0/4081819543 >> [v2:172.21.15.164:6816/1349657239,v1:172.21.15.164:6817/1349657239] conn(0x7f1b0806f570 0x7f1b08071990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:08.640 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.581+0000 7fb367fff640 1 --2- 172.21.15.164:0/57348807 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fb36c067370 0x7fb36c069830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:08.640 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.581+0000 7fb367fff640 1 -- 172.21.15.164:0/57348807 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fb370085a10 con 0x7fb3780c84b0 2024-03-23T14:57:08.640 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.581+0000 7fb367fff640 1 --2- 172.21.15.164:0/57348807 >> [v2:172.21.15.164:6808/887861252,v1:172.21.15.164:6809/887861252] conn(0x7fb36c06f5f0 0x7fb36c071a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:08.641 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.581+0000 7fb37e7fc640 1 --2- 172.21.15.164:0/57348807 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fb36c067370 0x7fb36c069830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:08.641 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.581+0000 7fb367fff640 1 -- 172.21.15.164:0/57348807 --> [v2:172.21.15.164:6808/887861252,v1:172.21.15.164:6809/887861252] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb36c0720c0 con 0x7fb36c06f5f0 2024-03-23T14:57:08.641 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.581+0000 7fb367fff640 1 -- 172.21.15.164:0/57348807 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fb370085dc0 con 0x7fb3780c84b0 2024-03-23T14:57:08.641 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.585+0000 7fb37f7fe640 1 --2- 172.21.15.164:0/57348807 >> [v2:172.21.15.164:6808/887861252,v1:172.21.15.164:6809/887861252] conn(0x7fb36c06f5f0 0x7fb36c071a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:08.641 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.585+0000 7fb37f7fe640 1 --2- 172.21.15.164:0/57348807 >> [v2:172.21.15.164:6808/887861252,v1:172.21.15.164:6809/887861252] conn(0x7fb36c06f5f0 0x7fb36c071a10 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:08.641 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.581+0000 7fdb8f7fe640 1 -- 172.21.15.164:0/2676194799 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdb900715d0 msgr2=0x7fdb901dff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:08.641 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.581+0000 7fdb8f7fe640 1 --2- 172.21.15.164:0/2676194799 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdb900715d0 0x7fdb901dff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:08.641 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.581+0000 7fdb8f7fe640 1 -- 172.21.15.164:0/2676194799 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdb901e04b0 msgr2=0x7fdb9013e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:08.641 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.581+0000 7fdb8f7fe640 1 --2- 172.21.15.164:0/2676194799 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdb901e04b0 0x7fdb9013e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:08.641 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.581+0000 7fdb8f7fe640 1 -- 172.21.15.164:0/2676194799 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb900719b0 con 0x7fdb9013eda0 2024-03-23T14:57:08.641 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.581+0000 7fdb8f7fe640 1 --2- 172.21.15.164:0/2676194799 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdb9013eda0 0x7fdb901dc620 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fdb8800c490 tx=0x7fdb78005800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:08.642 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.585+0000 7fdb77fff640 1 -- 172.21.15.164:0/2676194799 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb880a5ea0 con 0x7fdb9013eda0 2024-03-23T14:57:08.642 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.585+0000 7fdb77fff640 1 -- 172.21.15.164:0/2676194799 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb880011d0 con 0x7fdb9013eda0 2024-03-23T14:57:08.642 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.585+0000 7fdb8ffff640 1 -- 172.21.15.164:0/2676194799 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb90149090 con 0x7fdb9013eda0 2024-03-23T14:57:08.642 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.585+0000 7fdb77fff640 1 -- 172.21.15.164:0/2676194799 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb880a5d40 con 0x7fdb9013eda0 2024-03-23T14:57:08.642 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.585+0000 7fdb8ffff640 1 -- 172.21.15.164:0/2676194799 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb90148db0 con 0x7fdb9013eda0 2024-03-23T14:57:08.642 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.585+0000 7fd9a8ff9640 1 -- 172.21.15.164:0/1641416357 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fd99c120110 con 0x7fd9ac0b0440 2024-03-23T14:57:08.642 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.585+0000 7fd9a8ff9640 1 --2- 172.21.15.164:0/1641416357 >> [v2:172.21.15.173:6808/3631684087,v1:172.21.15.173:6809/3631684087] conn(0x7fd99806f380 0x7fd9980717a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:08.642 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.585+0000 7fd9a8ff9640 1 -- 172.21.15.164:0/1641416357 --> [v2:172.21.15.173:6808/3631684087,v1:172.21.15.173:6809/3631684087] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd998071e50 con 0x7fd99806f380 2024-03-23T14:57:08.642 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.585+0000 7fd9a8ff9640 1 -- 172.21.15.164:0/1641416357 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fd99c0ed260 con 0x7fd9ac0b0440 2024-03-23T14:57:08.643 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.585+0000 7fd9b08ff640 1 --2- 172.21.15.164:0/1641416357 >> [v2:172.21.15.173:6808/3631684087,v1:172.21.15.173:6809/3631684087] conn(0x7fd99806f380 0x7fd9980717a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:08.643 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.585+0000 7fdb8ffff640 1 -- 172.21.15.164:0/2676194799 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fdb900b4dc0 con 0x7fdb9013eda0 2024-03-23T14:57:08.643 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.585+0000 7fd9abfff640 1 --2- 172.21.15.164:0/1641416357 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fd9980672f0 0x7fd9980697b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd9a00099b0 tx=0x7fd9a0009110 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:08.643 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.585+0000 7fb37e7fc640 1 --2- 172.21.15.164:0/57348807 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fb36c067370 0x7fb36c069830 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb368005660 tx=0x7fb380096ea0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:08.644 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.585+0000 7fd281317640 1 -- 172.21.15.164:0/3058052105 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd27c071bf0 con 0x7fd27c06d590 2024-03-23T14:57:08.644 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.585+0000 7fd281317640 1 -- 172.21.15.164:0/3058052105 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd27c13fc00 con 0x7fd27c06d590 2024-03-23T14:57:08.644 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.585+0000 7fd281317640 1 -- 172.21.15.164:0/3058052105 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd27c0b0250 con 0x7fd27c06d590 2024-03-23T14:57:08.644 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.585+0000 7f3cca7fc640 1 -- 172.21.15.164:0/2713863714 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cc400b6c0 con 0x7f3cd40b6630 2024-03-23T14:57:08.644 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.585+0000 7f3cca7fc640 1 -- 172.21.15.164:0/2713863714 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3cc400b820 con 0x7f3cd40b6630 2024-03-23T14:57:08.644 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.585+0000 7f3cca7fc640 1 -- 172.21.15.164:0/2713863714 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cc400bb90 con 0x7f3cd40b6630 2024-03-23T14:57:08.644 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.585+0000 7fd9b08ff640 1 --2- 172.21.15.164:0/1641416357 >> [v2:172.21.15.173:6808/3631684087,v1:172.21.15.173:6809/3631684087] conn(0x7fd99806f380 0x7fd9980717a0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:08.645 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.585+0000 7f1b20aba640 1 --2- 172.21.15.164:0/4081819543 >> [v2:172.21.15.164:6816/1349657239,v1:172.21.15.164:6817/1349657239] conn(0x7f1b0806f570 0x7f1b08071990 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:08.645 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.585+0000 7fb367fff640 1 -- 172.21.15.164:0/57348807 <== osd.2 v2:172.21.15.164:6808/887861252 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb36c0720c0 con 0x7fb36c06f5f0 2024-03-23T14:57:08.645 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.585+0000 7f9b98ff9640 1 -- 172.21.15.164:0/479814664 <== osd.3 v2:172.21.15.164:6824/812540271 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9b800720c0 con 0x7f9b8006f5f0 2024-03-23T14:57:08.646 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.585+0000 7f3cdbfaa640 1 -- 172.21.15.164:0/2713863714 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cd406b580 con 0x7f3cd40b6630 2024-03-23T14:57:08.646 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.585+0000 7f3cdbfaa640 1 -- 172.21.15.164:0/2713863714 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cd41f0030 con 0x7f3cd40b6630 2024-03-23T14:57:08.646 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.585+0000 7fdb77fff640 1 -- 172.21.15.164:0/2676194799 <== mon.2 v2:172.21.15.173:3301/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7fdb880b4040 con 0x7fdb9013eda0 2024-03-23T14:57:08.646 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.585+0000 7f3cdbfaa640 1 -- 172.21.15.164:0/2713863714 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3cd40b3c90 con 0x7f3cd40b6630 2024-03-23T14:57:08.646 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.585+0000 7f0a697fa640 1 -- 172.21.15.164:0/1915291815 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7f0a64014ab0 con 0x7f0a6c1486e0 2024-03-23T14:57:08.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.585+0000 7fdb77fff640 1 --2- 172.21.15.164:0/2676194799 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fdb7c06fb30 0x7fdb7c071ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:08.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.585+0000 7fdb77fff640 1 -- 172.21.15.164:0/2676194799 <== mon.2 v2:172.21.15.173:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fdb8811eb50 con 0x7fdb9013eda0 2024-03-23T14:57:08.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.585+0000 7f0a71df8640 1 -- 172.21.15.164:0/1915291815 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0a6c073390 con 0x7f0a6c1486e0 2024-03-23T14:57:08.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.585+0000 7fdb77fff640 1 --2- 172.21.15.164:0/2676194799 >> [v2:172.21.15.173:6824/2656857421,v1:172.21.15.173:6825/2656857421] conn(0x7fdb7c077d50 0x7fdb7c07a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:08.648 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.585+0000 7f0a697fa640 1 --2- 172.21.15.164:0/1915291815 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f0a500672f0 0x7f0a500697b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:08.648 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.585+0000 7fdb77fff640 1 -- 172.21.15.164:0/2676194799 --> [v2:172.21.15.173:6824/2656857421,v1:172.21.15.173:6825/2656857421] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fdb7c07a820 con 0x7fdb7c077d50 2024-03-23T14:57:08.648 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.585+0000 7fdb77fff640 1 -- 172.21.15.164:0/2676194799 <== mon.2 v2:172.21.15.173:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fdb8811eff0 con 0x7fdb9013eda0 2024-03-23T14:57:08.648 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.589+0000 7fd278ff9640 1 -- 172.21.15.164:0/3058052105 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7fd274011790 con 0x7fd27c06d590 2024-03-23T14:57:08.649 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.589+0000 7f1b18ff9640 1 -- 172.21.15.164:0/4081819543 <== osd.1 v2:172.21.15.164:6816/1349657239 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1b08072040 con 0x7f1b0806f570 2024-03-23T14:57:08.649 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.589+0000 7f0a697fa640 1 -- 172.21.15.164:0/1915291815 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f0a640856a0 con 0x7f0a6c1486e0 2024-03-23T14:57:08.649 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.589+0000 7f0a697fa640 1 --2- 172.21.15.164:0/1915291815 >> [v2:172.21.15.173:6816/3261114948,v1:172.21.15.173:6817/3261114948] conn(0x7f0a5006f4f0 0x7f0a50071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:08.649 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.589+0000 7f0a697fa640 1 -- 172.21.15.164:0/1915291815 --> [v2:172.21.15.173:6816/3261114948,v1:172.21.15.173:6817/3261114948] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0a50071fc0 con 0x7f0a5006f4f0 2024-03-23T14:57:08.649 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.589+0000 7f0a70df6640 1 --2- 172.21.15.164:0/1915291815 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f0a500672f0 0x7f0a500697b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:08.649 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.589+0000 7f0a6bfff640 1 --2- 172.21.15.164:0/1915291815 >> [v2:172.21.15.173:6816/3261114948,v1:172.21.15.173:6817/3261114948] conn(0x7f0a5006f4f0 0x7f0a50071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:08.649 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.589+0000 7f0a697fa640 1 -- 172.21.15.164:0/1915291815 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f0a64052470 con 0x7f0a6c1486e0 2024-03-23T14:57:08.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.589+0000 7fdb8effd640 1 --2- 172.21.15.164:0/2676194799 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fdb7c06fb30 0x7fdb7c071ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:08.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.589+0000 7fdb8e7fc640 1 --2- 172.21.15.164:0/2676194799 >> [v2:172.21.15.173:6824/2656857421,v1:172.21.15.173:6825/2656857421] conn(0x7fdb7c077d50 0x7fdb7c07a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:08.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.589+0000 7fdb8e7fc640 1 --2- 172.21.15.164:0/2676194799 >> [v2:172.21.15.173:6824/2656857421,v1:172.21.15.173:6825/2656857421] conn(0x7fdb7c077d50 0x7fdb7c07a170 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:08.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.589+0000 7fdb8effd640 1 --2- 172.21.15.164:0/2676194799 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fdb7c06fb30 0x7fdb7c071ff0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fdb80006c60 tx=0x7fdb800093f0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:08.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.589+0000 7fd9a8ff9640 1 -- 172.21.15.164:0/1641416357 <== osd.6 v2:172.21.15.173:6808/3631684087 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd998071e50 con 0x7fd99806f380 2024-03-23T14:57:08.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.589+0000 7f0a70df6640 1 --2- 172.21.15.164:0/1915291815 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f0a500672f0 0x7f0a500697b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0a600ae000 tx=0x7f0a600b3040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:08.651 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.589+0000 7f3cca7fc640 1 -- 172.21.15.164:0/2713863714 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7f3cc400bcf0 con 0x7f3cd40b6630 2024-03-23T14:57:08.651 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.589+0000 7f3cca7fc640 1 --2- 172.21.15.164:0/2713863714 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f3cb8067370 0x7f3cb8069830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:08.651 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.589+0000 7f3cca7fc640 1 -- 172.21.15.164:0/2713863714 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f3cc4085ef0 con 0x7f3cd40b6630 2024-03-23T14:57:08.651 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.589+0000 7f3cd9545640 1 --2- 172.21.15.164:0/2713863714 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f3cb8067370 0x7f3cb8069830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:08.651 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.589+0000 7f3cca7fc640 1 --2- 172.21.15.164:0/2713863714 >> [v2:172.21.15.173:6800/3676483185,v1:172.21.15.173:6801/3676483185] conn(0x7f3cb806f5f0 0x7f3cb8071a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:08.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.589+0000 7f3cca7fc640 1 -- 172.21.15.164:0/2713863714 --> [v2:172.21.15.173:6800/3676483185,v1:172.21.15.173:6801/3676483185] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3cb80720c0 con 0x7f3cb806f5f0 2024-03-23T14:57:08.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.589+0000 7f3cca7fc640 1 -- 172.21.15.164:0/2713863714 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f3cc40862a0 con 0x7f3cd40b6630 2024-03-23T14:57:08.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.589+0000 7f3cda547640 1 --2- 172.21.15.164:0/2713863714 >> [v2:172.21.15.173:6800/3676483185,v1:172.21.15.173:6801/3676483185] conn(0x7f3cb806f5f0 0x7f3cb8071a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:08.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.589+0000 7f3cd9545640 1 --2- 172.21.15.164:0/2713863714 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f3cb8067370 0x7f3cb8069830 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f3ccc0ae000 tx=0x7f3ccc0a7070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:08.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.589+0000 7f0a6bfff640 1 --2- 172.21.15.164:0/1915291815 >> [v2:172.21.15.173:6816/3261114948,v1:172.21.15.173:6817/3261114948] conn(0x7f0a5006f4f0 0x7f0a50071910 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:08.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.593+0000 7f0a697fa640 1 -- 172.21.15.164:0/1915291815 <== osd.7 v2:172.21.15.173:6816/3261114948 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0a50071fc0 con 0x7f0a5006f4f0 2024-03-23T14:57:08.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.593+0000 7fdb77fff640 1 -- 172.21.15.164:0/2676194799 <== osd.4 v2:172.21.15.173:6824/2656857421 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fdb7c07a820 con 0x7fdb7c077d50 2024-03-23T14:57:08.653 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.593+0000 7f3cda547640 1 --2- 172.21.15.164:0/2713863714 >> [v2:172.21.15.173:6800/3676483185,v1:172.21.15.173:6801/3676483185] conn(0x7f3cb806f5f0 0x7f3cb8071a10 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:08.653 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.593+0000 7f3cca7fc640 1 -- 172.21.15.164:0/2713863714 <== osd.5 v2:172.21.15.173:6800/3676483185 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3cb80720c0 con 0x7f3cb806f5f0 2024-03-23T14:57:08.653 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.601+0000 7f9ba13ef640 1 -- 172.21.15.164:0/479814664 --> [v2:172.21.15.164:6824/812540271,v1:172.21.15.164:6825/812540271] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9b94039510 con 0x7f9b8006f5f0 2024-03-23T14:57:08.653 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.601+0000 7fb37ffff640 1 -- 172.21.15.164:0/57348807 --> [v2:172.21.15.164:6808/887861252,v1:172.21.15.164:6809/887861252] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb378036e40 con 0x7fb36c06f5f0 2024-03-23T14:57:08.654 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.601+0000 7fd278ff9640 1 --2- 172.21.15.164:0/3058052105 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fd25c0672d0 0x7fd25c069790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:08.654 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.601+0000 7fd278ff9640 1 -- 172.21.15.164:0/3058052105 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fd274085650 con 0x7fd27c06d590 2024-03-23T14:57:08.654 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.601+0000 7fd278ff9640 1 --2- 172.21.15.164:0/3058052105 >> [v2:172.21.15.164:6800/984716690,v1:172.21.15.164:6801/984716690] conn(0x7fd25c06f550 0x7fd25c071970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:08.654 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.601+0000 7fd27b7fe640 1 --2- 172.21.15.164:0/3058052105 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fd25c0672d0 0x7fd25c069790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:08.654 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.601+0000 7fb367fff640 1 -- 172.21.15.164:0/57348807 <== osd.2 v2:172.21.15.164:6808/887861252 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fb374006be0 con 0x7fb36c06f5f0 2024-03-23T14:57:08.655 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.601+0000 7f9b98ff9640 1 -- 172.21.15.164:0/479814664 <== osd.3 v2:172.21.15.164:6824/812540271 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f9b94039510 con 0x7f9b8006f5f0 2024-03-23T14:57:08.655 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.601+0000 7f9ba13ef640 1 -- 172.21.15.164:0/479814664 >> [v2:172.21.15.164:6824/812540271,v1:172.21.15.164:6825/812540271] conn(0x7f9b8006f5f0 msgr2=0x7f9b80071a10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:08.655 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.601+0000 7f9ba13ef640 1 --2- 172.21.15.164:0/479814664 >> [v2:172.21.15.164:6824/812540271,v1:172.21.15.164:6825/812540271] conn(0x7f9b8006f5f0 0x7f9b80071a10 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:08.655 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.601+0000 7f9ba13ef640 1 -- 172.21.15.164:0/479814664 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f9b80067370 msgr2=0x7f9b80069830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:08.656 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.601+0000 7f9ba13ef640 1 --2- 172.21.15.164:0/479814664 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f9b80067370 0x7f9b80069830 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c050790 tx=0x7f9b9c069c70 comp rx=0 tx=0).stop 2024-03-23T14:57:08.656 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.601+0000 7fd278ff9640 1 -- 172.21.15.164:0/3058052105 --> [v2:172.21.15.164:6800/984716690,v1:172.21.15.164:6801/984716690] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd25c072020 con 0x7fd25c06f550 2024-03-23T14:57:08.656 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.601+0000 7fd278ff9640 1 -- 172.21.15.164:0/3058052105 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fd274085a00 con 0x7fd27c06d590 2024-03-23T14:57:08.656 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.601+0000 7fd280b16640 1 --2- 172.21.15.164:0/3058052105 >> [v2:172.21.15.164:6800/984716690,v1:172.21.15.164:6801/984716690] conn(0x7fd25c06f550 0x7fd25c071970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:08.657 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.601+0000 7f9ba13ef640 1 -- 172.21.15.164:0/479814664 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9b94006eb0 msgr2=0x7f9b940b7150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:08.657 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.601+0000 7f9ba13ef640 1 --2- 172.21.15.164:0/479814664 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9b94006eb0 0x7f9b940b7150 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f9b9000c4d0 tx=0x7f9b9000c990 comp rx=0 tx=0).stop 2024-03-23T14:57:08.657 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.601+0000 7f9ba0bee640 1 -- 172.21.15.164:0/479814664 reap_dead start 2024-03-23T14:57:08.657 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.601+0000 7f9ba13ef640 1 -- 172.21.15.164:0/479814664 shutdown_connections 2024-03-23T14:57:08.657 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.601+0000 7f9ba13ef640 1 -- 172.21.15.164:0/479814664 >> 172.21.15.164:0/479814664 conn(0x7f9b9403a860 msgr2=0x7f9b94006650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:08.657 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.605+0000 7f9ba13ef640 1 -- 172.21.15.164:0/479814664 shutdown_connections 2024-03-23T14:57:08.657 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.605+0000 7f9ba13ef640 1 -- 172.21.15.164:0/479814664 wait complete. 2024-03-23T14:57:08.658 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.605+0000 7fd280b16640 1 --2- 172.21.15.164:0/3058052105 >> [v2:172.21.15.164:6800/984716690,v1:172.21.15.164:6801/984716690] conn(0x7fd25c06f550 0x7fd25c071970 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:08.658 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.605+0000 7fd27b7fe640 1 --2- 172.21.15.164:0/3058052105 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fd25c0672d0 0x7fd25c069790 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd27c13c510 tx=0x7fd2640079e0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:08.658 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.605+0000 7fd278ff9640 1 -- 172.21.15.164:0/3058052105 <== osd.0 v2:172.21.15.164:6800/984716690 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd25c072020 con 0x7fd25c06f550 2024-03-23T14:57:08.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.613+0000 7fb37ffff640 1 -- 172.21.15.164:0/57348807 >> [v2:172.21.15.164:6808/887861252,v1:172.21.15.164:6809/887861252] conn(0x7fb36c06f5f0 msgr2=0x7fb36c071a10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:08.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.613+0000 7fb37ffff640 1 --2- 172.21.15.164:0/57348807 >> [v2:172.21.15.164:6808/887861252,v1:172.21.15.164:6809/887861252] conn(0x7fb36c06f5f0 0x7fb36c071a10 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:08.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.613+0000 7fb37ffff640 1 -- 172.21.15.164:0/57348807 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fb36c067370 msgr2=0x7fb36c069830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:08.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.613+0000 7fb37ffff640 1 --2- 172.21.15.164:0/57348807 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fb36c067370 0x7fb36c069830 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb368005660 tx=0x7fb380096ea0 comp rx=0 tx=0).stop 2024-03-23T14:57:08.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.613+0000 7fb37ffff640 1 -- 172.21.15.164:0/57348807 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb3780c84b0 msgr2=0x7fb378155fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:08.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.613+0000 7fb37ffff640 1 --2- 172.21.15.164:0/57348807 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb3780c84b0 0x7fb378155fb0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb370004a50 tx=0x7fb370004f10 comp rx=0 tx=0).stop 2024-03-23T14:57:08.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.617+0000 7fb37f7fe640 1 -- 172.21.15.164:0/57348807 reap_dead start 2024-03-23T14:57:08.660 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.617+0000 7fb37ffff640 1 -- 172.21.15.164:0/57348807 shutdown_connections 2024-03-23T14:57:08.660 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.617+0000 7fb37ffff640 1 -- 172.21.15.164:0/57348807 >> 172.21.15.164:0/57348807 conn(0x7fb37803a860 msgr2=0x7fb37803ac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:08.660 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.617+0000 7fb37ffff640 1 -- 172.21.15.164:0/57348807 shutdown_connections 2024-03-23T14:57:08.660 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.617+0000 7fb37ffff640 1 -- 172.21.15.164:0/57348807 wait complete. 2024-03-23T14:57:08.660 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.617+0000 7fd9b1100640 1 -- 172.21.15.164:0/1641416357 --> [v2:172.21.15.173:6808/3631684087,v1:172.21.15.173:6809/3631684087] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd98c002d40 con 0x7fd99806f380 2024-03-23T14:57:08.660 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.617+0000 7fd9a8ff9640 1 -- 172.21.15.164:0/1641416357 <== osd.6 v2:172.21.15.173:6808/3631684087 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fd98c002d40 con 0x7fd99806f380 2024-03-23T14:57:08.661 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.617+0000 7fd9b1100640 1 -- 172.21.15.164:0/1641416357 >> [v2:172.21.15.173:6808/3631684087,v1:172.21.15.173:6809/3631684087] conn(0x7fd99806f380 msgr2=0x7fd9980717a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:08.661 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.617+0000 7fd9b1100640 1 --2- 172.21.15.164:0/1641416357 >> [v2:172.21.15.173:6808/3631684087,v1:172.21.15.173:6809/3631684087] conn(0x7fd99806f380 0x7fd9980717a0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:08.661 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.617+0000 7fd9b1100640 1 -- 172.21.15.164:0/1641416357 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fd9980672f0 msgr2=0x7fd9980697b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:08.661 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.617+0000 7fd9b1100640 1 --2- 172.21.15.164:0/1641416357 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fd9980672f0 0x7fd9980697b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd9a00099b0 tx=0x7fd9a0009110 comp rx=0 tx=0).stop 2024-03-23T14:57:08.662 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.617+0000 7fd9b1100640 1 -- 172.21.15.164:0/1641416357 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd9ac0b0440 msgr2=0x7fd9ac1469a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:08.662 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.617+0000 7fd9b1100640 1 --2- 172.21.15.164:0/1641416357 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd9ac0b0440 0x7fd9ac1469a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fd99c0ab000 tx=0x7fd99c019720 comp rx=0 tx=0).stop 2024-03-23T14:57:08.662 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.617+0000 7fd9b08ff640 1 -- 172.21.15.164:0/1641416357 reap_dead start 2024-03-23T14:57:08.662 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.617+0000 7fd9b1100640 1 -- 172.21.15.164:0/1641416357 shutdown_connections 2024-03-23T14:57:08.662 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.617+0000 7fd9b1100640 1 -- 172.21.15.164:0/1641416357 >> 172.21.15.164:0/1641416357 conn(0x7fd9ac06bc80 msgr2=0x7fd9ac0b6480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:08.662 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.617+0000 7fd9b1100640 1 -- 172.21.15.164:0/1641416357 shutdown_connections 2024-03-23T14:57:08.662 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.617+0000 7fd9b1100640 1 -- 172.21.15.164:0/1641416357 wait complete. 2024-03-23T14:57:08.662 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.621+0000 7f1b2251d640 1 -- 172.21.15.164:0/4081819543 --> [v2:172.21.15.164:6816/1349657239,v1:172.21.15.164:6817/1349657239] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1afc002d40 con 0x7f1b0806f570 2024-03-23T14:57:08.662 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.621+0000 7f0a71df8640 1 -- 172.21.15.164:0/1915291815 --> [v2:172.21.15.173:6816/3261114948,v1:172.21.15.173:6817/3261114948] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0a6c073390 con 0x7f0a5006f4f0 2024-03-23T14:57:08.663 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.621+0000 7f1b18ff9640 1 -- 172.21.15.164:0/4081819543 <== osd.1 v2:172.21.15.164:6816/1349657239 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f1afc002d40 con 0x7f1b0806f570 2024-03-23T14:57:08.664 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.621+0000 7f1b027fc640 1 -- 172.21.15.164:0/4081819543 >> [v2:172.21.15.164:6816/1349657239,v1:172.21.15.164:6817/1349657239] conn(0x7f1b0806f570 msgr2=0x7f1b08071990 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:08.664 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.621+0000 7f1b027fc640 1 --2- 172.21.15.164:0/4081819543 >> [v2:172.21.15.164:6816/1349657239,v1:172.21.15.164:6817/1349657239] conn(0x7f1b0806f570 0x7f1b08071990 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:08.664 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.621+0000 7f1b027fc640 1 -- 172.21.15.164:0/4081819543 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f1b08067370 msgr2=0x7f1b08069830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:08.664 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.621+0000 7f1b027fc640 1 --2- 172.21.15.164:0/4081819543 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f1b08067370 0x7f1b08069830 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f1b1c0b4cc0 tx=0x7f1b0c009040 comp rx=0 tx=0).stop 2024-03-23T14:57:08.664 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.621+0000 7f0a697fa640 1 -- 172.21.15.164:0/1915291815 <== osd.7 v2:172.21.15.173:6816/3261114948 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f0a58001560 con 0x7f0a5006f4f0 2024-03-23T14:57:08.664 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.621+0000 7f1b027fc640 1 -- 172.21.15.164:0/4081819543 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b1c0727b0 msgr2=0x7f1b1c14eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:08.665 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.621+0000 7f1b027fc640 1 --2- 172.21.15.164:0/4081819543 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b1c0727b0 0x7f1b1c14eda0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f1b1400efc0 tx=0x7f1b1400c490 comp rx=0 tx=0).stop 2024-03-23T14:57:08.665 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.621+0000 7f1b20aba640 1 -- 172.21.15.164:0/4081819543 reap_dead start 2024-03-23T14:57:08.665 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.621+0000 7f3cdbfaa640 1 -- 172.21.15.164:0/2713863714 --> [v2:172.21.15.173:6800/3676483185,v1:172.21.15.173:6801/3676483185] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3cd4151790 con 0x7f3cb806f5f0 2024-03-23T14:57:08.665 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.621+0000 7f0a71df8640 1 -- 172.21.15.164:0/1915291815 >> [v2:172.21.15.173:6816/3261114948,v1:172.21.15.173:6817/3261114948] conn(0x7f0a5006f4f0 msgr2=0x7f0a50071910 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:08.665 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.621+0000 7f0a71df8640 1 --2- 172.21.15.164:0/1915291815 >> [v2:172.21.15.173:6816/3261114948,v1:172.21.15.173:6817/3261114948] conn(0x7f0a5006f4f0 0x7f0a50071910 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:08.666 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.621+0000 7f1b027fc640 1 -- 172.21.15.164:0/4081819543 shutdown_connections 2024-03-23T14:57:08.666 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.621+0000 7f1b027fc640 1 -- 172.21.15.164:0/4081819543 >> 172.21.15.164:0/4081819543 conn(0x7f1b1c02bfc0 msgr2=0x7f1b1c0bb1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:08.666 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.621+0000 7f1b027fc640 1 -- 172.21.15.164:0/4081819543 shutdown_connections 2024-03-23T14:57:08.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.621+0000 7f0a71df8640 1 -- 172.21.15.164:0/1915291815 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f0a500672f0 msgr2=0x7f0a500697b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:08.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.621+0000 7f0a71df8640 1 --2- 172.21.15.164:0/1915291815 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f0a500672f0 0x7f0a500697b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0a600ae000 tx=0x7f0a600b3040 comp rx=0 tx=0).stop 2024-03-23T14:57:08.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.621+0000 7f0a71df8640 1 -- 172.21.15.164:0/1915291815 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0a6c1486e0 msgr2=0x7f0a6c1451d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:08.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.621+0000 7f0a71df8640 1 --2- 172.21.15.164:0/1915291815 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0a6c1486e0 0x7f0a6c1451d0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f0a64007e60 tx=0x7f0a6400e4b0 comp rx=0 tx=0).stop 2024-03-23T14:57:08.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.621+0000 7f0a715f7640 1 -- 172.21.15.164:0/1915291815 reap_dead start 2024-03-23T14:57:08.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.621+0000 7f0a71df8640 1 -- 172.21.15.164:0/1915291815 shutdown_connections 2024-03-23T14:57:08.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.621+0000 7f0a71df8640 1 -- 172.21.15.164:0/1915291815 >> 172.21.15.164:0/1915291815 conn(0x7f0a6c06b960 msgr2=0x7f0a6c066ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:08.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.621+0000 7f1b027fc640 1 -- 172.21.15.164:0/4081819543 wait complete. 2024-03-23T14:57:08.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.621+0000 7f3cca7fc640 1 -- 172.21.15.164:0/2713863714 <== osd.5 v2:172.21.15.173:6800/3676483185 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f3cd0004b80 con 0x7f3cb806f5f0 2024-03-23T14:57:08.668 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.621+0000 7f0a71df8640 1 -- 172.21.15.164:0/1915291815 shutdown_connections 2024-03-23T14:57:08.668 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.621+0000 7f0a71df8640 1 -- 172.21.15.164:0/1915291815 wait complete. 2024-03-23T14:57:08.669 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.621+0000 7f3cdbfaa640 1 -- 172.21.15.164:0/2713863714 >> [v2:172.21.15.173:6800/3676483185,v1:172.21.15.173:6801/3676483185] conn(0x7f3cb806f5f0 msgr2=0x7f3cb8071a10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:08.669 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.621+0000 7f3cdbfaa640 1 --2- 172.21.15.164:0/2713863714 >> [v2:172.21.15.173:6800/3676483185,v1:172.21.15.173:6801/3676483185] conn(0x7f3cb806f5f0 0x7f3cb8071a10 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:08.669 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.621+0000 7f3cdbfaa640 1 -- 172.21.15.164:0/2713863714 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f3cb8067370 msgr2=0x7f3cb8069830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:08.669 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.621+0000 7f3cdbfaa640 1 --2- 172.21.15.164:0/2713863714 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f3cb8067370 0x7f3cb8069830 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f3ccc0ae000 tx=0x7f3ccc0a7070 comp rx=0 tx=0).stop 2024-03-23T14:57:08.669 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.621+0000 7f3cdbfaa640 1 -- 172.21.15.164:0/2713863714 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3cd40b6630 msgr2=0x7f3cd40b11c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:08.669 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.621+0000 7f3cdbfaa640 1 --2- 172.21.15.164:0/2713863714 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3cd40b6630 0x7f3cd40b11c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f3cc400d800 tx=0x7f3cc400dcc0 comp rx=0 tx=0).stop 2024-03-23T14:57:08.669 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.621+0000 7f3cda547640 1 -- 172.21.15.164:0/2713863714 reap_dead start 2024-03-23T14:57:08.669 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.621+0000 7f3cdbfaa640 1 -- 172.21.15.164:0/2713863714 shutdown_connections 2024-03-23T14:57:08.669 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.621+0000 7f3cdbfaa640 1 -- 172.21.15.164:0/2713863714 >> 172.21.15.164:0/2713863714 conn(0x7f3cd406d4d0 msgr2=0x7f3cd413bb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:08.669 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.621+0000 7f3cdbfaa640 1 -- 172.21.15.164:0/2713863714 shutdown_connections 2024-03-23T14:57:08.669 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.621+0000 7f3cdbfaa640 1 -- 172.21.15.164:0/2713863714 wait complete. 2024-03-23T14:57:08.670 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.625+0000 7fdb8ffff640 1 -- 172.21.15.164:0/2676194799 --> [v2:172.21.15.173:6824/2656857421,v1:172.21.15.173:6825/2656857421] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fdb901dfc90 con 0x7fdb7c077d50 2024-03-23T14:57:08.670 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.625+0000 7fdb77fff640 1 -- 172.21.15.164:0/2676194799 <== osd.4 v2:172.21.15.173:6824/2656857421 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fdb901dfc90 con 0x7fdb7c077d50 2024-03-23T14:57:08.671 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.625+0000 7fdb75ffb640 1 -- 172.21.15.164:0/2676194799 >> [v2:172.21.15.173:6824/2656857421,v1:172.21.15.173:6825/2656857421] conn(0x7fdb7c077d50 msgr2=0x7fdb7c07a170 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:08.671 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.625+0000 7fdb75ffb640 1 --2- 172.21.15.164:0/2676194799 >> [v2:172.21.15.173:6824/2656857421,v1:172.21.15.173:6825/2656857421] conn(0x7fdb7c077d50 0x7fdb7c07a170 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:08.671 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.625+0000 7fdb75ffb640 1 -- 172.21.15.164:0/2676194799 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fdb7c06fb30 msgr2=0x7fdb7c071ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:08.671 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.625+0000 7fdb75ffb640 1 --2- 172.21.15.164:0/2676194799 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fdb7c06fb30 0x7fdb7c071ff0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fdb80006c60 tx=0x7fdb800093f0 comp rx=0 tx=0).stop 2024-03-23T14:57:08.671 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.625+0000 7fdb75ffb640 1 -- 172.21.15.164:0/2676194799 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdb9013eda0 msgr2=0x7fdb901dc620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:08.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.625+0000 7fdb75ffb640 1 --2- 172.21.15.164:0/2676194799 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdb9013eda0 0x7fdb901dc620 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fdb8800c490 tx=0x7fdb78005800 comp rx=0 tx=0).stop 2024-03-23T14:57:08.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.625+0000 7fdb8f7fe640 1 -- 172.21.15.164:0/2676194799 reap_dead start 2024-03-23T14:57:08.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.625+0000 7fdb75ffb640 1 -- 172.21.15.164:0/2676194799 shutdown_connections 2024-03-23T14:57:08.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.625+0000 7fdb75ffb640 1 -- 172.21.15.164:0/2676194799 >> 172.21.15.164:0/2676194799 conn(0x7fdb9006bc80 msgr2=0x7fdb900b5130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:08.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.629+0000 7fdb75ffb640 1 -- 172.21.15.164:0/2676194799 shutdown_connections 2024-03-23T14:57:08.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.629+0000 7fdb75ffb640 1 -- 172.21.15.164:0/2676194799 wait complete. 2024-03-23T14:57:08.673 INFO:teuthology.orchestra.run.smithi164.stdout:55834574857 2024-03-23T14:57:08.673 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-03-23T14:57:08.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.633+0000 7fd281317640 1 -- 172.21.15.164:0/3058052105 --> [v2:172.21.15.164:6800/984716690,v1:172.21.15.164:6801/984716690] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd27c071a50 con 0x7fd25c06f550 2024-03-23T14:57:08.674 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.633+0000 7fd278ff9640 1 -- 172.21.15.164:0/3058052105 <== osd.0 v2:172.21.15.164:6800/984716690 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fd27c071a50 con 0x7fd25c06f550 2024-03-23T14:57:08.674 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.633+0000 7fd281317640 1 -- 172.21.15.164:0/3058052105 >> [v2:172.21.15.164:6800/984716690,v1:172.21.15.164:6801/984716690] conn(0x7fd25c06f550 msgr2=0x7fd25c071970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:08.675 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.633+0000 7fd281317640 1 --2- 172.21.15.164:0/3058052105 >> [v2:172.21.15.164:6800/984716690,v1:172.21.15.164:6801/984716690] conn(0x7fd25c06f550 0x7fd25c071970 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:08.675 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.637+0000 7fd281317640 1 -- 172.21.15.164:0/3058052105 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fd25c0672d0 msgr2=0x7fd25c069790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:08.675 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.637+0000 7fd281317640 1 --2- 172.21.15.164:0/3058052105 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fd25c0672d0 0x7fd25c069790 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd27c13c510 tx=0x7fd2640079e0 comp rx=0 tx=0).stop 2024-03-23T14:57:08.675 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.637+0000 7fd281317640 1 -- 172.21.15.164:0/3058052105 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd27c06d590 msgr2=0x7fd27c13e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:08.675 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.637+0000 7fd281317640 1 --2- 172.21.15.164:0/3058052105 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd27c06d590 0x7fd27c13e680 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd27400b410 tx=0x7fd27400b8d0 comp rx=0 tx=0).stop 2024-03-23T14:57:08.675 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.637+0000 7fd280b16640 1 -- 172.21.15.164:0/3058052105 reap_dead start 2024-03-23T14:57:08.675 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.637+0000 7fd281317640 1 -- 172.21.15.164:0/3058052105 shutdown_connections 2024-03-23T14:57:08.675 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.637+0000 7fd281317640 1 -- 172.21.15.164:0/3058052105 >> 172.21.15.164:0/3058052105 conn(0x7fd27c06bc80 msgr2=0x7fd27c0b0b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:08.675 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.637+0000 7fd281317640 1 -- 172.21.15.164:0/3058052105 shutdown_connections 2024-03-23T14:57:08.675 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.637+0000 7fd281317640 1 -- 172.21.15.164:0/3058052105 wait complete. 2024-03-23T14:57:08.676 INFO:teuthology.orchestra.run.smithi164.stdout:55834574856 2024-03-23T14:57:08.676 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-03-23T14:57:08.677 INFO:teuthology.orchestra.run.smithi164.stdout:55834574856 2024-03-23T14:57:08.677 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-03-23T14:57:08.678 INFO:teuthology.orchestra.run.smithi164.stdout:55834574856 2024-03-23T14:57:08.678 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-03-23T14:57:08.679 INFO:teuthology.orchestra.run.smithi164.stdout:55834574856 2024-03-23T14:57:08.679 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-03-23T14:57:08.680 INFO:teuthology.orchestra.run.smithi164.stdout:55834574857 2024-03-23T14:57:08.680 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-03-23T14:57:08.681 INFO:teuthology.orchestra.run.smithi164.stdout:55834574855 2024-03-23T14:57:08.681 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-03-23T14:57:08.682 INFO:teuthology.orchestra.run.smithi164.stdout:55834574856 2024-03-23T14:57:08.682 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-03-23T14:57:08.889 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.885+0000 7f15f3fff640 1 Processor -- start 2024-03-23T14:57:08.891 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.885+0000 7f15f3fff640 1 -- start start 2024-03-23T14:57:08.891 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.885+0000 7f15f3fff640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f15f40b0420 0x7f15f414ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:08.891 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.885+0000 7f15f3fff640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15f413d880 con 0x7f15f41430e0 2024-03-23T14:57:08.891 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.885+0000 7f15f3fff640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15f40b1400 con 0x7f15f4143c10 2024-03-23T14:57:08.891 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.885+0000 7f15f3fff640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f15f40b1ae0 con 0x7f15f40b0420 2024-03-23T14:57:08.891 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.885+0000 7f15f27fc640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f15f41430e0 0x7f15f40b0cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:42438/0 (socket says 172.21.15.164:42438) 2024-03-23T14:57:08.891 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.885+0000 7f15f27fc640 1 -- 172.21.15.164:0/1964623396 learned_addr learned my addr 172.21.15.164:0/1964623396 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-03-23T14:57:08.891 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.885+0000 7f15f37fe640 1 --2- 172.21.15.164:0/1964623396 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f15f40b0420 0x7f15f414ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:08.891 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.885+0000 7f15f1ffb640 1 -- 172.21.15.164:0/1964623396 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 561780308 0 0) 0x7f15f413d880 con 0x7f15f41430e0 2024-03-23T14:57:08.891 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.885+0000 7f15f1ffb640 1 -- 172.21.15.164:0/1964623396 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f15dc003530 con 0x7f15f41430e0 2024-03-23T14:57:08.892 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.885+0000 7f15f1ffb640 1 -- 172.21.15.164:0/1964623396 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3902255285 0 0) 0x7f15f40b1400 con 0x7f15f4143c10 2024-03-23T14:57:08.892 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.885+0000 7f15f1ffb640 1 -- 172.21.15.164:0/1964623396 --> v1:172.21.15.173:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f15f413d880 con 0x7f15f4143c10 2024-03-23T14:57:08.892 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.885+0000 7f15f37fe640 1 -- 172.21.15.164:0/1964623396 >> v1:172.21.15.173:6789/0 conn(0x7f15f4143c10 legacy=0x7f15f40b59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:08.892 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.885+0000 7f15f37fe640 1 -- 172.21.15.164:0/1964623396 >> v1:172.21.15.164:6789/0 conn(0x7f15f41430e0 legacy=0x7f15f40b0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:08.892 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.885+0000 7f15f1ffb640 1 -- 172.21.15.164:0/1964623396 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3139408699 0 0) 0x7f15dc003530 con 0x7f15f41430e0 2024-03-23T14:57:08.892 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.885+0000 7f15f37fe640 1 -- 172.21.15.164:0/1964623396 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15f40afcb0 con 0x7f15f40b0420 2024-03-23T14:57:08.893 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.889+0000 7fb132229640 1 Processor -- start 2024-03-23T14:57:08.893 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.889+0000 7f3ad2020640 1 Processor -- start 2024-03-23T14:57:08.893 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.889+0000 7f3ad2020640 1 -- start start 2024-03-23T14:57:08.893 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.889+0000 7f3ad2020640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3acc0b8060 0x7f3acc14abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:08.894 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.889+0000 7fb132229640 1 -- start start 2024-03-23T14:57:08.894 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.889+0000 7fb132229640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fb12c0bc580 0x7fb12c140f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:08.894 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.889+0000 7fb132229640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb12c09aef0 0x7fb12c141480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:08.894 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.889+0000 7fb132229640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb12c09be80 con 0x7fb12c0bab20 2024-03-23T14:57:08.894 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.889+0000 7fb132229640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 -- 0x7fb12c0bbf10 con 0x7fb12c0bc580 2024-03-23T14:57:08.894 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.889+0000 7fb132229640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fb12c09b990 con 0x7fb12c09aef0 2024-03-23T14:57:08.894 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.889+0000 7f7a98a67640 1 Processor -- start 2024-03-23T14:57:08.895 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.889+0000 7f3ad2020640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3acc0b8b40 con 0x7f3acc06cf20 2024-03-23T14:57:08.895 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.889+0000 7f3ad2020640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3acc06c290 con 0x7f3acc06beb0 2024-03-23T14:57:08.895 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.889+0000 7f3ad2020640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f3acc0b8ee0 con 0x7f3acc0b8060 2024-03-23T14:57:08.895 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.889+0000 7f7a98a67640 1 -- start start 2024-03-23T14:57:08.895 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.889+0000 7f80e6793640 1 Processor -- start 2024-03-23T14:57:08.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.889+0000 7f80e6793640 1 -- start start 2024-03-23T14:57:08.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.889+0000 7f3acaffd640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3acc0b8060 0x7f3acc14abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:08.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.889+0000 7f3acaffd640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3acc0b8060 0x7f3acc14abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.164:60606/0 (socket says 172.21.15.164:60606) 2024-03-23T14:57:08.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.889+0000 7f3acaffd640 1 -- 172.21.15.164:0/2563718276 learned_addr learned my addr 172.21.15.164:0/2563718276 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:08.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.889+0000 7f15f37fe640 1 --2- 172.21.15.164:0/1964623396 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f15f40b0420 0x7f15f414ea70 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f15e80097f0 tx=0x7f15e801d450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=914b03abe292744c server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:08.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.889+0000 7f15f1ffb640 1 -- 172.21.15.164:0/1964623396 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15ec001a00 con 0x7f15f40b0420 2024-03-23T14:57:08.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.889+0000 7f15f1ffb640 1 -- 172.21.15.164:0/1964623396 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15e80ad420 con 0x7f15f40b0420 2024-03-23T14:57:08.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.889+0000 7f15f1ffb640 1 -- 172.21.15.164:0/1964623396 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15e80ad7b0 con 0x7f15f40b0420 2024-03-23T14:57:08.897 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.889+0000 7fb12bfff640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb12c09aef0 0x7fb12c141480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:08.897 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.889+0000 7fb12bfff640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb12c09aef0 0x7fb12c141480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.164:60616/0 (socket says 172.21.15.164:60616) 2024-03-23T14:57:08.897 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.889+0000 7fb12bfff640 1 -- 172.21.15.164:0/3280889938 learned_addr learned my addr 172.21.15.164:0/3280889938 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:08.897 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.889+0000 7fb12affd640 1 --2- 172.21.15.164:0/3280889938 >> v2:172.21.15.173:3300/0 conn(0x7fb12c0bc580 0x7fb12c140f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:08.897 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.889+0000 7fb12a7fc640 1 -- 172.21.15.164:0/3280889938 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 672202011 0 0) 0x7fb12c09be80 con 0x7fb12c0bab20 2024-03-23T14:57:08.897 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.889+0000 7fb12a7fc640 1 -- 172.21.15.164:0/3280889938 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb114003530 con 0x7fb12c0bab20 2024-03-23T14:57:08.897 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.889+0000 7f2df4c1e640 1 Processor -- start 2024-03-23T14:57:08.897 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.889+0000 7f2df4c1e640 1 -- start start 2024-03-23T14:57:08.898 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.889+0000 7f80e6793640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f80e006cf30 0x7f80e00b25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:08.898 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.889+0000 7f80e6793640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f80e00b45e0 0x7f80e0072390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:08.898 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.889+0000 7f80e6793640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80e006d310 con 0x7f80e00728d0 2024-03-23T14:57:08.898 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.889+0000 7f80e6793640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 -- 0x7f80e00b4a70 con 0x7f80e00b45e0 2024-03-23T14:57:08.898 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.889+0000 7f80e6793640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f80e00b0970 con 0x7f80e006cf30 2024-03-23T14:57:08.898 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.889+0000 7f48d9b32640 1 Processor -- start 2024-03-23T14:57:08.898 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.889+0000 7f48d9b32640 1 -- start start 2024-03-23T14:57:08.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.889+0000 7f7a98a67640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f7a94070000 0x7f7a9413de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:08.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.889+0000 7f7a98a67640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f7a940729b0 0x7f7a9413e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:08.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.889+0000 7f7a98a67640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a940b0b00 con 0x7f7a940b6650 2024-03-23T14:57:08.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.889+0000 7f3aca7fc640 1 -- 172.21.15.164:0/2563718276 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 810479443 0 0) 0x7f3acc06c290 con 0x7f3acc06beb0 2024-03-23T14:57:08.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.889+0000 7f3aca7fc640 1 -- 172.21.15.164:0/2563718276 --> v1:172.21.15.173:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3ab4003530 con 0x7f3acc06beb0 2024-03-23T14:57:08.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.889+0000 7f48d9b32640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f48d4149000 0x7f48d414f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:08.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.889+0000 7f48d9b32640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48d40b46a0 con 0x7f48d40b3ac0 2024-03-23T14:57:08.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.889+0000 7f48d9b32640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48d40b6ef0 con 0x7f48d414efd0 2024-03-23T14:57:08.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.889+0000 7f48d9b32640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f48d40b66a0 con 0x7f48d4149000 2024-03-23T14:57:08.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.889+0000 7f7a98a67640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f7a940b6a30 con 0x7f7a94070000 2024-03-23T14:57:08.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.889+0000 7f7a98a67640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 -- 0x7f7a940707b0 con 0x7f7a940729b0 2024-03-23T14:57:08.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.889+0000 7f3acaffd640 1 -- 172.21.15.164:0/2563718276 >> v1:172.21.15.173:6789/0 conn(0x7f3acc06beb0 legacy=0x7f3acc1411e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:08.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.889+0000 7f3acaffd640 1 -- 172.21.15.164:0/2563718276 >> v1:172.21.15.164:6789/0 conn(0x7f3acc06cf20 legacy=0x7f3acc151b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:08.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.889+0000 7f48d37fe640 1 --1- >> v1:172.21.15.173:6789/0 conn(0x7f48d414efd0 0x7f48d40b5f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.173:6789/0 says I am v1:172.21.15.164:35166/0 (socket says 172.21.15.164:35166) 2024-03-23T14:57:08.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.889+0000 7f48d37fe640 1 -- 172.21.15.164:0/2309485625 learned_addr learned my addr 172.21.15.164:0/2309485625 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-03-23T14:57:08.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.889+0000 7f7a92ffd640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f7a94070000 0x7f7a9413de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:08.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.889+0000 7f7a92ffd640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f7a94070000 0x7f7a9413de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:59210/0 (socket says 172.21.15.164:59210) 2024-03-23T14:57:08.902 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.889+0000 7f7a92ffd640 1 -- 172.21.15.164:0/392402440 learned_addr learned my addr 172.21.15.164:0/392402440 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:08.902 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.889+0000 7f48d3fff640 1 --2- 172.21.15.164:0/2309485625 >> v2:172.21.15.164:3300/0 conn(0x7f48d4149000 0x7f48d414f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:08.902 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.889+0000 7f7a93fff640 1 --2- 172.21.15.164:0/392402440 >> v2:172.21.15.173:3300/0 conn(0x7f7a940729b0 0x7f7a9413e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:08.902 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.889+0000 7fc12359e640 1 Processor -- start 2024-03-23T14:57:08.903 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.889+0000 7f80e4d30640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f80e00728d0 0x7f80e00b3110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:42466/0 (socket says 172.21.15.164:42466) 2024-03-23T14:57:08.903 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.889+0000 7f80e4d30640 1 -- 172.21.15.164:0/3167312462 learned_addr learned my addr 172.21.15.164:0/3167312462 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-03-23T14:57:08.903 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.889+0000 7f3acaffd640 1 -- 172.21.15.164:0/2563718276 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3acc0b84f0 con 0x7f3acc0b8060 2024-03-23T14:57:08.904 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.889+0000 7fb12affd640 1 -- 172.21.15.164:0/3280889938 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb12c09aef0 msgr2=0x7fb12c141480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:08.904 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.889+0000 7fb12affd640 1 --2- 172.21.15.164:0/3280889938 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb12c09aef0 0x7fb12c141480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:08.904 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.889+0000 7fb12affd640 1 -- 172.21.15.164:0/3280889938 >> v1:172.21.15.164:6789/0 conn(0x7fb12c0bab20 legacy=0x7fb12c14a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:08.904 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.889+0000 7fb12affd640 1 -- 172.21.15.164:0/3280889938 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb12c0bcfa0 con 0x7fb12c0bc580 2024-03-23T14:57:08.904 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.889+0000 7f2df4c1e640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2df00b4f50 0x7f2df009dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:08.904 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.889+0000 7f2df4c1e640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f2df008b740 0x7f2df00b3be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:08.904 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.889+0000 7f2df4c1e640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2df006c760 con 0x7f2df008b2e0 2024-03-23T14:57:08.904 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.889+0000 7f2df4c1e640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f2df00af180 con 0x7f2df008b740 2024-03-23T14:57:08.904 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.889+0000 7f2df4c1e640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f2df00af460 con 0x7f2df00b4f50 2024-03-23T14:57:08.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.893+0000 7f48d27fc640 1 -- 172.21.15.164:0/2309485625 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1536417900 0 0) 0x7f48d40b46a0 con 0x7f48d40b3ac0 2024-03-23T14:57:08.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.893+0000 7f48d27fc640 1 -- 172.21.15.164:0/2309485625 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f48bc003530 con 0x7f48d40b3ac0 2024-03-23T14:57:08.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.893+0000 7f80deffd640 1 -- 172.21.15.164:0/3167312462 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 791253526 0 0) 0x7f80e006d310 con 0x7f80e00728d0 2024-03-23T14:57:08.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.893+0000 7f80dffff640 1 --2- 172.21.15.164:0/3167312462 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f80e006cf30 0x7f80e00b25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:08.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.893+0000 7f80df7fe640 1 --2- 172.21.15.164:0/3167312462 >> v2:172.21.15.173:3300/0 conn(0x7f80e00b45e0 0x7f80e0072390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:08.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.893+0000 7f48d3fff640 1 -- 172.21.15.164:0/2309485625 >> v1:172.21.15.173:6789/0 conn(0x7f48d414efd0 legacy=0x7f48d40b5f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:08.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.893+0000 7f48d3fff640 1 -- 172.21.15.164:0/2309485625 >> v1:172.21.15.164:6789/0 conn(0x7f48d40b3ac0 legacy=0x7f48d41488f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:08.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.893+0000 7f48d3fff640 1 -- 172.21.15.164:0/2309485625 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48d406d4c0 con 0x7f48d4149000 2024-03-23T14:57:08.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.893+0000 7f2dee59c640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2df00b4f50 0x7f2df009dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:08.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.893+0000 7f2dedd9b640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f2df008b2e0 0x7f2df00b34d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:42488/0 (socket says 172.21.15.164:42488) 2024-03-23T14:57:08.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.893+0000 7f2dedd9b640 1 -- 172.21.15.164:0/2731315284 learned_addr learned my addr 172.21.15.164:0/2731315284 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-03-23T14:57:08.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.893+0000 7f2ded59a640 1 -- 172.21.15.164:0/2731315284 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3508110064 0 0) 0x7f2df006c760 con 0x7f2df008b2e0 2024-03-23T14:57:08.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.893+0000 7f2ded59a640 1 -- 172.21.15.164:0/2731315284 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2dd8005740 con 0x7f2df008b2e0 2024-03-23T14:57:08.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.893+0000 7f2ded59a640 1 -- 172.21.15.164:0/2731315284 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2831394454 0 0) 0x7f2dd8005740 con 0x7f2df008b2e0 2024-03-23T14:57:08.907 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.893+0000 7f2ded59a640 1 -- 172.21.15.164:0/2731315284 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2df00b4f50 msgr2=0x7f2df009dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:08.907 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.893+0000 7f2ded59a640 1 --2- 172.21.15.164:0/2731315284 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2df00b4f50 0x7f2df009dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:08.907 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.893+0000 7f2ded59a640 1 -- 172.21.15.164:0/2731315284 >> v2:172.21.15.164:3300/0 conn(0x7f2df008b740 msgr2=0x7f2df00b3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:08.907 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.893+0000 7f2ded59a640 1 --2- 172.21.15.164:0/2731315284 >> v2:172.21.15.164:3300/0 conn(0x7f2df008b740 0x7f2df00b3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:08.907 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.893+0000 7f2ded59a640 1 -- 172.21.15.164:0/2731315284 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2df00adad0 con 0x7f2df008b2e0 2024-03-23T14:57:08.907 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.893+0000 7f2ded59a640 1 -- 172.21.15.164:0/2731315284 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2526390548 0 0) 0x7f2de4000ee0 con 0x7f2df008b2e0 2024-03-23T14:57:08.907 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.893+0000 7f2ded59a640 1 -- 172.21.15.164:0/2731315284 >> v1:172.21.15.164:6789/0 conn(0x7f2df008b2e0 legacy=0x7f2df00b34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:08.907 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.893+0000 7f2ded59a640 1 --2- 172.21.15.164:0/2731315284 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2dd8005b90 0x7f2dd8005fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:08.907 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.893+0000 7f2ded59a640 1 --2- 172.21.15.164:0/2731315284 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2dd80069e0 0x7f2dd8006e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:08.907 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.893+0000 7f2ded59a640 1 --2- 172.21.15.164:0/2731315284 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2dd80073a0 0x7f2dd800dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:08.907 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.893+0000 7f2ded59a640 1 -- 172.21.15.164:0/2731315284 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2df00af180 con 0x7f2dd8005b90 2024-03-23T14:57:08.907 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.893+0000 7f2ded59a640 1 -- 172.21.15.164:0/2731315284 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f2df00af460 con 0x7f2dd80073a0 2024-03-23T14:57:08.907 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.893+0000 7f2ded59a640 1 -- 172.21.15.164:0/2731315284 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f2dd800e350 con 0x7f2dd80069e0 2024-03-23T14:57:08.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.893+0000 7fc12359e640 1 -- start start 2024-03-23T14:57:08.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.893+0000 7fc12359e640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fc11c042c40 0x7fc11c0d47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:08.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.893+0000 7fc12359e640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fc11c03d630 0x7fc11c0422b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:08.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.893+0000 7fc12359e640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc11c0d3fa0 con 0x7fc11c0427f0 2024-03-23T14:57:08.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.893+0000 7fc12359e640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fc11c03eb00 con 0x7fc11c042c40 2024-03-23T14:57:08.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.893+0000 7fc12359e640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 -- 0x7fc11c03e670 con 0x7fc11c03d630 2024-03-23T14:57:08.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.893+0000 7fc122d9d640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fc11c0427f0 0x7fc11c040bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:42494/0 (socket says 172.21.15.164:42494) 2024-03-23T14:57:08.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.893+0000 7fc122d9d640 1 -- 172.21.15.164:0/2852445345 learned_addr learned my addr 172.21.15.164:0/2852445345 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-03-23T14:57:08.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.893+0000 7fc12159a640 1 -- 172.21.15.164:0/2852445345 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3300819780 0 0) 0x7fc11c0d3fa0 con 0x7fc11c0427f0 2024-03-23T14:57:08.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.893+0000 7fc12159a640 1 -- 172.21.15.164:0/2852445345 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc10c003530 con 0x7fc11c0427f0 2024-03-23T14:57:08.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.893+0000 7f80deffd640 1 -- 172.21.15.164:0/3167312462 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f80c8003530 con 0x7f80e00728d0 2024-03-23T14:57:08.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.893+0000 7f80deffd640 1 -- 172.21.15.164:0/3167312462 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2959306328 0 0) 0x7f80c8003530 con 0x7f80e00728d0 2024-03-23T14:57:08.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.893+0000 7f80deffd640 1 -- 172.21.15.164:0/3167312462 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f80e006cf30 msgr2=0x7f80e00b25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:08.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.893+0000 7f80deffd640 1 --2- 172.21.15.164:0/3167312462 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f80e006cf30 0x7f80e00b25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:08.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.893+0000 7f80deffd640 1 -- 172.21.15.164:0/3167312462 >> v2:172.21.15.173:3300/0 conn(0x7f80e00b45e0 msgr2=0x7f80e0072390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:08.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.893+0000 7f80deffd640 1 --2- 172.21.15.164:0/3167312462 >> v2:172.21.15.173:3300/0 conn(0x7f80e00b45e0 0x7f80e0072390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:08.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.893+0000 7f80deffd640 1 -- 172.21.15.164:0/3167312462 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80e00b0c50 con 0x7f80e00728d0 2024-03-23T14:57:08.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.893+0000 7f80deffd640 1 -- 172.21.15.164:0/3167312462 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2526390548 0 0) 0x7f80d0001680 con 0x7f80e00728d0 2024-03-23T14:57:08.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.893+0000 7f80deffd640 1 -- 172.21.15.164:0/3167312462 >> v1:172.21.15.164:6789/0 conn(0x7f80e00728d0 legacy=0x7f80e00b3110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:08.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.893+0000 7f80deffd640 1 --2- 172.21.15.164:0/3167312462 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f80c8003980 0x7f80c8003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:08.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.893+0000 7f80deffd640 1 --2- 172.21.15.164:0/3167312462 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f80c80047d0 0x7f80c8004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:08.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.893+0000 7f80deffd640 1 --2- 172.21.15.164:0/3167312462 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f80c80052b0 0x7f80c800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:08.910 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.893+0000 7f80deffd640 1 -- 172.21.15.164:0/3167312462 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f80e00b4a70 con 0x7f80c80052b0 2024-03-23T14:57:08.910 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.893+0000 7f80deffd640 1 -- 172.21.15.164:0/3167312462 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f80e00b0970 con 0x7f80c80047d0 2024-03-23T14:57:08.910 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.893+0000 7f80deffd640 1 -- 172.21.15.164:0/3167312462 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f80c800c210 con 0x7f80c8003980 2024-03-23T14:57:08.910 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.893+0000 7f80dffff640 1 --2- 172.21.15.164:0/3167312462 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f80e006cf30 0x7f80e00b25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T14:57:08.910 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.893+0000 7f80df7fe640 1 --2- 172.21.15.164:0/3167312462 >> v2:172.21.15.173:3300/0 conn(0x7f80e00b45e0 0x7f80e0072390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T14:57:08.910 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.893+0000 7f80e4d30640 1 --2- 172.21.15.164:0/3167312462 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f80c80052b0 0x7f80c800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:08.910 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.893+0000 7f2dedd9b640 1 --2- 172.21.15.164:0/2731315284 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2dd80069e0 0x7f2dd8006e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:08.911 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.893+0000 7f2deed9d640 1 --2- 172.21.15.164:0/2731315284 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2dd80073a0 0x7f2dd800dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:08.911 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.893+0000 7f2dee59c640 1 --2- 172.21.15.164:0/2731315284 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2dd8005b90 0x7f2dd8005fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:08.911 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.893+0000 7f80dffff640 1 --2- 172.21.15.164:0/3167312462 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f80c8003980 0x7f80c8003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:08.911 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.893+0000 7f80df7fe640 1 --2- 172.21.15.164:0/3167312462 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f80c80047d0 0x7f80c8004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:08.911 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.893+0000 7fc12159a640 1 -- 172.21.15.164:0/2852445345 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1906371028 0 0) 0x7fc10c003530 con 0x7fc11c0427f0 2024-03-23T14:57:08.912 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.893+0000 7f2dedd9b640 1 -- 172.21.15.164:0/2731315284 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2dd80073a0 msgr2=0x7f2dd800dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:08.912 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.893+0000 7f2dedd9b640 1 --2- 172.21.15.164:0/2731315284 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2dd80073a0 0x7f2dd800dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:08.912 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.893+0000 7f2dedd9b640 1 -- 172.21.15.164:0/2731315284 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2dd8005b90 msgr2=0x7f2dd8005fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:08.912 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.893+0000 7f2dedd9b640 1 --2- 172.21.15.164:0/2731315284 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2dd8005b90 0x7f2dd8005fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:08.912 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.893+0000 7f2dedd9b640 1 -- 172.21.15.164:0/2731315284 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2df00adad0 con 0x7f2dd80069e0 2024-03-23T14:57:08.913 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.893+0000 7f15f3fff640 1 -- 172.21.15.164:0/1964623396 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f15f40b0420 msgr2=0x7f15f414ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:08.913 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.893+0000 7f15f3fff640 1 --2- 172.21.15.164:0/1964623396 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f15f40b0420 0x7f15f414ea70 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f15e80097f0 tx=0x7f15e801d450 comp rx=0 tx=0).stop 2024-03-23T14:57:08.913 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.893+0000 7f15f3fff640 1 -- 172.21.15.164:0/1964623396 shutdown_connections 2024-03-23T14:57:08.913 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.893+0000 7f15f3fff640 1 --2- 172.21.15.164:0/1964623396 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f15f40b0420 0x7f15f414ea70 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:08.913 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.893+0000 7f15f3fff640 1 -- 172.21.15.164:0/1964623396 >> 172.21.15.164:0/1964623396 conn(0x7f15f40b8d10 msgr2=0x7f15f40b5210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:08.913 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.889+0000 7f7a92ffd640 1 -- 172.21.15.164:0/392402440 >> v2:172.21.15.173:3300/0 conn(0x7f7a940729b0 msgr2=0x7f7a9413e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:08.913 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.889+0000 7f7a92ffd640 1 --2- 172.21.15.164:0/392402440 >> v2:172.21.15.173:3300/0 conn(0x7f7a940729b0 0x7f7a9413e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:08.913 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.889+0000 7f7a92ffd640 1 -- 172.21.15.164:0/392402440 >> v1:172.21.15.164:6789/0 conn(0x7f7a940b6650 legacy=0x7f7a94070f90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-23T14:57:08.913 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.889+0000 7f7a92ffd640 1 -- 172.21.15.164:0/392402440 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a940b58a0 con 0x7f7a94070000 2024-03-23T14:57:08.914 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.893+0000 7fc12159a640 1 -- 172.21.15.164:0/2852445345 >> v2:172.21.15.173:3300/0 conn(0x7fc11c03d630 msgr2=0x7fc11c0422b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-23T14:57:08.914 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.893+0000 7fc12159a640 1 --2- 172.21.15.164:0/2852445345 >> v2:172.21.15.173:3300/0 conn(0x7fc11c03d630 0x7fc11c0422b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:08.914 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.893+0000 7f48d3fff640 1 --2- 172.21.15.164:0/2309485625 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f48d4149000 0x7f48d414f3b0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f48d40b67e0 tx=0x7f48c401c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15e979b84ec6f0f6 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:08.915 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.893+0000 7fc12159a640 1 -- 172.21.15.164:0/2852445345 >> v2:172.21.15.164:3300/0 conn(0x7fc11c042c40 msgr2=0x7fc11c0d47b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-23T14:57:08.915 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.893+0000 7fc12159a640 1 --2- 172.21.15.164:0/2852445345 >> v2:172.21.15.164:3300/0 conn(0x7fc11c042c40 0x7fc11c0d47b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:08.915 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.893+0000 7fc12159a640 1 -- 172.21.15.164:0/2852445345 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc11c03d1c0 con 0x7fc11c0427f0 2024-03-23T14:57:08.915 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.893+0000 7f15f3fff640 1 -- 172.21.15.164:0/1964623396 shutdown_connections 2024-03-23T14:57:08.915 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.893+0000 7f15f3fff640 1 -- 172.21.15.164:0/1964623396 wait complete. 2024-03-23T14:57:08.916 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.893+0000 7fc12159a640 1 -- 172.21.15.164:0/2852445345 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2526390548 0 0) 0x7fc118001680 con 0x7fc11c0427f0 2024-03-23T14:57:08.916 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.893+0000 7fc12159a640 1 -- 172.21.15.164:0/2852445345 >> v1:172.21.15.164:6789/0 conn(0x7fc11c0427f0 legacy=0x7fc11c040bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:08.916 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.893+0000 7f48d27fc640 1 -- 172.21.15.164:0/2309485625 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48c40ab600 con 0x7f48d4149000 2024-03-23T14:57:08.916 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.893+0000 7f48d27fc640 1 -- 172.21.15.164:0/2309485625 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48c40ab760 con 0x7f48d4149000 2024-03-23T14:57:08.916 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.893+0000 7f48d27fc640 1 -- 172.21.15.164:0/2309485625 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48c40abb30 con 0x7f48d4149000 2024-03-23T14:57:08.917 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.893+0000 7f7a92ffd640 1 --2- 172.21.15.164:0/392402440 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7a94070000 0x7f7a9413de10 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f7a94143fb0 tx=0x7f7a8801c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d34302985bd84e1 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:08.917 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.893+0000 7f3acaffd640 1 --2- 172.21.15.164:0/2563718276 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3acc0b8060 0x7f3acc14abe0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f3ac4009e80 tx=0x7f3ac401ca40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af29b8cdfba39669 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:08.917 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.893+0000 7f7a927fc640 1 -- 172.21.15.164:0/392402440 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a880ab620 con 0x7f7a94070000 2024-03-23T14:57:08.917 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.897+0000 7fc12159a640 1 --2- 172.21.15.164:0/2852445345 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc10c003980 0x7fc10c003d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:08.918 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.893+0000 7f3aca7fc640 1 -- 172.21.15.164:0/2563718276 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ac40a5930 con 0x7f3acc0b8060 2024-03-23T14:57:08.918 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.893+0000 7f3aca7fc640 1 -- 172.21.15.164:0/2563718276 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ac40a5a90 con 0x7f3acc0b8060 2024-03-23T14:57:08.918 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.893+0000 7f3aca7fc640 1 -- 172.21.15.164:0/2563718276 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ac40a5e20 con 0x7f3acc0b8060 2024-03-23T14:57:08.918 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.893+0000 7f48d9b32640 1 -- 172.21.15.164:0/2309485625 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f48d4149000 msgr2=0x7f48d414f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:08.918 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.893+0000 7f48d9b32640 1 --2- 172.21.15.164:0/2309485625 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f48d4149000 0x7f48d414f3b0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f48d40b67e0 tx=0x7f48c401c420 comp rx=0 tx=0).stop 2024-03-23T14:57:08.918 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.897+0000 7fc12159a640 1 --2- 172.21.15.164:0/2852445345 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc10c0047d0 0x7fc10c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:08.919 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.897+0000 7f48d9b32640 1 -- 172.21.15.164:0/2309485625 shutdown_connections 2024-03-23T14:57:08.919 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.897+0000 7f48d9b32640 1 --2- 172.21.15.164:0/2309485625 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f48d4149000 0x7f48d414f3b0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:08.919 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.897+0000 7f48d9b32640 1 -- 172.21.15.164:0/2309485625 >> 172.21.15.164:0/2309485625 conn(0x7f48d40bcc70 msgr2=0x7f48d40bd080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:08.919 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.897+0000 7f48d9b32640 1 -- 172.21.15.164:0/2309485625 shutdown_connections 2024-03-23T14:57:08.919 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.897+0000 7f48d9b32640 1 -- 172.21.15.164:0/2309485625 wait complete. 2024-03-23T14:57:08.920 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.897+0000 7fc12159a640 1 --2- 172.21.15.164:0/2852445345 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc10c005370 0x7fc10c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:08.920 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.897+0000 7fc121d9b640 1 --2- 172.21.15.164:0/2852445345 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc10c0047d0 0x7fc10c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:08.920 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.897+0000 7fc12159a640 1 -- 172.21.15.164:0/2852445345 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc11c03eb00 con 0x7fc10c005370 2024-03-23T14:57:08.920 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.897+0000 7fc12159a640 1 -- 172.21.15.164:0/2852445345 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fc11c03e670 con 0x7fc10c0047d0 2024-03-23T14:57:08.920 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.897+0000 7f15f3fff640 1 Processor -- start 2024-03-23T14:57:08.920 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.897+0000 7f15f3fff640 1 -- start start 2024-03-23T14:57:08.921 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.897+0000 7fc12259c640 1 --2- 172.21.15.164:0/2852445345 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc10c003980 0x7fc10c003d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:08.921 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.893+0000 7f7a927fc640 1 -- 172.21.15.164:0/392402440 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a880ab780 con 0x7f7a94070000 2024-03-23T14:57:08.921 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.893+0000 7f7a927fc640 1 -- 172.21.15.164:0/392402440 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a880abb10 con 0x7f7a94070000 2024-03-23T14:57:08.921 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.897+0000 7f7a98a67640 1 -- 172.21.15.164:0/392402440 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7a94070000 msgr2=0x7f7a9413de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:08.921 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.897+0000 7f7a98a67640 1 --2- 172.21.15.164:0/392402440 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7a94070000 0x7f7a9413de10 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f7a94143fb0 tx=0x7f7a8801c420 comp rx=0 tx=0).stop 2024-03-23T14:57:08.921 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.897+0000 7f7a98a67640 1 -- 172.21.15.164:0/392402440 shutdown_connections 2024-03-23T14:57:08.922 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.897+0000 7f7a98a67640 1 --2- 172.21.15.164:0/392402440 >> v2:172.21.15.173:3300/0 conn(0x7f7a940729b0 0x7f7a9413e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:08.922 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.897+0000 7f7a98a67640 1 --2- 172.21.15.164:0/392402440 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7a94070000 0x7f7a9413de10 secure :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f7a94143fb0 tx=0x7f7a8801c420 comp rx=0 tx=0).stop 2024-03-23T14:57:08.922 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.897+0000 7f7a98a67640 1 -- 172.21.15.164:0/392402440 >> 172.21.15.164:0/392402440 conn(0x7f7a9406b1a0 msgr2=0x7f7a9406b5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:08.922 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.897+0000 7fc12159a640 1 -- 172.21.15.164:0/2852445345 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fc10c00c330 con 0x7fc10c003980 2024-03-23T14:57:08.922 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.897+0000 7f3ad2020640 1 -- 172.21.15.164:0/2563718276 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3acc0b8060 msgr2=0x7f3acc14abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:08.922 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.897+0000 7f3ad2020640 1 --2- 172.21.15.164:0/2563718276 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3acc0b8060 0x7f3acc14abe0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f3ac4009e80 tx=0x7f3ac401ca40 comp rx=0 tx=0).stop 2024-03-23T14:57:08.923 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.897+0000 7fc122d9d640 1 --2- 172.21.15.164:0/2852445345 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc10c005370 0x7fc10c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:08.923 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.897+0000 7f3ad2020640 1 -- 172.21.15.164:0/2563718276 shutdown_connections 2024-03-23T14:57:08.923 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.897+0000 7f3ad2020640 1 --2- 172.21.15.164:0/2563718276 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3acc0b8060 0x7f3acc14abe0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:08.923 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.897+0000 7f3ad2020640 1 -- 172.21.15.164:0/2563718276 >> 172.21.15.164:0/2563718276 conn(0x7f3acc0b9720 msgr2=0x7f3acc0b9b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:08.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.897+0000 7f7a98a67640 1 -- 172.21.15.164:0/392402440 shutdown_connections 2024-03-23T14:57:08.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.897+0000 7f7a98a67640 1 -- 172.21.15.164:0/392402440 wait complete. 2024-03-23T14:57:08.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.893+0000 7fb12affd640 1 --2- 172.21.15.164:0/3280889938 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb12c0bc580 0x7fb12c140f40 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb11c0097c0 tx=0x7fb11c01c270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f2fb4e8ed5de7fe server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:08.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.893+0000 7f80e4d30640 1 -- 172.21.15.164:0/3167312462 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f80c8003980 msgr2=0x7f80c8003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:08.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.893+0000 7f80e4d30640 1 --2- 172.21.15.164:0/3167312462 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f80c8003980 0x7f80c8003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:08.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.893+0000 7f80e4d30640 1 -- 172.21.15.164:0/3167312462 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f80c80047d0 msgr2=0x7f80c8004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:08.925 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.893+0000 7f80e4d30640 1 --2- 172.21.15.164:0/3167312462 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f80c80047d0 0x7f80c8004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:08.925 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.893+0000 7f80e4d30640 1 -- 172.21.15.164:0/3167312462 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80e00b0c50 con 0x7f80c80052b0 2024-03-23T14:57:08.925 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.897+0000 7fb12a7fc640 1 -- 172.21.15.164:0/3280889938 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb120001680 con 0x7fb12c0bc580 2024-03-23T14:57:08.925 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.897+0000 7f3ad2020640 1 -- 172.21.15.164:0/2563718276 shutdown_connections 2024-03-23T14:57:08.926 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.897+0000 7fc121d9b640 1 -- 172.21.15.164:0/2852445345 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc10c003980 msgr2=0x7fc10c003d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:08.926 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.897+0000 7fc121d9b640 1 --2- 172.21.15.164:0/2852445345 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc10c003980 0x7fc10c003d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:08.926 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.897+0000 7fc121d9b640 1 -- 172.21.15.164:0/2852445345 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc10c005370 msgr2=0x7fc10c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:08.926 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.897+0000 7f48d9b32640 1 Processor -- start 2024-03-23T14:57:08.926 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.897+0000 7f80e4d30640 1 --2- 172.21.15.164:0/3167312462 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f80c80052b0 0x7f80c800b990 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f80d000c260 tx=0x7f80d001f430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d48ef9381adf1cb5 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:08.927 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.897+0000 7f15f3fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f15f40b0420 0x7f15f4137890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:08.927 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.897+0000 7f15f3fff640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f15f4143c10 0x7f15f4070740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:08.927 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.897+0000 7f15f3fff640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f15f41366b0 0x7f15f4070c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:08.927 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.897+0000 7f15f3fff640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f15f413dfd0 con 0x7f15f40b0420 2024-03-23T14:57:08.927 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.897+0000 7f15f3fff640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f15f41384e0 con 0x7f15f41366b0 2024-03-23T14:57:08.927 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.897+0000 7f15f3fff640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f15f413e520 con 0x7f15f4143c10 2024-03-23T14:57:08.927 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.897+0000 7f15f2ffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f15f40b0420 0x7f15f4137890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:08.928 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.897+0000 7f15f2ffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f15f40b0420 0x7f15f4137890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:59246/0 (socket says 172.21.15.164:59246) 2024-03-23T14:57:08.928 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.897+0000 7f15f2ffd640 1 -- 172.21.15.164:0/1701667169 learned_addr learned my addr 172.21.15.164:0/1701667169 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:08.928 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.897+0000 7f15f37fe640 1 --2- 172.21.15.164:0/1701667169 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f15f41366b0 0x7f15f4070c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:08.928 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.897+0000 7f2dedd9b640 1 --2- 172.21.15.164:0/2731315284 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2dd80069e0 0x7f2dd8006e60 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f2de400d300 tx=0x7f2de401d1e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=85ba1a51e345a4c5 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:08.928 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.897+0000 7f80deffd640 1 -- 172.21.15.164:0/3167312462 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80d0005ae0 con 0x7f80c80052b0 2024-03-23T14:57:08.929 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.897+0000 7f80deffd640 1 -- 172.21.15.164:0/3167312462 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80d00ab070 con 0x7f80c80052b0 2024-03-23T14:57:08.929 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.897+0000 7f80deffd640 1 -- 172.21.15.164:0/3167312462 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80d0007070 con 0x7f80c80052b0 2024-03-23T14:57:08.929 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.897+0000 7f3ad2020640 1 -- 172.21.15.164:0/2563718276 wait complete. 2024-03-23T14:57:08.929 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.897+0000 7f2ded59a640 1 -- 172.21.15.164:0/2731315284 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de40a6920 con 0x7f2dd80069e0 2024-03-23T14:57:08.929 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.897+0000 7f2ded59a640 1 -- 172.21.15.164:0/2731315284 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2de4009040 con 0x7f2dd80069e0 2024-03-23T14:57:08.929 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.897+0000 7f48d9b32640 1 -- start start 2024-03-23T14:57:08.930 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.897+0000 7f48d9b32640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f48d40b3ac0 0x7f48d413fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:08.930 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.897+0000 7f2ded59a640 1 -- 172.21.15.164:0/2731315284 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de40a6d20 con 0x7f2dd80069e0 2024-03-23T14:57:08.930 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.897+0000 7f48d9b32640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f48d4149000 0x7f48d4146ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:08.930 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.897+0000 7f48d9b32640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f48d414efd0 0x7f48d41417e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:08.930 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.897+0000 7f48d9b32640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f48d4143bc0 con 0x7f48d414efd0 2024-03-23T14:57:08.930 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.897+0000 7f48d9b32640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f48d4144420 con 0x7f48d40b3ac0 2024-03-23T14:57:08.930 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.897+0000 7f48d9b32640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f48d41463f0 con 0x7f48d4149000 2024-03-23T14:57:08.931 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.897+0000 7f15f2ffd640 1 -- 172.21.15.164:0/1701667169 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f15f4143c10 msgr2=0x7f15f4070740 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-23T14:57:08.931 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.897+0000 7f15f2ffd640 1 --2- 172.21.15.164:0/1701667169 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f15f4143c10 0x7f15f4070740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:08.931 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.897+0000 7f15f2ffd640 1 -- 172.21.15.164:0/1701667169 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f15f41366b0 msgr2=0x7f15f4070c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:08.931 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.897+0000 7f15f2ffd640 1 --2- 172.21.15.164:0/1701667169 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f15f41366b0 0x7f15f4070c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:08.931 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.897+0000 7f15f2ffd640 1 -- 172.21.15.164:0/1701667169 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15e8009420 con 0x7f15f40b0420 2024-03-23T14:57:08.931 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.897+0000 7f7a98a67640 1 Processor -- start 2024-03-23T14:57:08.932 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.897+0000 7f48d3fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f48d414efd0 0x7f48d41417e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:08.932 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.897+0000 7f48d3fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f48d414efd0 0x7f48d41417e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:59250/0 (socket says 172.21.15.164:59250) 2024-03-23T14:57:08.932 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.897+0000 7f48d3fff640 1 -- 172.21.15.164:0/3317854406 learned_addr learned my addr 172.21.15.164:0/3317854406 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:08.932 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.897+0000 7f48d2ffd640 1 --2- 172.21.15.164:0/3317854406 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f48d4149000 0x7f48d4146ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:08.933 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.897+0000 7f3ad2020640 1 Processor -- start 2024-03-23T14:57:08.933 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.897+0000 7f3ad2020640 1 -- start start 2024-03-23T14:57:08.933 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.897+0000 7f48d3fff640 1 -- 172.21.15.164:0/3317854406 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f48d4149000 msgr2=0x7f48d4146ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:08.933 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.897+0000 7f48d3fff640 1 --2- 172.21.15.164:0/3317854406 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f48d4149000 0x7f48d4146ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:08.933 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.897+0000 7f48d3fff640 1 -- 172.21.15.164:0/3317854406 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f48d40b3ac0 msgr2=0x7f48d413fc40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-23T14:57:08.933 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.897+0000 7f48d3fff640 1 --2- 172.21.15.164:0/3317854406 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f48d40b3ac0 0x7f48d413fc40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:08.933 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.897+0000 7f15f2ffd640 1 --2- 172.21.15.164:0/1701667169 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f15f40b0420 0x7f15f4137890 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f15e400c430 tx=0x7f15e400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:08.934 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.897+0000 7f48d3fff640 1 -- 172.21.15.164:0/3317854406 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48c4009420 con 0x7f48d414efd0 2024-03-23T14:57:08.934 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.897+0000 7f2df4c1e640 1 -- 172.21.15.164:0/2731315284 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2dd80069e0 msgr2=0x7f2dd8006e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:08.934 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.897+0000 7f2df4c1e640 1 --2- 172.21.15.164:0/2731315284 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2dd80069e0 0x7f2dd8006e60 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f2de400d300 tx=0x7f2de401d1e0 comp rx=0 tx=0).stop 2024-03-23T14:57:08.934 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.897+0000 7f2deed9d640 1 -- 172.21.15.164:0/2731315284 reap_dead start 2024-03-23T14:57:08.935 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.897+0000 7fb12a7fc640 1 -- 172.21.15.164:0/3280889938 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb11c0a7050 con 0x7fb12c0bc580 2024-03-23T14:57:08.935 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.897+0000 7fb12a7fc640 1 -- 172.21.15.164:0/3280889938 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb11c0ab620 con 0x7fb12c0bc580 2024-03-23T14:57:08.935 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.897+0000 7fb132229640 1 -- 172.21.15.164:0/3280889938 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb12c0bc580 msgr2=0x7fb12c140f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:08.935 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.897+0000 7fb132229640 1 --2- 172.21.15.164:0/3280889938 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb12c0bc580 0x7fb12c140f40 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb11c0097c0 tx=0x7fb11c01c270 comp rx=0 tx=0).stop 2024-03-23T14:57:08.935 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.897+0000 7f80e6793640 1 -- 172.21.15.164:0/3167312462 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f80c80052b0 msgr2=0x7f80c800b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:08.935 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.897+0000 7f80e6793640 1 --2- 172.21.15.164:0/3167312462 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f80c80052b0 0x7f80c800b990 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f80d000c260 tx=0x7f80d001f430 comp rx=0 tx=0).stop 2024-03-23T14:57:08.936 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.897+0000 7f80e4d30640 1 -- 172.21.15.164:0/3167312462 reap_dead start 2024-03-23T14:57:08.936 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.897+0000 7f2df4c1e640 1 -- 172.21.15.164:0/2731315284 shutdown_connections 2024-03-23T14:57:08.936 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.897+0000 7f2df4c1e640 1 -- 172.21.15.164:0/2731315284 >> 172.21.15.164:0/2731315284 conn(0x7f2df00b8ad0 msgr2=0x7f2df00b8ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:08.936 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.897+0000 7f48d3fff640 1 --2- 172.21.15.164:0/3317854406 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f48d414efd0 0x7f48d41417e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f48c40ad040 tx=0x7f48c401ac60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:08.936 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.897+0000 7f80e6793640 1 -- 172.21.15.164:0/3167312462 shutdown_connections 2024-03-23T14:57:08.936 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.897+0000 7f80e6793640 1 -- 172.21.15.164:0/3167312462 >> 172.21.15.164:0/3167312462 conn(0x7f80e00b8d10 msgr2=0x7f80e00b5210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:08.937 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.897+0000 7fb132229640 1 -- 172.21.15.164:0/3280889938 shutdown_connections 2024-03-23T14:57:08.937 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.897+0000 7fb132229640 1 --2- 172.21.15.164:0/3280889938 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb12c09aef0 0x7fb12c141480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:08.937 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.897+0000 7fb132229640 1 --2- 172.21.15.164:0/3280889938 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb12c0bc580 0x7fb12c140f40 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:08.937 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.897+0000 7fb132229640 1 -- 172.21.15.164:0/3280889938 >> 172.21.15.164:0/3280889938 conn(0x7fb12c09ec30 msgr2=0x7fb12c09f040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:08.937 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.897+0000 7f2df4c1e640 1 -- 172.21.15.164:0/2731315284 shutdown_connections 2024-03-23T14:57:08.938 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.897+0000 7f2df4c1e640 1 -- 172.21.15.164:0/2731315284 wait complete. 2024-03-23T14:57:08.938 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.897+0000 7f7a98a67640 1 -- start start 2024-03-23T14:57:08.938 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.901+0000 7f80e6793640 1 -- 172.21.15.164:0/3167312462 shutdown_connections 2024-03-23T14:57:08.939 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.901+0000 7f48bbfff640 1 -- 172.21.15.164:0/3317854406 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48c40bf610 con 0x7f48d414efd0 2024-03-23T14:57:08.939 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.901+0000 7f48bbfff640 1 -- 172.21.15.164:0/3317854406 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48c40bf770 con 0x7f48d414efd0 2024-03-23T14:57:08.939 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.901+0000 7f48bbfff640 1 -- 172.21.15.164:0/3317854406 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48c40bfae0 con 0x7f48d414efd0 2024-03-23T14:57:08.939 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.901+0000 7f15db7fe640 1 -- 172.21.15.164:0/1701667169 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15e40052d0 con 0x7f15f40b0420 2024-03-23T14:57:08.939 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.901+0000 7f15f3fff640 1 -- 172.21.15.164:0/1701667169 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15f4135af0 con 0x7f15f40b0420 2024-03-23T14:57:08.939 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.901+0000 7f15f3fff640 1 -- 172.21.15.164:0/1701667169 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15f413e240 con 0x7f15f40b0420 2024-03-23T14:57:08.940 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.901+0000 7f3ad2020640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3acc06beb0 0x7f3acc131c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:08.940 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.901+0000 7f3ad2020640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3acc06cf20 0x7f3acc145450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:08.940 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.901+0000 7f3ad2020640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3acc09ee10 0x7f3acc146fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:08.940 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.901+0000 7f3ad2020640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f3acc09e0e0 con 0x7f3acc06beb0 2024-03-23T14:57:08.940 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.901+0000 7f3ad2020640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f3acc146370 con 0x7f3acc06cf20 2024-03-23T14:57:08.940 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.901+0000 7f3ad2020640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f3acc148200 con 0x7f3acc09ee10 2024-03-23T14:57:08.940 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.901+0000 7f3acb7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3acc06beb0 0x7f3acc131c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:08.941 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.901+0000 7f3acb7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3acc06beb0 0x7f3acc131c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:59260/0 (socket says 172.21.15.164:59260) 2024-03-23T14:57:08.941 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.901+0000 7f3acb7fe640 1 -- 172.21.15.164:0/4004176476 learned_addr learned my addr 172.21.15.164:0/4004176476 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:08.941 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.901+0000 7f3acbfff640 1 --2- 172.21.15.164:0/4004176476 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3acc09ee10 0x7f3acc146fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:08.941 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.901+0000 7f3acaffd640 1 --2- 172.21.15.164:0/4004176476 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3acc06cf20 0x7f3acc145450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:08.941 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.901+0000 7f80e6793640 1 -- 172.21.15.164:0/3167312462 wait complete. 2024-03-23T14:57:08.941 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.901+0000 7f48d9b32640 1 -- 172.21.15.164:0/3317854406 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48d4143dd0 con 0x7f48d414efd0 2024-03-23T14:57:08.941 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.901+0000 7f48d9b32640 1 -- 172.21.15.164:0/3317854406 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48d41479a0 con 0x7f48d414efd0 2024-03-23T14:57:08.942 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.901+0000 7f15db7fe640 1 -- 172.21.15.164:0/1701667169 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15e4005450 con 0x7f15f40b0420 2024-03-23T14:57:08.942 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.901+0000 7f7a98a67640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7a940729b0 0x7f7a941429f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:08.942 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.901+0000 7f7a98a67640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7a940b6650 0x7f7a9413e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:08.942 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.901+0000 7f3acbfff640 1 -- 172.21.15.164:0/4004176476 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3acc06cf20 msgr2=0x7f3acc145450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:08.942 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.901+0000 7f3acbfff640 1 --2- 172.21.15.164:0/4004176476 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3acc06cf20 0x7f3acc145450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:08.942 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.901+0000 7f3acbfff640 1 -- 172.21.15.164:0/4004176476 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3acc06beb0 msgr2=0x7f3acc131c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:08.943 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.901+0000 7f3acbfff640 1 --2- 172.21.15.164:0/4004176476 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3acc06beb0 0x7f3acc131c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:08.943 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.901+0000 7f7a98a67640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7a94146ff0 0x7f7a941379f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:08.943 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.901+0000 7f7a98a67640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f7a94146940 con 0x7f7a940b6650 2024-03-23T14:57:08.943 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.901+0000 7f7a98a67640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f7a9413f4e0 con 0x7f7a940729b0 2024-03-23T14:57:08.943 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.901+0000 7f7a98a67640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f7a94146660 con 0x7f7a94146ff0 2024-03-23T14:57:08.943 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.901+0000 7f3acbfff640 1 -- 172.21.15.164:0/4004176476 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ac4009b30 con 0x7f3acc09ee10 2024-03-23T14:57:08.944 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.901+0000 7f2df4c1e640 1 Processor -- start 2024-03-23T14:57:08.944 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.901+0000 7fb132229640 1 -- 172.21.15.164:0/3280889938 shutdown_connections 2024-03-23T14:57:08.944 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.901+0000 7f7a93fff640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7a94146ff0 0x7f7a941379f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:08.944 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.901+0000 7f7a93fff640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7a94146ff0 0x7f7a941379f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.164:60702/0 (socket says 172.21.15.164:60702) 2024-03-23T14:57:08.944 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.901+0000 7f7a93fff640 1 -- 172.21.15.164:0/3377315891 learned_addr learned my addr 172.21.15.164:0/3377315891 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:08.945 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.901+0000 7f3acbfff640 1 --2- 172.21.15.164:0/4004176476 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3acc09ee10 0x7f3acc146fc0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f3abc00c390 tx=0x7f3abc00c850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:08.945 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.901+0000 7f7a937fe640 1 --2- 172.21.15.164:0/3377315891 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7a940729b0 0x7f7a941429f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:08.945 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.901+0000 7f7a92ffd640 1 --2- 172.21.15.164:0/3377315891 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7a940b6650 0x7f7a9413e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:08.945 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.901+0000 7f3ab3fff640 1 -- 172.21.15.164:0/4004176476 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3abc0052d0 con 0x7f3acc09ee10 2024-03-23T14:57:08.946 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.901+0000 7f3ab3fff640 1 -- 172.21.15.164:0/4004176476 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3abc005450 con 0x7f3acc09ee10 2024-03-23T14:57:08.946 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.901+0000 7f3ab3fff640 1 -- 172.21.15.164:0/4004176476 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3abc007260 con 0x7f3acc09ee10 2024-03-23T14:57:08.946 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.901+0000 7f48d9b32640 1 -- 172.21.15.164:0/3317854406 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48d40690a0 con 0x7f48d414efd0 2024-03-23T14:57:08.946 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.901+0000 7f3ad2020640 1 -- 172.21.15.164:0/4004176476 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3acc146850 con 0x7f3acc09ee10 2024-03-23T14:57:08.946 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.901+0000 7f3ad2020640 1 -- 172.21.15.164:0/4004176476 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f3acc09dd70 con 0x7f3acc09ee10 2024-03-23T14:57:08.947 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.901+0000 7f80e6793640 1 Processor -- start 2024-03-23T14:57:08.947 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.901+0000 7f80e6793640 1 -- start start 2024-03-23T14:57:08.947 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.901+0000 7f7a93fff640 1 -- 172.21.15.164:0/3377315891 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7a940729b0 msgr2=0x7f7a941429f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:08.947 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.901+0000 7f7a93fff640 1 --2- 172.21.15.164:0/3377315891 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7a940729b0 0x7f7a941429f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:08.948 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.901+0000 7f7a93fff640 1 -- 172.21.15.164:0/3377315891 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7a940b6650 msgr2=0x7f7a9413e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:08.948 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.901+0000 7f7a93fff640 1 --2- 172.21.15.164:0/3377315891 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7a940b6650 0x7f7a9413e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:08.948 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.901+0000 7f7a93fff640 1 -- 172.21.15.164:0/3377315891 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a880094d0 con 0x7f7a94146ff0 2024-03-23T14:57:08.948 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.901+0000 7f7a92ffd640 1 --2- 172.21.15.164:0/3377315891 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7a940b6650 0x7f7a9413e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T14:57:08.948 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.901+0000 7f7a93fff640 1 --2- 172.21.15.164:0/3377315891 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7a94146ff0 0x7f7a941379f0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f7a8c007d80 tx=0x7f7a8c00e4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:08.948 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.901+0000 7fb132229640 1 -- 172.21.15.164:0/3280889938 wait complete. 2024-03-23T14:57:08.948 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.901+0000 7fb132229640 1 Processor -- start 2024-03-23T14:57:08.949 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.901+0000 7f80e6793640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f80c80052b0 0x7f80e0147750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:08.949 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.901+0000 7f80e6793640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f80e0147c90 0x7f80e01e3c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:08.949 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.901+0000 7f80e6793640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f80e01e41a0 0x7f80e00aea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:08.949 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.901+0000 7f80e6793640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f80e0148fd0 con 0x7f80e0147c90 2024-03-23T14:57:08.949 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.901+0000 7f80e6793640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f80e01481c0 con 0x7f80e01e41a0 2024-03-23T14:57:08.949 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.901+0000 7f80e6793640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f80e0146800 con 0x7f80c80052b0 2024-03-23T14:57:08.949 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.901+0000 7f15db7fe640 1 -- 172.21.15.164:0/1701667169 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15e4003840 con 0x7f15f40b0420 2024-03-23T14:57:08.949 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.901+0000 7f15f3fff640 1 -- 172.21.15.164:0/1701667169 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15d40052a0 con 0x7f15f40b0420 2024-03-23T14:57:08.950 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.901+0000 7f80e4d30640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f80e01e41a0 0x7f80e00aea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:08.950 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.901+0000 7f80e4d30640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f80e01e41a0 0x7f80e00aea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.164:39664/0 (socket says 172.21.15.164:39664) 2024-03-23T14:57:08.950 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.901+0000 7f80e4d30640 1 -- 172.21.15.164:0/2164178415 learned_addr learned my addr 172.21.15.164:0/2164178415 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:08.950 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.901+0000 7f7a7bfff640 1 -- 172.21.15.164:0/3377315891 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a8c014420 con 0x7f7a94146ff0 2024-03-23T14:57:08.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.901+0000 7fb132229640 1 -- start start 2024-03-23T14:57:08.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.901+0000 7fb132229640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb12c09aef0 0x7fb12c146b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:08.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.901+0000 7fb132229640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb12c0bab20 0x7fb12c13e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:08.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.901+0000 7fb132229640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb12c0bc580 0x7fb12c13e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:08.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.901+0000 7fb132229640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb12c1404d0 con 0x7fb12c0bc580 2024-03-23T14:57:08.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.901+0000 7fb132229640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fb12c147eb0 con 0x7fb12c09aef0 2024-03-23T14:57:08.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.901+0000 7fb132229640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fb12c13d900 con 0x7fb12c0bab20 2024-03-23T14:57:08.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.901+0000 7f80e4d30640 1 -- 172.21.15.164:0/2164178415 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f80c80052b0 msgr2=0x7f80e0147750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:08.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.901+0000 7f80e4d30640 1 --2- 172.21.15.164:0/2164178415 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f80c80052b0 0x7f80e0147750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:08.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.901+0000 7f80e4d30640 1 -- 172.21.15.164:0/2164178415 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f80e0147c90 msgr2=0x7f80e01e3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:08.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.901+0000 7f80e4d30640 1 --2- 172.21.15.164:0/2164178415 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f80e0147c90 0x7f80e01e3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:08.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.901+0000 7f80e4d30640 1 -- 172.21.15.164:0/2164178415 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80e01e4620 con 0x7f80e01e41a0 2024-03-23T14:57:08.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.901+0000 7f7a7bfff640 1 -- 172.21.15.164:0/3377315891 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a8c014580 con 0x7f7a94146ff0 2024-03-23T14:57:08.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.901+0000 7f7a7bfff640 1 -- 172.21.15.164:0/3377315891 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a8c014930 con 0x7f7a94146ff0 2024-03-23T14:57:08.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.901+0000 7f7a98a67640 1 -- 172.21.15.164:0/3377315891 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a940b0740 con 0x7f7a94146ff0 2024-03-23T14:57:08.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.901+0000 7f7a98a67640 1 -- 172.21.15.164:0/3377315891 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a9413ee60 con 0x7f7a94146ff0 2024-03-23T14:57:08.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.905+0000 7f80e4d30640 1 --2- 172.21.15.164:0/2164178415 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f80e01e41a0 0x7f80e00aea00 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f80d001f260 tx=0x7f80c8005800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:08.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.901+0000 7f2df4c1e640 1 -- start start 2024-03-23T14:57:08.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.905+0000 7f2df4c1e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2dd80069e0 0x7f2df0071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:08.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.905+0000 7f2df4c1e640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2df009b570 0x7f2df01f74b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:08.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.905+0000 7f2df4c1e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2df0157b60 0x7f2df0157fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:08.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.905+0000 7f2df4c1e640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2df0089c70 con 0x7f2df0157b60 2024-03-23T14:57:08.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.905+0000 7f2df4c1e640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f2df0089630 con 0x7f2dd80069e0 2024-03-23T14:57:08.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.905+0000 7f2df4c1e640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f2df009bd60 con 0x7f2df009b570 2024-03-23T14:57:08.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.901+0000 7fb12affd640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb12c0bab20 0x7fb12c13e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:08.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.905+0000 7fb12affd640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb12c0bab20 0x7fb12c13e2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.164:60720/0 (socket says 172.21.15.164:60720) 2024-03-23T14:57:08.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.905+0000 7fb12affd640 1 -- 172.21.15.164:0/1383172781 learned_addr learned my addr 172.21.15.164:0/1383172781 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:08.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.905+0000 7fb12b7fe640 1 --2- 172.21.15.164:0/1383172781 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb12c09aef0 0x7fb12c146b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:08.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.905+0000 7fb12b7fe640 1 -- 172.21.15.164:0/1383172781 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb12c0bab20 msgr2=0x7fb12c13e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:08.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.905+0000 7fb12b7fe640 1 --2- 172.21.15.164:0/1383172781 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb12c0bab20 0x7fb12c13e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:08.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.905+0000 7fb12b7fe640 1 -- 172.21.15.164:0/1383172781 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb12c0bc580 msgr2=0x7fb12c13e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:08.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.905+0000 7fb12b7fe640 1 --2- 172.21.15.164:0/1383172781 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb12c0bc580 0x7fb12c13e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:08.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.905+0000 7fb12b7fe640 1 -- 172.21.15.164:0/1383172781 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb11c009420 con 0x7fb12c09aef0 2024-03-23T14:57:08.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.905+0000 7fc121d9b640 1 --2- 172.21.15.164:0/2852445345 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc10c005370 0x7fc10c00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:08.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.905+0000 7fc121d9b640 1 -- 172.21.15.164:0/2852445345 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc10c003530 con 0x7fc10c0047d0 2024-03-23T14:57:08.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.905+0000 7fc122d9d640 1 -- 172.21.15.164:0/2852445345 reap_dead start 2024-03-23T14:57:08.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.905+0000 7f2dedd9b640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2df009b570 0x7f2df01f74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:08.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.905+0000 7f2dedd9b640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2df009b570 0x7f2df01f74b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.164:60736/0 (socket says 172.21.15.164:60736) 2024-03-23T14:57:08.956 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.905+0000 7f2dedd9b640 1 -- 172.21.15.164:0/1346441029 learned_addr learned my addr 172.21.15.164:0/1346441029 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:08.956 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.905+0000 7f3ab3fff640 1 -- 172.21.15.164:0/4004176476 <== mon.2 v2:172.21.15.173:3301/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7f3abc0073c0 con 0x7f3acc09ee10 2024-03-23T14:57:08.956 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.905+0000 7f15db7fe640 1 -- 172.21.15.164:0/1701667169 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7f15e40039a0 con 0x7f15f40b0420 2024-03-23T14:57:08.956 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.905+0000 7fb12b7fe640 1 --2- 172.21.15.164:0/1383172781 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb12c09aef0 0x7fb12c146b30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb120007fd0 tx=0x7fb12000cbc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:08.956 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.905+0000 7fb113fff640 1 -- 172.21.15.164:0/1383172781 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb120005480 con 0x7fb12c09aef0 2024-03-23T14:57:08.957 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.905+0000 7fb113fff640 1 -- 172.21.15.164:0/1383172781 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb12000ce10 con 0x7fb12c09aef0 2024-03-23T14:57:08.957 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.905+0000 7fb113fff640 1 -- 172.21.15.164:0/1383172781 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb12000a860 con 0x7fb12c09aef0 2024-03-23T14:57:08.957 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.905+0000 7f2deed9d640 1 --2- 172.21.15.164:0/1346441029 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2df0157b60 0x7f2df0157fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:08.957 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.905+0000 7f2dedd9b640 1 -- 172.21.15.164:0/1346441029 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2dd80069e0 msgr2=0x7f2df0071380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:08.957 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.905+0000 7f2dedd9b640 1 --2- 172.21.15.164:0/1346441029 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2dd80069e0 0x7f2df0071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:08.957 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.905+0000 7f2dedd9b640 1 -- 172.21.15.164:0/1346441029 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2df0157b60 msgr2=0x7f2df0157fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:08.957 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.905+0000 7f2dedd9b640 1 --2- 172.21.15.164:0/1346441029 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2df0157b60 0x7f2df0157fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:08.957 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.905+0000 7f2dedd9b640 1 -- 172.21.15.164:0/1346441029 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2df009c040 con 0x7f2df009b570 2024-03-23T14:57:08.957 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.905+0000 7f2deed9d640 1 --2- 172.21.15.164:0/1346441029 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2df0157b60 0x7f2df0157fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T14:57:08.958 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.905+0000 7f15db7fe640 1 --2- 172.21.15.164:0/1701667169 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f15e00672f0 0x7f15e00697b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:08.958 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.905+0000 7f7a98a67640 1 -- 172.21.15.164:0/3377315891 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a9402d0d0 con 0x7f7a94146ff0 2024-03-23T14:57:08.958 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.905+0000 7f3ab3fff640 1 --2- 172.21.15.164:0/4004176476 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f3ab80672a0 0x7f3ab8069760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:08.958 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.905+0000 7f3ab3fff640 1 -- 172.21.15.164:0/4004176476 <== mon.2 v2:172.21.15.173:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f3abc0842a0 con 0x7f3acc09ee10 2024-03-23T14:57:08.958 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.905+0000 7f15f27fc640 1 --2- 172.21.15.164:0/1701667169 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f15e00672f0 0x7f15e00697b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:08.959 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.905+0000 7f15f27fc640 1 --2- 172.21.15.164:0/1701667169 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f15e00672f0 0x7f15e00697b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f15ec001570 tx=0x7f15ec001480 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:08.959 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.905+0000 7fb132229640 1 -- 172.21.15.164:0/1383172781 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb12c1471d0 con 0x7fb12c09aef0 2024-03-23T14:57:08.959 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.905+0000 7fb132229640 1 -- 172.21.15.164:0/1383172781 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb12c13dbe0 con 0x7fb12c09aef0 2024-03-23T14:57:08.959 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.905+0000 7f2dedd9b640 1 --2- 172.21.15.164:0/1346441029 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2df009b570 0x7f2df01f74b0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f2de400d300 tx=0x7f2dd8007a10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:08.959 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.905+0000 7f2ddeffd640 1 -- 172.21.15.164:0/1346441029 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de400d070 con 0x7f2df009b570 2024-03-23T14:57:08.959 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.905+0000 7f2ddeffd640 1 -- 172.21.15.164:0/1346441029 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2de40a6920 con 0x7f2df009b570 2024-03-23T14:57:08.959 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.905+0000 7f2ddeffd640 1 -- 172.21.15.164:0/1346441029 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de40a6c90 con 0x7f2df009b570 2024-03-23T14:57:08.960 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.905+0000 7f7a7bfff640 1 -- 172.21.15.164:0/3377315891 <== mon.2 v2:172.21.15.173:3301/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7f7a8c014a90 con 0x7f7a94146ff0 2024-03-23T14:57:08.960 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.905+0000 7fc121d9b640 1 --2- 172.21.15.164:0/2852445345 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc10c0047d0 0x7fc10c004c50 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc1140097e0 tx=0x7fc11401c2b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3472ca3a66ab1d82 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:08.960 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.905+0000 7f3ad2020640 1 -- 172.21.15.164:0/4004176476 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3acc09eb30 con 0x7f3acc09ee10 2024-03-23T14:57:08.960 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.905+0000 7f15db7fe640 1 -- 172.21.15.164:0/1701667169 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f15e4084550 con 0x7f15f40b0420 2024-03-23T14:57:08.961 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.905+0000 7f7a7bfff640 1 --2- 172.21.15.164:0/3377315891 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f7a800672d0 0x7f7a80069790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:08.961 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.905+0000 7f48bbfff640 1 -- 172.21.15.164:0/3317854406 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7f48c40bfc40 con 0x7f48d414efd0 2024-03-23T14:57:08.961 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.905+0000 7f7a7bfff640 1 -- 172.21.15.164:0/3377315891 <== mon.2 v2:172.21.15.173:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f7a8c086220 con 0x7f7a94146ff0 2024-03-23T14:57:08.961 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.905+0000 7f7a937fe640 1 --2- 172.21.15.164:0/3377315891 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f7a800672d0 0x7f7a80069790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:08.962 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.905+0000 7f2df4c1e640 1 -- 172.21.15.164:0/1346441029 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2df0089990 con 0x7f2df009b570 2024-03-23T14:57:08.962 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.905+0000 7f2df4c1e640 1 -- 172.21.15.164:0/1346441029 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f2df009b9f0 con 0x7f2df009b570 2024-03-23T14:57:08.962 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.905+0000 7f2df4c1e640 1 -- 172.21.15.164:0/1346441029 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2df009bd60 con 0x7f2df009b570 2024-03-23T14:57:08.962 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.905+0000 7f80dcff9640 1 -- 172.21.15.164:0/2164178415 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80d00b7070 con 0x7f80e01e41a0 2024-03-23T14:57:08.963 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.909+0000 7f7a937fe640 1 --2- 172.21.15.164:0/3377315891 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f7a800672d0 0x7f7a80069790 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7a84006fd0 tx=0x7f7a84009060 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:08.963 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.909+0000 7f48bbfff640 1 --2- 172.21.15.164:0/3317854406 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f48c00672a0 0x7f48c0069760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:08.963 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.909+0000 7f48bbfff640 1 -- 172.21.15.164:0/3317854406 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f48c411ffd0 con 0x7f48d414efd0 2024-03-23T14:57:08.963 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.909+0000 7f48d37fe640 1 --2- 172.21.15.164:0/3317854406 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f48c00672a0 0x7f48c0069760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:08.964 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.905+0000 7fc12159a640 1 -- 172.21.15.164:0/2852445345 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1140ab600 con 0x7fc10c0047d0 2024-03-23T14:57:08.964 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.905+0000 7fc12159a640 1 -- 172.21.15.164:0/2852445345 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1140a7050 con 0x7fc10c0047d0 2024-03-23T14:57:08.964 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.905+0000 7fc12159a640 1 -- 172.21.15.164:0/2852445345 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1140ab9c0 con 0x7fc10c0047d0 2024-03-23T14:57:08.964 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.909+0000 7fc12359e640 1 -- 172.21.15.164:0/2852445345 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc10c0047d0 msgr2=0x7fc10c004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:08.964 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.909+0000 7fc12359e640 1 --2- 172.21.15.164:0/2852445345 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc10c0047d0 0x7fc10c004c50 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc1140097e0 tx=0x7fc11401c2b0 comp rx=0 tx=0).stop 2024-03-23T14:57:08.964 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.909+0000 7fc12359e640 1 -- 172.21.15.164:0/2852445345 shutdown_connections 2024-03-23T14:57:08.964 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.909+0000 7fc12359e640 1 --2- 172.21.15.164:0/2852445345 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc10c0047d0 0x7fc10c004c50 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:08.964 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.909+0000 7fc12359e640 1 -- 172.21.15.164:0/2852445345 >> 172.21.15.164:0/2852445345 conn(0x7fc11c006e90 msgr2=0x7fc11c0072a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:08.964 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.909+0000 7f48d37fe640 1 --2- 172.21.15.164:0/3317854406 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f48c00672a0 0x7f48c0069760 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f48c8000f70 tx=0x7f48c8007a10 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:08.965 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.909+0000 7fc12359e640 1 -- 172.21.15.164:0/2852445345 shutdown_connections 2024-03-23T14:57:08.965 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.909+0000 7f80e6793640 1 -- 172.21.15.164:0/2164178415 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80e01491b0 con 0x7f80e01e41a0 2024-03-23T14:57:08.965 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.909+0000 7f80e6793640 1 -- 172.21.15.164:0/2164178415 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80e0146d60 con 0x7f80e01e41a0 2024-03-23T14:57:08.965 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.909+0000 7fb132229640 1 -- 172.21.15.164:0/1383172781 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb12c13e100 con 0x7fb12c09aef0 2024-03-23T14:57:08.965 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.909+0000 7fb113fff640 1 -- 172.21.15.164:0/1383172781 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7fb12000a9c0 con 0x7fb12c09aef0 2024-03-23T14:57:08.966 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.909+0000 7f80e6793640 1 -- 172.21.15.164:0/2164178415 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80e0146f50 con 0x7f80e01e41a0 2024-03-23T14:57:08.966 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.909+0000 7f80dcff9640 1 -- 172.21.15.164:0/2164178415 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80d0005c90 con 0x7f80e01e41a0 2024-03-23T14:57:08.966 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.909+0000 7fb113fff640 1 --2- 172.21.15.164:0/1383172781 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fb1180672d0 0x7fb118069790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:08.966 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.909+0000 7f80dcff9640 1 -- 172.21.15.164:0/2164178415 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80d0001440 con 0x7f80e01e41a0 2024-03-23T14:57:08.967 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.909+0000 7fb12affd640 1 --2- 172.21.15.164:0/1383172781 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fb1180672d0 0x7fb118069790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:08.967 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.909+0000 7fb113fff640 1 -- 172.21.15.164:0/1383172781 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fb1200847a0 con 0x7fb12c09aef0 2024-03-23T14:57:08.967 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.909+0000 7fc12359e640 1 -- 172.21.15.164:0/2852445345 wait complete. 2024-03-23T14:57:08.967 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.909+0000 7fc12359e640 1 Processor -- start 2024-03-23T14:57:08.968 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.909+0000 7f3acb7fe640 1 --2- 172.21.15.164:0/4004176476 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f3ab80672a0 0x7f3ab8069760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:08.968 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.909+0000 7fc12359e640 1 -- start start 2024-03-23T14:57:08.968 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.909+0000 7fc12359e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc10c0047d0 0x7fc11c045e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:08.968 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.909+0000 7fc12359e640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc11c0cb3a0 0x7fc11c16cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:08.968 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.909+0000 7fc12359e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc11c16d050 0x7fc11c0c30c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:08.968 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.909+0000 7fc12359e640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc11c0c4ed0 con 0x7fc11c16d050 2024-03-23T14:57:08.968 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.909+0000 7fc12359e640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fc11c0cc160 con 0x7fc10c0047d0 2024-03-23T14:57:08.968 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.909+0000 7fc12359e640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fc11c0c52f0 con 0x7fc11c0cb3a0 2024-03-23T14:57:08.969 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.909+0000 7f80dcff9640 1 -- 172.21.15.164:0/2164178415 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7f80d00ad020 con 0x7f80e01e41a0 2024-03-23T14:57:08.969 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.909+0000 7f3acb7fe640 1 --2- 172.21.15.164:0/4004176476 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f3ab80672a0 0x7f3ab8069760 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3ac00011f0 tx=0x7f3ac0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:08.969 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.909+0000 7fc121d9b640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc11c0cb3a0 0x7fc11c16cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:08.969 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.909+0000 7fc12259c640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc10c0047d0 0x7fc11c045e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:08.970 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.909+0000 7fc121d9b640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc11c0cb3a0 0x7fc11c16cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.164:60742/0 (socket says 172.21.15.164:60742) 2024-03-23T14:57:08.970 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.909+0000 7fc121d9b640 1 -- 172.21.15.164:0/1472382973 learned_addr learned my addr 172.21.15.164:0/1472382973 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:08.970 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.909+0000 7fb12affd640 1 --2- 172.21.15.164:0/1383172781 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fb1180672d0 0x7fb118069790 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb11c01cb10 tx=0x7fb11c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:08.970 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.909+0000 7fc122d9d640 1 --2- 172.21.15.164:0/1472382973 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc11c16d050 0x7fc11c0c30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:08.970 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.909+0000 7f80dcff9640 1 --2- 172.21.15.164:0/2164178415 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f80cc067370 0x7f80cc069830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:08.971 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.909+0000 7fc121d9b640 1 -- 172.21.15.164:0/1472382973 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc10c0047d0 msgr2=0x7fc11c045e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:08.971 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.909+0000 7fc121d9b640 1 --2- 172.21.15.164:0/1472382973 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc10c0047d0 0x7fc11c045e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:08.971 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.909+0000 7fc121d9b640 1 -- 172.21.15.164:0/1472382973 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc11c16d050 msgr2=0x7fc11c0c30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:08.971 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.909+0000 7f80dcff9640 1 -- 172.21.15.164:0/2164178415 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f80d0121780 con 0x7f80e01e41a0 2024-03-23T14:57:08.971 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.909+0000 7f80dffff640 1 --2- 172.21.15.164:0/2164178415 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f80cc067370 0x7f80cc069830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:08.972 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.909+0000 7fc121d9b640 1 --2- 172.21.15.164:0/1472382973 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc11c16d050 0x7fc11c0c30c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:08.972 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.909+0000 7fc121d9b640 1 -- 172.21.15.164:0/1472382973 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc11c0c4b50 con 0x7fc11c0cb3a0 2024-03-23T14:57:08.972 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.909+0000 7fc122d9d640 1 --2- 172.21.15.164:0/1472382973 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc11c16d050 0x7fc11c0c30c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T14:57:08.972 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.909+0000 7fc121d9b640 1 --2- 172.21.15.164:0/1472382973 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc11c0cb3a0 0x7fc11c16cb10 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fc1140ae040 tx=0x7fc114019050 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:08.972 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.909+0000 7f80dffff640 1 --2- 172.21.15.164:0/2164178415 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f80cc067370 0x7f80cc069830 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f80e00b9790 tx=0x7f80d80073f0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:08.973 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.913+0000 7f48bbfff640 1 -- 172.21.15.164:0/3317854406 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7f48c40ecda0 con 0x7f48d414efd0 2024-03-23T14:57:08.973 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.913+0000 7f7a7bfff640 1 -- 172.21.15.164:0/3377315891 <== mon.2 v2:172.21.15.173:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7f7a8c052f10 con 0x7f7a94146ff0 2024-03-23T14:57:08.974 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.913+0000 7f2ddeffd640 1 -- 172.21.15.164:0/1346441029 <== mon.2 v2:172.21.15.173:3301/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7f2de40ac070 con 0x7f2df009b570 2024-03-23T14:57:08.974 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.917+0000 7f2ddeffd640 1 --2- 172.21.15.164:0/1346441029 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f2dd00672a0 0x7f2dd0069760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:08.974 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.917+0000 7f2dee59c640 1 --2- 172.21.15.164:0/1346441029 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f2dd00672a0 0x7f2dd0069760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:08.974 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.917+0000 7f2dee59c640 1 --2- 172.21.15.164:0/1346441029 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f2dd00672a0 0x7f2dd0069760 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2de0006e90 tx=0x7f2de0006de0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:08.974 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.913+0000 7fc112ffd640 1 -- 172.21.15.164:0/1472382973 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1140abd80 con 0x7fc11c0cb3a0 2024-03-23T14:57:08.974 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.917+0000 7f15db7fe640 1 -- 172.21.15.164:0/1701667169 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7f15e4051230 con 0x7f15f40b0420 2024-03-23T14:57:08.975 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.917+0000 7fc12359e640 1 -- 172.21.15.164:0/1472382973 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc11c0c5f10 con 0x7fc11c0cb3a0 2024-03-23T14:57:08.975 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.917+0000 7fc12359e640 1 -- 172.21.15.164:0/1472382973 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc11c046b00 con 0x7fc11c0cb3a0 2024-03-23T14:57:08.975 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.917+0000 7fc12359e640 1 -- 172.21.15.164:0/1472382973 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc11c0c5010 con 0x7fc11c0cb3a0 2024-03-23T14:57:08.975 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.921+0000 7fc112ffd640 1 -- 172.21.15.164:0/1472382973 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1140b7450 con 0x7fc11c0cb3a0 2024-03-23T14:57:08.975 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.921+0000 7fc112ffd640 1 -- 172.21.15.164:0/1472382973 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1140b77c0 con 0x7fc11c0cb3a0 2024-03-23T14:57:08.976 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.921+0000 7f2ddeffd640 1 -- 172.21.15.164:0/1346441029 <== mon.2 v2:172.21.15.173:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f2de4007070 con 0x7f2df009b570 2024-03-23T14:57:08.976 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.921+0000 7f3ab3fff640 1 -- 172.21.15.164:0/4004176476 <== mon.2 v2:172.21.15.173:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7f3acc09eb30 con 0x7f3acc09ee10 2024-03-23T14:57:08.976 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.921+0000 7fb113fff640 1 -- 172.21.15.164:0/1383172781 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7fb120051370 con 0x7fb12c09aef0 2024-03-23T14:57:08.976 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.925+0000 7f2ddeffd640 1 -- 172.21.15.164:0/1346441029 <== mon.2 v2:172.21.15.173:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7f2de40ed280 con 0x7f2df009b570 2024-03-23T14:57:08.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.933+0000 7fc112ffd640 1 -- 172.21.15.164:0/1472382973 <== mon.2 v2:172.21.15.173:3301/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7fc1140b7920 con 0x7fc11c0cb3a0 2024-03-23T14:57:08.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.933+0000 7fc112ffd640 1 --2- 172.21.15.164:0/1472382973 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fc1040672d0 0x7fc104069790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:08.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.933+0000 7fc12259c640 1 --2- 172.21.15.164:0/1472382973 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fc1040672d0 0x7fc104069790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:08.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.933+0000 7fc112ffd640 1 -- 172.21.15.164:0/1472382973 <== mon.2 v2:172.21.15.173:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fc11411f5d0 con 0x7fc11c0cb3a0 2024-03-23T14:57:08.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.933+0000 7fc112ffd640 1 -- 172.21.15.164:0/1472382973 <== mon.2 v2:172.21.15.173:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7fc11411fa50 con 0x7fc11c0cb3a0 2024-03-23T14:57:08.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.933+0000 7fc12259c640 1 --2- 172.21.15.164:0/1472382973 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fc1040672d0 0x7fc104069790 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc1240502d0 tx=0x7fc12406a9c0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:08.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:08.937+0000 7f80dcff9640 1 -- 172.21.15.164:0/2164178415 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7f80d00ee3c0 con 0x7f80e01e41a0 2024-03-23T14:57:09.219 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.213+0000 7f7a98a67640 1 -- 172.21.15.164:0/3377315891 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f7a9413f2b0 con 0x7f7a94146ff0 2024-03-23T14:57:09.219 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.213+0000 7f48d9b32640 1 -- 172.21.15.164:0/3317854406 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f48d414e7c0 con 0x7f48d414efd0 2024-03-23T14:57:09.220 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.213+0000 7f7a7bfff640 1 -- 172.21.15.164:0/3377315891 <== mon.2 v2:172.21.15.173:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f7a9413f2b0 con 0x7f7a94146ff0 2024-03-23T14:57:09.220 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.217+0000 7f48bbfff640 1 -- 172.21.15.164:0/3317854406 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f48c40c4090 con 0x7f48d414efd0 2024-03-23T14:57:09.220 INFO:teuthology.orchestra.run.smithi164.stdout:55834574856 2024-03-23T14:57:09.222 INFO:teuthology.orchestra.run.smithi164.stdout:55834574856 2024-03-23T14:57:09.224 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.221+0000 7f7a98a67640 1 -- 172.21.15.164:0/3377315891 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f7a800672d0 msgr2=0x7f7a80069790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:09.224 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.221+0000 7f7a98a67640 1 --2- 172.21.15.164:0/3377315891 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f7a800672d0 0x7f7a80069790 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7a84006fd0 tx=0x7f7a84009060 comp rx=0 tx=0).stop 2024-03-23T14:57:09.224 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.221+0000 7f7a98a67640 1 -- 172.21.15.164:0/3377315891 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7a94146ff0 msgr2=0x7f7a941379f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:09.225 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.221+0000 7f7a98a67640 1 --2- 172.21.15.164:0/3377315891 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7a94146ff0 0x7f7a941379f0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f7a8c007d80 tx=0x7f7a8c00e4b0 comp rx=0 tx=0).stop 2024-03-23T14:57:09.225 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.221+0000 7f7a98a67640 1 -- 172.21.15.164:0/3377315891 shutdown_connections 2024-03-23T14:57:09.225 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.221+0000 7f7a98a67640 1 --2- 172.21.15.164:0/3377315891 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7a94146ff0 0x7f7a941379f0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:09.225 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.221+0000 7f7a98a67640 1 --2- 172.21.15.164:0/3377315891 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7a940b6650 0x7f7a9413e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:09.225 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.221+0000 7f7a98a67640 1 --2- 172.21.15.164:0/3377315891 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f7a800672d0 0x7f7a80069790 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:09.225 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.221+0000 7f7a98a67640 1 --2- 172.21.15.164:0/3377315891 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7a940729b0 0x7f7a941429f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:09.225 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.221+0000 7f7a98a67640 1 -- 172.21.15.164:0/3377315891 >> 172.21.15.164:0/3377315891 conn(0x7f7a9406b1a0 msgr2=0x7f7a940b30b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:09.225 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.221+0000 7f7a98a67640 1 -- 172.21.15.164:0/3377315891 shutdown_connections 2024-03-23T14:57:09.225 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.221+0000 7f7a98a67640 1 -- 172.21.15.164:0/3377315891 wait complete. 2024-03-23T14:57:09.227 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.221+0000 7f48d9b32640 1 -- 172.21.15.164:0/3317854406 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f48c00672a0 msgr2=0x7f48c0069760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:09.227 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.221+0000 7f48d9b32640 1 --2- 172.21.15.164:0/3317854406 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f48c00672a0 0x7f48c0069760 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f48c8000f70 tx=0x7f48c8007a10 comp rx=0 tx=0).stop 2024-03-23T14:57:09.227 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.221+0000 7f48d9b32640 1 -- 172.21.15.164:0/3317854406 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f48d414efd0 msgr2=0x7f48d41417e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:09.227 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.221+0000 7f48d9b32640 1 --2- 172.21.15.164:0/3317854406 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f48d414efd0 0x7f48d41417e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f48c40ad040 tx=0x7f48c401ac60 comp rx=0 tx=0).stop 2024-03-23T14:57:09.227 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.221+0000 7f48d9b32640 1 -- 172.21.15.164:0/3317854406 shutdown_connections 2024-03-23T14:57:09.227 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.221+0000 7f48d9b32640 1 --2- 172.21.15.164:0/3317854406 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f48d414efd0 0x7f48d41417e0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:09.227 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.221+0000 7f48d9b32640 1 --2- 172.21.15.164:0/3317854406 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f48d4149000 0x7f48d4146ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:09.227 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.221+0000 7f48d9b32640 1 --2- 172.21.15.164:0/3317854406 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f48c00672a0 0x7f48c0069760 secure :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f48c8000f70 tx=0x7f48c8007a10 comp rx=0 tx=0).stop 2024-03-23T14:57:09.227 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.221+0000 7f48d9b32640 1 --2- 172.21.15.164:0/3317854406 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f48d40b3ac0 0x7f48d413fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:09.228 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.221+0000 7f48d9b32640 1 -- 172.21.15.164:0/3317854406 >> 172.21.15.164:0/3317854406 conn(0x7f48d40bcc70 msgr2=0x7f48d41486c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:09.232 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.229+0000 7f48d9b32640 1 -- 172.21.15.164:0/3317854406 shutdown_connections 2024-03-23T14:57:09.232 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.229+0000 7f48d9b32640 1 -- 172.21.15.164:0/3317854406 wait complete. 2024-03-23T14:57:09.240 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.237+0000 7f15f3fff640 1 -- 172.21.15.164:0/1701667169 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f15d4005550 con 0x7f15f40b0420 2024-03-23T14:57:09.240 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.237+0000 7f15db7fe640 1 -- 172.21.15.164:0/1701667169 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f15e40548f0 con 0x7f15f40b0420 2024-03-23T14:57:09.241 INFO:teuthology.orchestra.run.smithi164.stdout:55834574856 2024-03-23T14:57:09.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.241+0000 7f15f3fff640 1 -- 172.21.15.164:0/1701667169 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f15e00672f0 msgr2=0x7f15e00697b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:09.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.241+0000 7fb132229640 1 -- 172.21.15.164:0/1383172781 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fb12c0bd410 con 0x7fb12c09aef0 2024-03-23T14:57:09.245 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.241+0000 7f15f3fff640 1 --2- 172.21.15.164:0/1701667169 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f15e00672f0 0x7f15e00697b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f15ec001570 tx=0x7f15ec001480 comp rx=0 tx=0).stop 2024-03-23T14:57:09.245 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.241+0000 7f15f3fff640 1 -- 172.21.15.164:0/1701667169 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f15f40b0420 msgr2=0x7f15f4137890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:09.246 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.241+0000 7f15f3fff640 1 --2- 172.21.15.164:0/1701667169 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f15f40b0420 0x7f15f4137890 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f15e400c430 tx=0x7f15e400c8f0 comp rx=0 tx=0).stop 2024-03-23T14:57:09.246 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.241+0000 7f15f3fff640 1 -- 172.21.15.164:0/1701667169 shutdown_connections 2024-03-23T14:57:09.246 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.241+0000 7f15f3fff640 1 --2- 172.21.15.164:0/1701667169 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f15e00672f0 0x7f15e00697b0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:09.246 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.241+0000 7f15f3fff640 1 --2- 172.21.15.164:0/1701667169 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f15f41366b0 0x7f15f4070c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:09.246 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.241+0000 7f15f3fff640 1 --2- 172.21.15.164:0/1701667169 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f15f4143c10 0x7f15f4070740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:09.246 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.241+0000 7f15f3fff640 1 --2- 172.21.15.164:0/1701667169 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f15f40b0420 0x7f15f4137890 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:09.246 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.241+0000 7f15f3fff640 1 -- 172.21.15.164:0/1701667169 >> 172.21.15.164:0/1701667169 conn(0x7f15f40b8d10 msgr2=0x7f15f4066ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:09.246 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.241+0000 7fb113fff640 1 -- 172.21.15.164:0/1383172781 <== mon.1 v2:172.21.15.173:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fb12c0bd410 con 0x7fb12c09aef0 2024-03-23T14:57:09.247 INFO:teuthology.orchestra.run.smithi164.stdout:55834574857 2024-03-23T14:57:09.247 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.241+0000 7f15f3fff640 1 -- 172.21.15.164:0/1701667169 shutdown_connections 2024-03-23T14:57:09.247 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.241+0000 7f15f3fff640 1 -- 172.21.15.164:0/1701667169 wait complete. 2024-03-23T14:57:09.248 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.241+0000 7f3ad2020640 1 -- 172.21.15.164:0/4004176476 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f3acc09eb30 con 0x7f3acc09ee10 2024-03-23T14:57:09.248 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.241+0000 7f3ab3fff640 1 -- 172.21.15.164:0/4004176476 <== mon.2 v2:172.21.15.173:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f3acc09eb30 con 0x7f3acc09ee10 2024-03-23T14:57:09.248 INFO:teuthology.orchestra.run.smithi164.stdout:55834574855 2024-03-23T14:57:09.250 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.6 2024-03-23T14:57:09.250 DEBUG:teuthology.parallel:result is None 2024-03-23T14:57:09.251 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.245+0000 7fb132229640 1 -- 172.21.15.164:0/1383172781 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fb1180672d0 msgr2=0x7fb118069790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:09.251 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.245+0000 7fb132229640 1 --2- 172.21.15.164:0/1383172781 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fb1180672d0 0x7fb118069790 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb11c01cb10 tx=0x7fb11c008040 comp rx=0 tx=0).stop 2024-03-23T14:57:09.251 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.245+0000 7fb132229640 1 -- 172.21.15.164:0/1383172781 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb12c09aef0 msgr2=0x7fb12c146b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:09.251 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.245+0000 7fb132229640 1 --2- 172.21.15.164:0/1383172781 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb12c09aef0 0x7fb12c146b30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb120007fd0 tx=0x7fb12000cbc0 comp rx=0 tx=0).stop 2024-03-23T14:57:09.251 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.245+0000 7fb132229640 1 -- 172.21.15.164:0/1383172781 shutdown_connections 2024-03-23T14:57:09.251 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.245+0000 7fb132229640 1 --2- 172.21.15.164:0/1383172781 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb12c0bc580 0x7fb12c13e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:09.251 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.245+0000 7fb132229640 1 --2- 172.21.15.164:0/1383172781 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb12c0bab20 0x7fb12c13e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:09.252 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.245+0000 7fb132229640 1 --2- 172.21.15.164:0/1383172781 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fb1180672d0 0x7fb118069790 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:09.252 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.245+0000 7fb132229640 1 --2- 172.21.15.164:0/1383172781 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb12c09aef0 0x7fb12c146b30 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:09.252 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.245+0000 7fb132229640 1 -- 172.21.15.164:0/1383172781 >> 172.21.15.164:0/1383172781 conn(0x7fb12c09ec30 msgr2=0x7fb12c138be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:09.252 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.245+0000 7fb132229640 1 -- 172.21.15.164:0/1383172781 shutdown_connections 2024-03-23T14:57:09.252 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.245+0000 7f3ad2020640 1 -- 172.21.15.164:0/4004176476 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f3ab80672a0 msgr2=0x7f3ab8069760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:09.252 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.245+0000 7f3ad2020640 1 --2- 172.21.15.164:0/4004176476 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f3ab80672a0 0x7f3ab8069760 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3ac00011f0 tx=0x7f3ac0009040 comp rx=0 tx=0).stop 2024-03-23T14:57:09.252 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.245+0000 7f3ad2020640 1 -- 172.21.15.164:0/4004176476 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3acc09ee10 msgr2=0x7f3acc146fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:09.252 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.245+0000 7f3ad2020640 1 --2- 172.21.15.164:0/4004176476 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3acc09ee10 0x7f3acc146fc0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f3abc00c390 tx=0x7f3abc00c850 comp rx=0 tx=0).stop 2024-03-23T14:57:09.252 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.245+0000 7f3ad2020640 1 -- 172.21.15.164:0/4004176476 shutdown_connections 2024-03-23T14:57:09.253 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.245+0000 7f3ad2020640 1 --2- 172.21.15.164:0/4004176476 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3acc09ee10 0x7f3acc146fc0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:09.253 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.245+0000 7f3ad2020640 1 --2- 172.21.15.164:0/4004176476 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f3ab80672a0 0x7f3ab8069760 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:09.253 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.245+0000 7f3ad2020640 1 --2- 172.21.15.164:0/4004176476 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3acc06cf20 0x7f3acc145450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:09.253 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.245+0000 7f3ad2020640 1 --2- 172.21.15.164:0/4004176476 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3acc06beb0 0x7f3acc131c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:09.253 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.245+0000 7f3ad2020640 1 -- 172.21.15.164:0/4004176476 >> 172.21.15.164:0/4004176476 conn(0x7f3acc0b9720 msgr2=0x7f3acc140fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:09.253 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.245+0000 7fb132229640 1 -- 172.21.15.164:0/1383172781 wait complete. 2024-03-23T14:57:09.254 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.245+0000 7f3ad2020640 1 -- 172.21.15.164:0/4004176476 shutdown_connections 2024-03-23T14:57:09.254 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.245+0000 7f3ad2020640 1 -- 172.21.15.164:0/4004176476 wait complete. 2024-03-23T14:57:09.256 INFO:teuthology.orchestra.run.smithi164.stdout:55834574856 2024-03-23T14:57:09.257 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.249+0000 7f2df4c1e640 1 -- 172.21.15.164:0/1346441029 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f2df00690a0 con 0x7f2df009b570 2024-03-23T14:57:09.257 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.249+0000 7f2ddeffd640 1 -- 172.21.15.164:0/1346441029 <== mon.2 v2:172.21.15.173:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f2de40f0940 con 0x7f2df009b570 2024-03-23T14:57:09.257 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.253+0000 7f2df4c1e640 1 -- 172.21.15.164:0/1346441029 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f2dd00672a0 msgr2=0x7f2dd0069760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:09.257 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.253+0000 7f2df4c1e640 1 --2- 172.21.15.164:0/1346441029 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f2dd00672a0 0x7f2dd0069760 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2de0006e90 tx=0x7f2de0006de0 comp rx=0 tx=0).stop 2024-03-23T14:57:09.258 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.253+0000 7f2df4c1e640 1 -- 172.21.15.164:0/1346441029 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2df009b570 msgr2=0x7f2df01f74b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:09.258 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.253+0000 7f2df4c1e640 1 --2- 172.21.15.164:0/1346441029 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2df009b570 0x7f2df01f74b0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f2de400d300 tx=0x7f2dd8007a10 comp rx=0 tx=0).stop 2024-03-23T14:57:09.258 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.253+0000 7f2df4c1e640 1 -- 172.21.15.164:0/1346441029 shutdown_connections 2024-03-23T14:57:09.258 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.253+0000 7f2df4c1e640 1 --2- 172.21.15.164:0/1346441029 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2df0157b60 0x7f2df0157fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:09.258 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.253+0000 7f2df4c1e640 1 --2- 172.21.15.164:0/1346441029 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2df009b570 0x7f2df01f74b0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:09.258 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.253+0000 7f2df4c1e640 1 --2- 172.21.15.164:0/1346441029 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f2dd00672a0 0x7f2dd0069760 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:09.258 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.253+0000 7f2df4c1e640 1 --2- 172.21.15.164:0/1346441029 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2dd80069e0 0x7f2df0071380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:09.258 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.253+0000 7f2df4c1e640 1 -- 172.21.15.164:0/1346441029 >> 172.21.15.164:0/1346441029 conn(0x7f2df00b8ad0 msgr2=0x7f2df00b3250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:09.258 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.253+0000 7f2df4c1e640 1 -- 172.21.15.164:0/1346441029 shutdown_connections 2024-03-23T14:57:09.258 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.253+0000 7f2df4c1e640 1 -- 172.21.15.164:0/1346441029 wait complete. 2024-03-23T14:57:09.264 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.4 2024-03-23T14:57:09.264 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.261+0000 7fc12359e640 1 -- 172.21.15.164:0/1472382973 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fc11c0c5010 con 0x7fc11c0cb3a0 2024-03-23T14:57:09.264 DEBUG:teuthology.parallel:result is None 2024-03-23T14:57:09.264 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.261+0000 7fc112ffd640 1 -- 172.21.15.164:0/1472382973 <== mon.2 v2:172.21.15.173:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc11c0c5010 con 0x7fc11c0cb3a0 2024-03-23T14:57:09.265 INFO:teuthology.orchestra.run.smithi164.stdout:55834574856 2024-03-23T14:57:09.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.265+0000 7f80e6793640 1 -- 172.21.15.164:0/2164178415 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f80e0069030 con 0x7f80e01e41a0 2024-03-23T14:57:09.269 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.265+0000 7fc12359e640 1 -- 172.21.15.164:0/1472382973 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fc1040672d0 msgr2=0x7fc104069790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:09.269 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.265+0000 7fc12359e640 1 --2- 172.21.15.164:0/1472382973 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fc1040672d0 0x7fc104069790 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc1240502d0 tx=0x7fc12406a9c0 comp rx=0 tx=0).stop 2024-03-23T14:57:09.269 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.265+0000 7fc12359e640 1 -- 172.21.15.164:0/1472382973 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc11c0cb3a0 msgr2=0x7fc11c16cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:09.269 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.265+0000 7fc12359e640 1 --2- 172.21.15.164:0/1472382973 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc11c0cb3a0 0x7fc11c16cb10 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fc1140ae040 tx=0x7fc114019050 comp rx=0 tx=0).stop 2024-03-23T14:57:09.269 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.265+0000 7fc12359e640 1 -- 172.21.15.164:0/1472382973 shutdown_connections 2024-03-23T14:57:09.269 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.265+0000 7fc12359e640 1 --2- 172.21.15.164:0/1472382973 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc11c16d050 0x7fc11c0c30c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:09.269 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.265+0000 7fc12359e640 1 --2- 172.21.15.164:0/1472382973 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc11c0cb3a0 0x7fc11c16cb10 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:09.269 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.265+0000 7fc12359e640 1 --2- 172.21.15.164:0/1472382973 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fc1040672d0 0x7fc104069790 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:09.269 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.265+0000 7fc12359e640 1 --2- 172.21.15.164:0/1472382973 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc10c0047d0 0x7fc11c045e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:09.269 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.265+0000 7fc12359e640 1 -- 172.21.15.164:0/1472382973 >> 172.21.15.164:0/1472382973 conn(0x7fc11c006e90 msgr2=0x7fc11c0066c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:09.269 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.265+0000 7fc12359e640 1 -- 172.21.15.164:0/1472382973 shutdown_connections 2024-03-23T14:57:09.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.265+0000 7fc12359e640 1 -- 172.21.15.164:0/1472382973 wait complete. 2024-03-23T14:57:09.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.265+0000 7f80dcff9640 1 -- 172.21.15.164:0/2164178415 <== mon.1 v2:172.21.15.173:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f80d00c1020 con 0x7f80e01e41a0 2024-03-23T14:57:09.270 INFO:teuthology.orchestra.run.smithi164.stdout:55834574857 2024-03-23T14:57:09.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.265+0000 7f80e6793640 1 -- 172.21.15.164:0/2164178415 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f80cc067370 msgr2=0x7f80cc069830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:09.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.265+0000 7f80e6793640 1 --2- 172.21.15.164:0/2164178415 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f80cc067370 0x7f80cc069830 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f80e00b9790 tx=0x7f80d80073f0 comp rx=0 tx=0).stop 2024-03-23T14:57:09.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.265+0000 7f80e6793640 1 -- 172.21.15.164:0/2164178415 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f80e01e41a0 msgr2=0x7f80e00aea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:09.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.265+0000 7f80e6793640 1 --2- 172.21.15.164:0/2164178415 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f80e01e41a0 0x7f80e00aea00 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f80d001f260 tx=0x7f80c8005800 comp rx=0 tx=0).stop 2024-03-23T14:57:09.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.265+0000 7f80e6793640 1 -- 172.21.15.164:0/2164178415 shutdown_connections 2024-03-23T14:57:09.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.265+0000 7f80e6793640 1 --2- 172.21.15.164:0/2164178415 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f80cc067370 0x7f80cc069830 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:09.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.265+0000 7f80e6793640 1 --2- 172.21.15.164:0/2164178415 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f80e01e41a0 0x7f80e00aea00 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:09.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.265+0000 7f80e6793640 1 --2- 172.21.15.164:0/2164178415 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f80e0147c90 0x7f80e01e3c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:09.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.265+0000 7f80e6793640 1 --2- 172.21.15.164:0/2164178415 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f80c80052b0 0x7f80e0147750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:09.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.265+0000 7f80e6793640 1 -- 172.21.15.164:0/2164178415 >> 172.21.15.164:0/2164178415 conn(0x7f80e00b8d10 msgr2=0x7f80e00b2e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:09.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.265+0000 7f80e6793640 1 -- 172.21.15.164:0/2164178415 shutdown_connections 2024-03-23T14:57:09.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.265+0000 7f80e6793640 1 -- 172.21.15.164:0/2164178415 wait complete. 2024-03-23T14:57:09.278 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.0 2024-03-23T14:57:09.278 DEBUG:teuthology.parallel:result is None 2024-03-23T14:57:09.279 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.2 2024-03-23T14:57:09.279 DEBUG:teuthology.parallel:result is None 2024-03-23T14:57:09.282 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.3 2024-03-23T14:57:09.282 DEBUG:teuthology.parallel:result is None 2024-03-23T14:57:09.288 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.5 2024-03-23T14:57:09.289 DEBUG:teuthology.parallel:result is None 2024-03-23T14:57:09.293 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.1 2024-03-23T14:57:09.293 DEBUG:teuthology.parallel:result is None 2024-03-23T14:57:09.294 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.7 2024-03-23T14:57:09.294 DEBUG:teuthology.parallel:result is None 2024-03-23T14:57:09.294 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-03-23T14:57:09.295 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-23T14:57:09.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.433+0000 7fba60273640 1 Processor -- start 2024-03-23T14:57:09.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.433+0000 7fba60273640 1 -- start start 2024-03-23T14:57:09.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.437+0000 7fba60273640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fba580b0230 0x7fba580b34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:09.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.437+0000 7fba60273640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba580b51d0 con 0x7fba5814db70 2024-03-23T14:57:09.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.437+0000 7fba60273640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba580b0610 con 0x7fba58148cf0 2024-03-23T14:57:09.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.437+0000 7fba60273640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fba580b1500 con 0x7fba580b0230 2024-03-23T14:57:09.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.437+0000 7fba5e00f640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fba580b0230 0x7fba580b34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:09.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.437+0000 7fba5e810640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fba5814db70 0x7fba5814ff60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:42504/0 (socket says 172.21.15.164:42504) 2024-03-23T14:57:09.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.437+0000 7fba5e810640 1 -- 172.21.15.164:0/972464389 learned_addr learned my addr 172.21.15.164:0/972464389 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-03-23T14:57:09.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.437+0000 7fba5e00f640 1 -- 172.21.15.164:0/972464389 >> v1:172.21.15.173:6789/0 conn(0x7fba58148cf0 legacy=0x7fba58149180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:09.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.437+0000 7fba5e00f640 1 -- 172.21.15.164:0/972464389 >> v1:172.21.15.164:6789/0 conn(0x7fba5814db70 legacy=0x7fba5814ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:09.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.437+0000 7fba5e00f640 1 -- 172.21.15.164:0/972464389 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba5806d250 con 0x7fba580b0230 2024-03-23T14:57:09.443 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.437+0000 7fba5e00f640 1 --2- 172.21.15.164:0/972464389 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fba580b0230 0x7fba580b34d0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fba4c0097c0 tx=0x7fba4c01c310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fd6d7444442fbac server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:09.443 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.437+0000 7fba5d00d640 1 -- 172.21.15.164:0/972464389 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba4c0ac640 con 0x7fba580b0230 2024-03-23T14:57:09.443 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.437+0000 7fba5d00d640 1 -- 172.21.15.164:0/972464389 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba4c0ac7a0 con 0x7fba580b0230 2024-03-23T14:57:09.443 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.437+0000 7fba5d00d640 1 -- 172.21.15.164:0/972464389 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba4c0acb30 con 0x7fba580b0230 2024-03-23T14:57:09.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.437+0000 7fba60273640 1 -- 172.21.15.164:0/972464389 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fba580b0230 msgr2=0x7fba580b34d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:09.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.437+0000 7fba60273640 1 --2- 172.21.15.164:0/972464389 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fba580b0230 0x7fba580b34d0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fba4c0097c0 tx=0x7fba4c01c310 comp rx=0 tx=0).stop 2024-03-23T14:57:09.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.437+0000 7fba60273640 1 -- 172.21.15.164:0/972464389 shutdown_connections 2024-03-23T14:57:09.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.437+0000 7fba60273640 1 --2- 172.21.15.164:0/972464389 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fba580b0230 0x7fba580b34d0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:09.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.437+0000 7fba60273640 1 -- 172.21.15.164:0/972464389 >> 172.21.15.164:0/972464389 conn(0x7fba580b8ca0 msgr2=0x7fba580b90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:09.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.437+0000 7fba60273640 1 -- 172.21.15.164:0/972464389 shutdown_connections 2024-03-23T14:57:09.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.437+0000 7fba60273640 1 -- 172.21.15.164:0/972464389 wait complete. 2024-03-23T14:57:09.446 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.441+0000 7fba60273640 1 Processor -- start 2024-03-23T14:57:09.446 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.441+0000 7fba60273640 1 -- start start 2024-03-23T14:57:09.446 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.441+0000 7fba60273640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fba580b0230 0x7fba5809bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:09.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.441+0000 7fba60273640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fba58148cf0 0x7fba5809ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:09.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.441+0000 7fba60273640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fba5814db70 0x7fba5809cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:09.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.441+0000 7fba60273640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fba5809c810 con 0x7fba58148cf0 2024-03-23T14:57:09.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.441+0000 7fba60273640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fba58142ec0 con 0x7fba5814db70 2024-03-23T14:57:09.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.441+0000 7fba60273640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fba5809b300 con 0x7fba580b0230 2024-03-23T14:57:09.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.441+0000 7fba5d80e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fba58148cf0 0x7fba5809ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:09.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.441+0000 7fba5d80e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fba58148cf0 0x7fba5809ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:59328/0 (socket says 172.21.15.164:59328) 2024-03-23T14:57:09.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.441+0000 7fba5d80e640 1 -- 172.21.15.164:0/1373232681 learned_addr learned my addr 172.21.15.164:0/1373232681 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:09.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.441+0000 7fba5e810640 1 --2- 172.21.15.164:0/1373232681 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fba5814db70 0x7fba5809cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:09.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.441+0000 7fba5e00f640 1 --2- 172.21.15.164:0/1373232681 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fba580b0230 0x7fba5809bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:09.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.441+0000 7fba5d80e640 1 -- 172.21.15.164:0/1373232681 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fba580b0230 msgr2=0x7fba5809bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:09.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.441+0000 7fba5d80e640 1 --2- 172.21.15.164:0/1373232681 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fba580b0230 0x7fba5809bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:09.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.441+0000 7fba5d80e640 1 -- 172.21.15.164:0/1373232681 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fba5814db70 msgr2=0x7fba5809cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:09.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.441+0000 7fba5d80e640 1 --2- 172.21.15.164:0/1373232681 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fba5814db70 0x7fba5809cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:09.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.441+0000 7fba5d80e640 1 -- 172.21.15.164:0/1373232681 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba4c009420 con 0x7fba58148cf0 2024-03-23T14:57:09.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.441+0000 7fba5d80e640 1 --2- 172.21.15.164:0/1373232681 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fba58148cf0 0x7fba5809ca80 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fba5400c7c0 tx=0x7fba5400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:09.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.441+0000 7fba4affd640 1 -- 172.21.15.164:0/1373232681 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba5400aa50 con 0x7fba58148cf0 2024-03-23T14:57:09.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.441+0000 7fba4affd640 1 -- 172.21.15.164:0/1373232681 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba5400abb0 con 0x7fba58148cf0 2024-03-23T14:57:09.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.441+0000 7fba60273640 1 -- 172.21.15.164:0/1373232681 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba5809d720 con 0x7fba58148cf0 2024-03-23T14:57:09.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.441+0000 7fba4affd640 1 -- 172.21.15.164:0/1373232681 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba54012420 con 0x7fba58148cf0 2024-03-23T14:57:09.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.441+0000 7fba60273640 1 -- 172.21.15.164:0/1373232681 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba581ddeb0 con 0x7fba58148cf0 2024-03-23T14:57:09.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.441+0000 7fba60273640 1 -- 172.21.15.164:0/1373232681 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba58072490 con 0x7fba58148cf0 2024-03-23T14:57:09.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.441+0000 7fba4affd640 1 -- 172.21.15.164:0/1373232681 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7fba54012580 con 0x7fba58148cf0 2024-03-23T14:57:09.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.441+0000 7fba4affd640 1 --2- 172.21.15.164:0/1373232681 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fba3c0672a0 0x7fba3c069760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:09.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.445+0000 7fba4affd640 1 -- 172.21.15.164:0/1373232681 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fba54085260 con 0x7fba58148cf0 2024-03-23T14:57:09.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.445+0000 7fba5e00f640 1 --2- 172.21.15.164:0/1373232681 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fba3c0672a0 0x7fba3c069760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:09.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.445+0000 7fba5e00f640 1 --2- 172.21.15.164:0/1373232681 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fba3c0672a0 0x7fba3c069760 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fba4c0ae000 tx=0x7fba4c0a6070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:09.453 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.449+0000 7fba4affd640 1 -- 172.21.15.164:0/1373232681 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7fba54051ea0 con 0x7fba58148cf0 2024-03-23T14:57:09.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.589+0000 7fba60273640 1 -- 172.21.15.164:0/1373232681 --> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fba580b1180 con 0x7fba3c0672a0 2024-03-23T14:57:09.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.605+0000 7fba4affd640 1 -- 172.21.15.164:0/1373232681 <== mgr.4107 v2:172.21.15.164:6832/2139087985 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251836 (secure 0 0 0) 0x7fba4c0bd090 con 0x7fba3c0672a0 2024-03-23T14:57:09.612 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-23T14:57:09.615 INFO:teuthology.orchestra.run.smithi164.stderr:dumped all 2024-03-23T14:57:09.616 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.609+0000 7fba60273640 1 -- 172.21.15.164:0/1373232681 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fba3c0672a0 msgr2=0x7fba3c069760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:09.617 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.609+0000 7fba60273640 1 --2- 172.21.15.164:0/1373232681 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fba3c0672a0 0x7fba3c069760 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fba4c0ae000 tx=0x7fba4c0a6070 comp rx=0 tx=0).stop 2024-03-23T14:57:09.617 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.609+0000 7fba60273640 1 -- 172.21.15.164:0/1373232681 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fba58148cf0 msgr2=0x7fba5809ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:09.617 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.609+0000 7fba60273640 1 --2- 172.21.15.164:0/1373232681 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fba58148cf0 0x7fba5809ca80 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fba5400c7c0 tx=0x7fba5400cc80 comp rx=0 tx=0).stop 2024-03-23T14:57:09.617 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.613+0000 7fba60273640 1 -- 172.21.15.164:0/1373232681 shutdown_connections 2024-03-23T14:57:09.617 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.613+0000 7fba60273640 1 --2- 172.21.15.164:0/1373232681 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fba3c0672a0 0x7fba3c069760 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:09.617 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.613+0000 7fba60273640 1 --2- 172.21.15.164:0/1373232681 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fba5814db70 0x7fba5809cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:09.617 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.613+0000 7fba60273640 1 --2- 172.21.15.164:0/1373232681 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fba58148cf0 0x7fba5809ca80 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:09.617 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.613+0000 7fba60273640 1 --2- 172.21.15.164:0/1373232681 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fba580b0230 0x7fba5809bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:09.617 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.613+0000 7fba60273640 1 -- 172.21.15.164:0/1373232681 >> 172.21.15.164:0/1373232681 conn(0x7fba580b8ca0 msgr2=0x7fba580b2350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:09.617 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.613+0000 7fba60273640 1 -- 172.21.15.164:0/1373232681 shutdown_connections 2024-03-23T14:57:09.617 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.613+0000 7fba60273640 1 -- 172.21.15.164:0/1373232681 wait complete. 2024-03-23T14:57:09.631 INFO:teuthology.orchestra.run.smithi164.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-03-23T14:57:09.112064+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219008,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755712,"statfs":{"total":773094113280,"available":772869849088,"internally_reserved":0,"allocated":2891776,"data_stored":1753028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.499113"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.518848+0000","last_change":"2024-03-23T14:56:59.652438+0000","last_active":"2024-03-23T14:57:01.518848+0000","last_peered":"2024-03-23T14:57:01.518848+0000","last_clean":"2024-03-23T14:57:01.518848+0000","last_became_active":"2024-03-23T14:56:59.652169+0000","last_became_peered":"2024-03-23T14:56:59.652169+0000","last_unstale":"2024-03-23T14:57:01.518848+0000","last_undegraded":"2024-03-23T14:57:01.518848+0000","last_fullsized":"2024-03-23T14:57:01.518848+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:58:31.542399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.621127+0000","last_change":"2024-03-23T14:56:59.632308+0000","last_active":"2024-03-23T14:57:00.621127+0000","last_peered":"2024-03-23T14:57:00.621127+0000","last_clean":"2024-03-23T14:57:00.621127+0000","last_became_active":"2024-03-23T14:56:59.631895+0000","last_became_peered":"2024-03-23T14:56:59.631895+0000","last_unstale":"2024-03-23T14:57:00.621127+0000","last_undegraded":"2024-03-23T14:57:00.621127+0000","last_fullsized":"2024-03-23T14:57:00.621127+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:59:19.727729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.555798+0000","last_change":"2024-03-23T14:56:59.636862+0000","last_active":"2024-03-23T14:57:01.555798+0000","last_peered":"2024-03-23T14:57:01.555798+0000","last_clean":"2024-03-23T14:57:01.555798+0000","last_became_active":"2024-03-23T14:56:59.636209+0000","last_became_peered":"2024-03-23T14:56:59.636209+0000","last_unstale":"2024-03-23T14:57:01.555798+0000","last_undegraded":"2024-03-23T14:57:01.555798+0000","last_fullsized":"2024-03-23T14:57:01.555798+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:11:59.565543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.551115+0000","last_change":"2024-03-23T14:56:59.662732+0000","last_active":"2024-03-23T14:57:01.551115+0000","last_peered":"2024-03-23T14:57:01.551115+0000","last_clean":"2024-03-23T14:57:01.551115+0000","last_became_active":"2024-03-23T14:56:59.661656+0000","last_became_peered":"2024-03-23T14:56:59.661656+0000","last_unstale":"2024-03-23T14:57:01.551115+0000","last_undegraded":"2024-03-23T14:57:01.551115+0000","last_fullsized":"2024-03-23T14:57:01.551115+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:35:32.710122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.518258+0000","last_change":"2024-03-23T14:56:59.651141+0000","last_active":"2024-03-23T14:57:01.518258+0000","last_peered":"2024-03-23T14:57:01.518258+0000","last_clean":"2024-03-23T14:57:01.518258+0000","last_became_active":"2024-03-23T14:56:59.650922+0000","last_became_peered":"2024-03-23T14:56:59.650922+0000","last_unstale":"2024-03-23T14:57:01.518258+0000","last_undegraded":"2024-03-23T14:57:01.518258+0000","last_fullsized":"2024-03-23T14:57:01.518258+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:14:04.996547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.551288+0000","last_change":"2024-03-23T14:56:59.662551+0000","last_active":"2024-03-23T14:57:01.551288+0000","last_peered":"2024-03-23T14:57:01.551288+0000","last_clean":"2024-03-23T14:57:01.551288+0000","last_became_active":"2024-03-23T14:56:59.661732+0000","last_became_peered":"2024-03-23T14:56:59.661732+0000","last_unstale":"2024-03-23T14:57:01.551288+0000","last_undegraded":"2024-03-23T14:57:01.551288+0000","last_fullsized":"2024-03-23T14:57:01.551288+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:35:26.329408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.518244+0000","last_change":"2024-03-23T14:56:59.646745+0000","last_active":"2024-03-23T14:57:01.518244+0000","last_peered":"2024-03-23T14:57:01.518244+0000","last_clean":"2024-03-23T14:57:01.518244+0000","last_became_active":"2024-03-23T14:56:59.646354+0000","last_became_peered":"2024-03-23T14:56:59.646354+0000","last_unstale":"2024-03-23T14:57:01.518244+0000","last_undegraded":"2024-03-23T14:57:01.518244+0000","last_fullsized":"2024-03-23T14:57:01.518244+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:58:40.996298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.617783+0000","last_change":"2024-03-23T14:56:59.659502+0000","last_active":"2024-03-23T14:57:00.617783+0000","last_peered":"2024-03-23T14:57:00.617783+0000","last_clean":"2024-03-23T14:57:00.617783+0000","last_became_active":"2024-03-23T14:56:59.659222+0000","last_became_peered":"2024-03-23T14:56:59.659222+0000","last_unstale":"2024-03-23T14:57:00.617783+0000","last_undegraded":"2024-03-23T14:57:00.617783+0000","last_fullsized":"2024-03-23T14:57:00.617783+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:07:26.887157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.519017+0000","last_change":"2024-03-23T14:56:59.659935+0000","last_active":"2024-03-23T14:57:01.519017+0000","last_peered":"2024-03-23T14:57:01.519017+0000","last_clean":"2024-03-23T14:57:01.519017+0000","last_became_active":"2024-03-23T14:56:59.659501+0000","last_became_peered":"2024-03-23T14:56:59.659501+0000","last_unstale":"2024-03-23T14:57:01.519017+0000","last_undegraded":"2024-03-23T14:57:01.519017+0000","last_fullsized":"2024-03-23T14:57:01.519017+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:21:16.376914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.529493+0000","last_change":"2024-03-23T14:56:59.659013+0000","last_active":"2024-03-23T14:57:01.529493+0000","last_peered":"2024-03-23T14:57:01.529493+0000","last_clean":"2024-03-23T14:57:01.529493+0000","last_became_active":"2024-03-23T14:56:59.658667+0000","last_became_peered":"2024-03-23T14:56:59.658667+0000","last_unstale":"2024-03-23T14:57:01.529493+0000","last_undegraded":"2024-03-23T14:57:01.529493+0000","last_fullsized":"2024-03-23T14:57:01.529493+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:12:59.763661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.529535+0000","last_change":"2024-03-23T14:56:59.663729+0000","last_active":"2024-03-23T14:57:01.529535+0000","last_peered":"2024-03-23T14:57:01.529535+0000","last_clean":"2024-03-23T14:57:01.529535+0000","last_became_active":"2024-03-23T14:56:59.663313+0000","last_became_peered":"2024-03-23T14:56:59.663313+0000","last_unstale":"2024-03-23T14:57:01.529535+0000","last_undegraded":"2024-03-23T14:57:01.529535+0000","last_fullsized":"2024-03-23T14:57:01.529535+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:57:06.681361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.619818+0000","last_change":"2024-03-23T14:56:59.640819+0000","last_active":"2024-03-23T14:57:00.619818+0000","last_peered":"2024-03-23T14:57:00.619818+0000","last_clean":"2024-03-23T14:57:00.619818+0000","last_became_active":"2024-03-23T14:56:59.640330+0000","last_became_peered":"2024-03-23T14:56:59.640330+0000","last_unstale":"2024-03-23T14:57:00.619818+0000","last_undegraded":"2024-03-23T14:57:00.619818+0000","last_fullsized":"2024-03-23T14:57:00.619818+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:01:26.168657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.618487+0000","last_change":"2024-03-23T14:56:59.645597+0000","last_active":"2024-03-23T14:57:00.618487+0000","last_peered":"2024-03-23T14:57:00.618487+0000","last_clean":"2024-03-23T14:57:00.618487+0000","last_became_active":"2024-03-23T14:56:59.644700+0000","last_became_peered":"2024-03-23T14:56:59.644700+0000","last_unstale":"2024-03-23T14:57:00.618487+0000","last_undegraded":"2024-03-23T14:57:00.618487+0000","last_fullsized":"2024-03-23T14:57:00.618487+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:59:19.727729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.556462+0000","last_change":"2024-03-23T14:56:59.636241+0000","last_active":"2024-03-23T14:57:01.556462+0000","last_peered":"2024-03-23T14:57:01.556462+0000","last_clean":"2024-03-23T14:57:01.556462+0000","last_became_active":"2024-03-23T14:56:59.635803+0000","last_became_peered":"2024-03-23T14:56:59.635803+0000","last_unstale":"2024-03-23T14:57:01.556462+0000","last_undegraded":"2024-03-23T14:57:01.556462+0000","last_fullsized":"2024-03-23T14:57:01.556462+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:07:26.887157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.620719+0000","last_change":"2024-03-23T14:56:59.642239+0000","last_active":"2024-03-23T14:57:00.620719+0000","last_peered":"2024-03-23T14:57:00.620719+0000","last_clean":"2024-03-23T14:57:00.620719+0000","last_became_active":"2024-03-23T14:56:59.641872+0000","last_became_peered":"2024-03-23T14:56:59.641872+0000","last_unstale":"2024-03-23T14:57:00.620719+0000","last_undegraded":"2024-03-23T14:57:00.620719+0000","last_fullsized":"2024-03-23T14:57:00.620719+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:09:00.536071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.551044+0000","last_change":"2024-03-23T14:56:59.634551+0000","last_active":"2024-03-23T14:57:01.551044+0000","last_peered":"2024-03-23T14:57:01.551044+0000","last_clean":"2024-03-23T14:57:01.551044+0000","last_became_active":"2024-03-23T14:56:59.634255+0000","last_became_peered":"2024-03-23T14:56:59.634255+0000","last_unstale":"2024-03-23T14:57:01.551044+0000","last_undegraded":"2024-03-23T14:57:01.551044+0000","last_fullsized":"2024-03-23T14:57:01.551044+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:39:22.051918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.552128+0000","last_change":"2024-03-23T14:56:59.657535+0000","last_active":"2024-03-23T14:57:01.552128+0000","last_peered":"2024-03-23T14:57:01.552128+0000","last_clean":"2024-03-23T14:57:01.552128+0000","last_became_active":"2024-03-23T14:56:59.656959+0000","last_became_peered":"2024-03-23T14:56:59.656959+0000","last_unstale":"2024-03-23T14:57:01.552128+0000","last_undegraded":"2024-03-23T14:57:01.552128+0000","last_fullsized":"2024-03-23T14:57:01.552128+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:58:31.542399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.555866+0000","last_change":"2024-03-23T14:56:59.639104+0000","last_active":"2024-03-23T14:57:01.555866+0000","last_peered":"2024-03-23T14:57:01.555866+0000","last_clean":"2024-03-23T14:57:01.555866+0000","last_became_active":"2024-03-23T14:56:59.638744+0000","last_became_peered":"2024-03-23T14:56:59.638744+0000","last_unstale":"2024-03-23T14:57:01.555866+0000","last_undegraded":"2024-03-23T14:57:01.555866+0000","last_fullsized":"2024-03-23T14:57:01.555866+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:29:08.139571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.518933+0000","last_change":"2024-03-23T14:56:59.647334+0000","last_active":"2024-03-23T14:57:01.518933+0000","last_peered":"2024-03-23T14:57:01.518933+0000","last_clean":"2024-03-23T14:57:01.518933+0000","last_became_active":"2024-03-23T14:56:59.646801+0000","last_became_peered":"2024-03-23T14:56:59.646801+0000","last_unstale":"2024-03-23T14:57:01.518933+0000","last_undegraded":"2024-03-23T14:57:01.518933+0000","last_fullsized":"2024-03-23T14:57:01.518933+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:02:12.733995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.529380+0000","last_change":"2024-03-23T14:56:59.639076+0000","last_active":"2024-03-23T14:57:01.529380+0000","last_peered":"2024-03-23T14:57:01.529380+0000","last_clean":"2024-03-23T14:57:01.529380+0000","last_became_active":"2024-03-23T14:56:59.638267+0000","last_became_peered":"2024-03-23T14:56:59.638267+0000","last_unstale":"2024-03-23T14:57:01.529380+0000","last_undegraded":"2024-03-23T14:57:01.529380+0000","last_fullsized":"2024-03-23T14:57:01.529380+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:59:19.727729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.556655+0000","last_change":"2024-03-23T14:56:59.647220+0000","last_active":"2024-03-23T14:57:01.556655+0000","last_peered":"2024-03-23T14:57:01.556655+0000","last_clean":"2024-03-23T14:57:01.556655+0000","last_became_active":"2024-03-23T14:56:59.646833+0000","last_became_peered":"2024-03-23T14:56:59.646833+0000","last_unstale":"2024-03-23T14:57:01.556655+0000","last_undegraded":"2024-03-23T14:57:01.556655+0000","last_fullsized":"2024-03-23T14:57:01.556655+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:38:24.739838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.551049+0000","last_change":"2024-03-23T14:56:59.662969+0000","last_active":"2024-03-23T14:57:01.551049+0000","last_peered":"2024-03-23T14:57:01.551049+0000","last_clean":"2024-03-23T14:57:01.551049+0000","last_became_active":"2024-03-23T14:56:59.661554+0000","last_became_peered":"2024-03-23T14:56:59.661554+0000","last_unstale":"2024-03-23T14:57:01.551049+0000","last_undegraded":"2024-03-23T14:57:01.551049+0000","last_fullsized":"2024-03-23T14:57:01.551049+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:52:36.323039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.618010+0000","last_change":"2024-03-23T14:56:59.643866+0000","last_active":"2024-03-23T14:57:00.618010+0000","last_peered":"2024-03-23T14:57:00.618010+0000","last_clean":"2024-03-23T14:57:00.618010+0000","last_became_active":"2024-03-23T14:56:59.643577+0000","last_became_peered":"2024-03-23T14:56:59.643577+0000","last_unstale":"2024-03-23T14:57:00.618010+0000","last_undegraded":"2024-03-23T14:57:00.618010+0000","last_fullsized":"2024-03-23T14:57:00.618010+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:37:56.745366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.550754+0000","last_change":"2024-03-23T14:56:59.651798+0000","last_active":"2024-03-23T14:57:01.550754+0000","last_peered":"2024-03-23T14:57:01.550754+0000","last_clean":"2024-03-23T14:57:01.550754+0000","last_became_active":"2024-03-23T14:56:59.651534+0000","last_became_peered":"2024-03-23T14:56:59.651534+0000","last_unstale":"2024-03-23T14:57:01.550754+0000","last_undegraded":"2024-03-23T14:57:01.550754+0000","last_fullsized":"2024-03-23T14:57:01.550754+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:57:06.681361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.619373+0000","last_change":"2024-03-23T14:56:59.645160+0000","last_active":"2024-03-23T14:57:00.619373+0000","last_peered":"2024-03-23T14:57:00.619373+0000","last_clean":"2024-03-23T14:57:00.619373+0000","last_became_active":"2024-03-23T14:56:59.643402+0000","last_became_peered":"2024-03-23T14:56:59.643402+0000","last_unstale":"2024-03-23T14:57:00.619373+0000","last_undegraded":"2024-03-23T14:57:00.619373+0000","last_fullsized":"2024-03-23T14:57:00.619373+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:35:32.710122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.529573+0000","last_change":"2024-03-23T14:56:59.640718+0000","last_active":"2024-03-23T14:57:01.529573+0000","last_peered":"2024-03-23T14:57:01.529573+0000","last_clean":"2024-03-23T14:57:01.529573+0000","last_became_active":"2024-03-23T14:56:59.640386+0000","last_became_peered":"2024-03-23T14:56:59.640386+0000","last_unstale":"2024-03-23T14:57:01.529573+0000","last_undegraded":"2024-03-23T14:57:01.529573+0000","last_fullsized":"2024-03-23T14:57:01.529573+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:39:22.051918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.621420+0000","last_change":"2024-03-23T14:56:59.650572+0000","last_active":"2024-03-23T14:57:00.621420+0000","last_peered":"2024-03-23T14:57:00.621420+0000","last_clean":"2024-03-23T14:57:00.621420+0000","last_became_active":"2024-03-23T14:56:59.650260+0000","last_became_peered":"2024-03-23T14:56:59.650260+0000","last_unstale":"2024-03-23T14:57:00.621420+0000","last_undegraded":"2024-03-23T14:57:00.621420+0000","last_fullsized":"2024-03-23T14:57:00.621420+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:12:59.763661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.619035+0000","last_change":"2024-03-23T14:56:59.646883+0000","last_active":"2024-03-23T14:57:00.619035+0000","last_peered":"2024-03-23T14:57:00.619035+0000","last_clean":"2024-03-23T14:57:00.619035+0000","last_became_active":"2024-03-23T14:56:59.643427+0000","last_became_peered":"2024-03-23T14:56:59.643427+0000","last_unstale":"2024-03-23T14:57:00.619035+0000","last_undegraded":"2024-03-23T14:57:00.619035+0000","last_fullsized":"2024-03-23T14:57:00.619035+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:52:36.323039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.557036+0000","last_change":"2024-03-23T14:56:59.646524+0000","last_active":"2024-03-23T14:57:01.557036+0000","last_peered":"2024-03-23T14:57:01.557036+0000","last_clean":"2024-03-23T14:57:01.557036+0000","last_became_active":"2024-03-23T14:56:59.646037+0000","last_became_peered":"2024-03-23T14:56:59.646037+0000","last_unstale":"2024-03-23T14:57:01.557036+0000","last_undegraded":"2024-03-23T14:57:01.557036+0000","last_fullsized":"2024-03-23T14:57:01.557036+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:57:06.681361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.551413+0000","last_change":"2024-03-23T14:56:59.658015+0000","last_active":"2024-03-23T14:57:01.551413+0000","last_peered":"2024-03-23T14:57:01.551413+0000","last_clean":"2024-03-23T14:57:01.551413+0000","last_became_active":"2024-03-23T14:56:59.657623+0000","last_became_peered":"2024-03-23T14:56:59.657623+0000","last_unstale":"2024-03-23T14:57:01.551413+0000","last_undegraded":"2024-03-23T14:57:01.551413+0000","last_fullsized":"2024-03-23T14:57:01.551413+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:14:04.996547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.555261+0000","last_change":"2024-03-23T14:56:59.660146+0000","last_active":"2024-03-23T14:57:01.555261+0000","last_peered":"2024-03-23T14:57:01.555261+0000","last_clean":"2024-03-23T14:57:01.555261+0000","last_became_active":"2024-03-23T14:56:59.659765+0000","last_became_peered":"2024-03-23T14:56:59.659765+0000","last_unstale":"2024-03-23T14:57:01.555261+0000","last_undegraded":"2024-03-23T14:57:01.555261+0000","last_fullsized":"2024-03-23T14:57:01.555261+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:49:20.982742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.529104+0000","last_change":"2024-03-23T14:56:59.637937+0000","last_active":"2024-03-23T14:57:01.529104+0000","last_peered":"2024-03-23T14:57:01.529104+0000","last_clean":"2024-03-23T14:57:01.529104+0000","last_became_active":"2024-03-23T14:56:59.637465+0000","last_became_peered":"2024-03-23T14:56:59.637465+0000","last_unstale":"2024-03-23T14:57:01.529104+0000","last_undegraded":"2024-03-23T14:57:01.529104+0000","last_fullsized":"2024-03-23T14:57:01.529104+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:38:24.739838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.619136+0000","last_change":"2024-03-23T14:56:59.633433+0000","last_active":"2024-03-23T14:57:00.619136+0000","last_peered":"2024-03-23T14:57:00.619136+0000","last_clean":"2024-03-23T14:57:00.619136+0000","last_became_active":"2024-03-23T14:56:59.633171+0000","last_became_peered":"2024-03-23T14:56:59.633171+0000","last_unstale":"2024-03-23T14:57:00.619136+0000","last_undegraded":"2024-03-23T14:57:00.619136+0000","last_fullsized":"2024-03-23T14:57:00.619136+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:36:11.384485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.618779+0000","last_change":"2024-03-23T14:56:58.621210+0000","last_active":"2024-03-23T14:57:00.618779+0000","last_peered":"2024-03-23T14:57:00.618779+0000","last_clean":"2024-03-23T14:57:00.618779+0000","last_became_active":"2024-03-23T14:56:58.620750+0000","last_became_peered":"2024-03-23T14:56:58.620750+0000","last_unstale":"2024-03-23T14:57:00.618779+0000","last_undegraded":"2024-03-23T14:57:00.618779+0000","last_fullsized":"2024-03-23T14:57:00.618779+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:24:18.799175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.556969+0000","last_change":"2024-03-23T14:56:59.633767+0000","last_active":"2024-03-23T14:57:01.556969+0000","last_peered":"2024-03-23T14:57:01.556969+0000","last_clean":"2024-03-23T14:57:01.556969+0000","last_became_active":"2024-03-23T14:56:59.633390+0000","last_became_peered":"2024-03-23T14:56:59.633390+0000","last_unstale":"2024-03-23T14:57:01.556969+0000","last_undegraded":"2024-03-23T14:57:01.556969+0000","last_fullsized":"2024-03-23T14:57:01.556969+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:12:59.763661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.529138+0000","last_change":"2024-03-23T14:56:58.613611+0000","last_active":"2024-03-23T14:57:01.529138+0000","last_peered":"2024-03-23T14:57:01.529138+0000","last_clean":"2024-03-23T14:57:01.529138+0000","last_became_active":"2024-03-23T14:56:58.608339+0000","last_became_peered":"2024-03-23T14:56:58.608339+0000","last_unstale":"2024-03-23T14:57:01.529138+0000","last_undegraded":"2024-03-23T14:57:01.529138+0000","last_fullsized":"2024-03-23T14:57:01.529138+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:48:12.389105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.618094+0000","last_change":"2024-03-23T14:56:58.621716+0000","last_active":"2024-03-23T14:57:00.618094+0000","last_peered":"2024-03-23T14:57:00.618094+0000","last_clean":"2024-03-23T14:57:00.618094+0000","last_became_active":"2024-03-23T14:56:58.620675+0000","last_became_peered":"2024-03-23T14:56:58.620675+0000","last_unstale":"2024-03-23T14:57:00.618094+0000","last_undegraded":"2024-03-23T14:57:00.618094+0000","last_fullsized":"2024-03-23T14:57:00.618094+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:07:10.915534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.617954+0000","last_change":"2024-03-23T14:56:59.644784+0000","last_active":"2024-03-23T14:57:00.617954+0000","last_peered":"2024-03-23T14:57:00.617954+0000","last_clean":"2024-03-23T14:57:00.617954+0000","last_became_active":"2024-03-23T14:56:59.644312+0000","last_became_peered":"2024-03-23T14:56:59.644312+0000","last_unstale":"2024-03-23T14:57:00.617954+0000","last_undegraded":"2024-03-23T14:57:00.617954+0000","last_fullsized":"2024-03-23T14:57:00.617954+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:38:24.739838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.550888+0000","last_change":"2024-03-23T14:56:58.611793+0000","last_active":"2024-03-23T14:57:01.550888+0000","last_peered":"2024-03-23T14:57:01.550888+0000","last_clean":"2024-03-23T14:57:01.550888+0000","last_became_active":"2024-03-23T14:56:58.611131+0000","last_became_peered":"2024-03-23T14:56:58.611131+0000","last_unstale":"2024-03-23T14:57:01.550888+0000","last_undegraded":"2024-03-23T14:57:01.550888+0000","last_fullsized":"2024-03-23T14:57:01.550888+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:36:10.376200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.620461+0000","last_change":"2024-03-23T14:56:59.649256+0000","last_active":"2024-03-23T14:57:00.620461+0000","last_peered":"2024-03-23T14:57:00.620461+0000","last_clean":"2024-03-23T14:57:00.620461+0000","last_became_active":"2024-03-23T14:56:59.648957+0000","last_became_peered":"2024-03-23T14:56:59.648957+0000","last_unstale":"2024-03-23T14:57:00.620461+0000","last_undegraded":"2024-03-23T14:57:00.620461+0000","last_fullsized":"2024-03-23T14:57:00.620461+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:07:26.887157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.529168+0000","last_change":"2024-03-23T14:56:58.630337+0000","last_active":"2024-03-23T14:57:01.529168+0000","last_peered":"2024-03-23T14:57:01.529168+0000","last_clean":"2024-03-23T14:57:01.529168+0000","last_became_active":"2024-03-23T14:56:58.629899+0000","last_became_peered":"2024-03-23T14:56:58.629899+0000","last_unstale":"2024-03-23T14:57:01.529168+0000","last_undegraded":"2024-03-23T14:57:01.529168+0000","last_fullsized":"2024-03-23T14:57:01.529168+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:24:18.799175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.551507+0000","last_change":"2024-03-23T14:56:59.657632+0000","last_active":"2024-03-23T14:57:01.551507+0000","last_peered":"2024-03-23T14:57:01.551507+0000","last_clean":"2024-03-23T14:57:01.551507+0000","last_became_active":"2024-03-23T14:56:59.655136+0000","last_became_peered":"2024-03-23T14:56:59.655136+0000","last_unstale":"2024-03-23T14:57:01.551507+0000","last_undegraded":"2024-03-23T14:57:01.551507+0000","last_fullsized":"2024-03-23T14:57:01.551507+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:46:57.866240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.551534+0000","last_change":"2024-03-23T14:56:58.627359+0000","last_active":"2024-03-23T14:57:01.551534+0000","last_peered":"2024-03-23T14:57:01.551534+0000","last_clean":"2024-03-23T14:57:01.551534+0000","last_became_active":"2024-03-23T14:56:58.626614+0000","last_became_peered":"2024-03-23T14:56:58.626614+0000","last_unstale":"2024-03-23T14:57:01.551534+0000","last_undegraded":"2024-03-23T14:57:01.551534+0000","last_fullsized":"2024-03-23T14:57:01.551534+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:05:00.726517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.556310+0000","last_change":"2024-03-23T14:56:59.659806+0000","last_active":"2024-03-23T14:57:01.556310+0000","last_peered":"2024-03-23T14:57:01.556310+0000","last_clean":"2024-03-23T14:57:01.556310+0000","last_became_active":"2024-03-23T14:56:59.658983+0000","last_became_peered":"2024-03-23T14:56:59.658983+0000","last_unstale":"2024-03-23T14:57:01.556310+0000","last_undegraded":"2024-03-23T14:57:01.556310+0000","last_fullsized":"2024-03-23T14:57:01.556310+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:58:29.591659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.621558+0000","last_change":"2024-03-23T14:56:58.625877+0000","last_active":"2024-03-23T14:57:00.621558+0000","last_peered":"2024-03-23T14:57:00.621558+0000","last_clean":"2024-03-23T14:57:00.621558+0000","last_became_active":"2024-03-23T14:56:58.625410+0000","last_became_peered":"2024-03-23T14:56:58.625410+0000","last_unstale":"2024-03-23T14:57:00.621558+0000","last_undegraded":"2024-03-23T14:57:00.621558+0000","last_fullsized":"2024-03-23T14:57:00.621558+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:05:00.726517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.619847+0000","last_change":"2024-03-23T14:56:59.647811+0000","last_active":"2024-03-23T14:57:00.619847+0000","last_peered":"2024-03-23T14:57:00.619847+0000","last_clean":"2024-03-23T14:57:00.619847+0000","last_became_active":"2024-03-23T14:56:59.647138+0000","last_became_peered":"2024-03-23T14:56:59.647138+0000","last_unstale":"2024-03-23T14:57:00.619847+0000","last_undegraded":"2024-03-23T14:57:00.619847+0000","last_fullsized":"2024-03-23T14:57:00.619847+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:49:20.982742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.529463+0000","last_change":"2024-03-23T14:56:58.630507+0000","last_active":"2024-03-23T14:57:01.529463+0000","last_peered":"2024-03-23T14:57:01.529463+0000","last_clean":"2024-03-23T14:57:01.529463+0000","last_became_active":"2024-03-23T14:56:58.629991+0000","last_became_peered":"2024-03-23T14:56:58.629991+0000","last_unstale":"2024-03-23T14:57:01.529463+0000","last_undegraded":"2024-03-23T14:57:01.529463+0000","last_fullsized":"2024-03-23T14:57:01.529463+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:01:25.160372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.518629+0000","last_change":"2024-03-23T14:56:59.649839+0000","last_active":"2024-03-23T14:57:01.518629+0000","last_peered":"2024-03-23T14:57:01.518629+0000","last_clean":"2024-03-23T14:57:01.518629+0000","last_became_active":"2024-03-23T14:56:59.648333+0000","last_became_peered":"2024-03-23T14:56:59.648333+0000","last_unstale":"2024-03-23T14:57:01.518629+0000","last_undegraded":"2024-03-23T14:57:01.518629+0000","last_fullsized":"2024-03-23T14:57:01.518629+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:46:57.866240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.557119+0000","last_change":"2024-03-23T14:56:58.609669+0000","last_active":"2024-03-23T14:57:01.557119+0000","last_peered":"2024-03-23T14:57:01.557119+0000","last_clean":"2024-03-23T14:57:01.557119+0000","last_became_active":"2024-03-23T14:56:58.608906+0000","last_became_peered":"2024-03-23T14:56:58.608906+0000","last_unstale":"2024-03-23T14:57:01.557119+0000","last_undegraded":"2024-03-23T14:57:01.557119+0000","last_fullsized":"2024-03-23T14:57:01.557119+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:34:51.972168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.556854+0000","last_change":"2024-03-23T14:56:59.648070+0000","last_active":"2024-03-23T14:57:01.556854+0000","last_peered":"2024-03-23T14:57:01.556854+0000","last_clean":"2024-03-23T14:57:01.556854+0000","last_became_active":"2024-03-23T14:56:59.647803+0000","last_became_peered":"2024-03-23T14:56:59.647803+0000","last_unstale":"2024-03-23T14:57:01.556854+0000","last_undegraded":"2024-03-23T14:57:01.556854+0000","last_fullsized":"2024-03-23T14:57:01.556854+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:37:56.745366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.551552+0000","last_change":"2024-03-23T14:56:58.626829+0000","last_active":"2024-03-23T14:57:01.551552+0000","last_peered":"2024-03-23T14:57:01.551552+0000","last_clean":"2024-03-23T14:57:01.551552+0000","last_became_active":"2024-03-23T14:56:58.626375+0000","last_became_peered":"2024-03-23T14:56:58.626375+0000","last_unstale":"2024-03-23T14:57:01.551552+0000","last_undegraded":"2024-03-23T14:57:01.551552+0000","last_fullsized":"2024-03-23T14:57:01.551552+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:48:12.389105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.518496+0000","last_change":"2024-03-23T14:56:59.653570+0000","last_active":"2024-03-23T14:57:01.518496+0000","last_peered":"2024-03-23T14:57:01.518496+0000","last_clean":"2024-03-23T14:57:01.518496+0000","last_became_active":"2024-03-23T14:56:59.653284+0000","last_became_peered":"2024-03-23T14:56:59.653284+0000","last_unstale":"2024-03-23T14:57:01.518496+0000","last_undegraded":"2024-03-23T14:57:01.518496+0000","last_fullsized":"2024-03-23T14:57:01.518496+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:34:30.083232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.528879+0000","last_change":"2024-03-23T14:56:58.614950+0000","last_active":"2024-03-23T14:57:01.528879+0000","last_peered":"2024-03-23T14:57:01.528879+0000","last_clean":"2024-03-23T14:57:01.528879+0000","last_became_active":"2024-03-23T14:56:58.614441+0000","last_became_peered":"2024-03-23T14:56:58.614441+0000","last_unstale":"2024-03-23T14:57:01.528879+0000","last_undegraded":"2024-03-23T14:57:01.528879+0000","last_fullsized":"2024-03-23T14:57:01.528879+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:05:00.726517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.528950+0000","last_change":"2024-03-23T14:56:59.636998+0000","last_active":"2024-03-23T14:57:01.528950+0000","last_peered":"2024-03-23T14:57:01.528950+0000","last_clean":"2024-03-23T14:57:01.528950+0000","last_became_active":"2024-03-23T14:56:59.636444+0000","last_became_peered":"2024-03-23T14:56:59.636444+0000","last_unstale":"2024-03-23T14:57:01.528950+0000","last_undegraded":"2024-03-23T14:57:01.528950+0000","last_fullsized":"2024-03-23T14:57:01.528950+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:09:00.536071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.555118+0000","last_change":"2024-03-23T14:56:58.604409+0000","last_active":"2024-03-23T14:57:01.555118+0000","last_peered":"2024-03-23T14:57:01.555118+0000","last_clean":"2024-03-23T14:57:01.555118+0000","last_became_active":"2024-03-23T14:56:58.604175+0000","last_became_peered":"2024-03-23T14:56:58.604175+0000","last_unstale":"2024-03-23T14:57:01.555118+0000","last_undegraded":"2024-03-23T14:57:01.555118+0000","last_fullsized":"2024-03-23T14:57:01.555118+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:07:10.915534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.518264+0000","last_change":"2024-03-23T14:56:59.651120+0000","last_active":"2024-03-23T14:57:01.518264+0000","last_peered":"2024-03-23T14:57:01.518264+0000","last_clean":"2024-03-23T14:57:01.518264+0000","last_became_active":"2024-03-23T14:56:59.650822+0000","last_became_peered":"2024-03-23T14:56:59.650822+0000","last_unstale":"2024-03-23T14:57:01.518264+0000","last_undegraded":"2024-03-23T14:57:01.518264+0000","last_fullsized":"2024-03-23T14:57:01.518264+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:35:26.329408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.619806+0000","last_change":"2024-03-23T14:56:58.606372+0000","last_active":"2024-03-23T14:57:00.619806+0000","last_peered":"2024-03-23T14:57:00.619806+0000","last_clean":"2024-03-23T14:57:00.619806+0000","last_became_active":"2024-03-23T14:56:58.606011+0000","last_became_peered":"2024-03-23T14:56:58.606011+0000","last_unstale":"2024-03-23T14:57:00.619806+0000","last_undegraded":"2024-03-23T14:57:00.619806+0000","last_fullsized":"2024-03-23T14:57:00.619806+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:34:51.972168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.518669+0000","last_change":"2024-03-23T14:56:59.652521+0000","last_active":"2024-03-23T14:57:01.518669+0000","last_peered":"2024-03-23T14:57:01.518669+0000","last_clean":"2024-03-23T14:57:01.518669+0000","last_became_active":"2024-03-23T14:56:59.652270+0000","last_became_peered":"2024-03-23T14:56:59.652270+0000","last_unstale":"2024-03-23T14:57:01.518669+0000","last_undegraded":"2024-03-23T14:57:01.518669+0000","last_fullsized":"2024-03-23T14:57:01.518669+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:51:59.393587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.556006+0000","last_change":"2024-03-23T14:56:58.610985+0000","last_active":"2024-03-23T14:57:01.556006+0000","last_peered":"2024-03-23T14:57:01.556006+0000","last_clean":"2024-03-23T14:57:01.556006+0000","last_became_active":"2024-03-23T14:56:58.610717+0000","last_became_peered":"2024-03-23T14:56:58.610717+0000","last_unstale":"2024-03-23T14:57:01.556006+0000","last_undegraded":"2024-03-23T14:57:01.556006+0000","last_fullsized":"2024-03-23T14:57:01.556006+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:24:18.799175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.519128+0000","last_change":"2024-03-23T14:56:59.652993+0000","last_active":"2024-03-23T14:57:01.519128+0000","last_peered":"2024-03-23T14:57:01.519128+0000","last_clean":"2024-03-23T14:57:01.519128+0000","last_became_active":"2024-03-23T14:56:59.652718+0000","last_became_peered":"2024-03-23T14:56:59.652718+0000","last_unstale":"2024-03-23T14:57:01.519128+0000","last_undegraded":"2024-03-23T14:57:01.519128+0000","last_fullsized":"2024-03-23T14:57:01.519128+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:06:34.693789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.621685+0000","last_change":"2024-03-23T14:56:58.608418+0000","last_active":"2024-03-23T14:57:00.621685+0000","last_peered":"2024-03-23T14:57:00.621685+0000","last_clean":"2024-03-23T14:57:00.621685+0000","last_became_active":"2024-03-23T14:56:58.607992+0000","last_became_peered":"2024-03-23T14:56:58.607992+0000","last_unstale":"2024-03-23T14:57:00.621685+0000","last_undegraded":"2024-03-23T14:57:00.621685+0000","last_fullsized":"2024-03-23T14:57:00.621685+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:36:10.376200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.551352+0000","last_change":"2024-03-23T14:56:59.657882+0000","last_active":"2024-03-23T14:57:01.551352+0000","last_peered":"2024-03-23T14:57:01.551352+0000","last_clean":"2024-03-23T14:57:01.551352+0000","last_became_active":"2024-03-23T14:56:59.657004+0000","last_became_peered":"2024-03-23T14:56:59.657004+0000","last_unstale":"2024-03-23T14:57:01.551352+0000","last_undegraded":"2024-03-23T14:57:01.551352+0000","last_fullsized":"2024-03-23T14:57:01.551352+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:58:40.996298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.557066+0000","last_change":"2024-03-23T14:56:58.617074+0000","last_active":"2024-03-23T14:57:01.557066+0000","last_peered":"2024-03-23T14:57:01.557066+0000","last_clean":"2024-03-23T14:57:01.557066+0000","last_became_active":"2024-03-23T14:56:58.616732+0000","last_became_peered":"2024-03-23T14:56:58.616732+0000","last_unstale":"2024-03-23T14:57:01.557066+0000","last_undegraded":"2024-03-23T14:57:01.557066+0000","last_fullsized":"2024-03-23T14:57:01.557066+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:05:00.726517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.518726+0000","last_change":"2024-03-23T14:56:59.664625+0000","last_active":"2024-03-23T14:57:01.518726+0000","last_peered":"2024-03-23T14:57:01.518726+0000","last_clean":"2024-03-23T14:57:01.518726+0000","last_became_active":"2024-03-23T14:56:59.664268+0000","last_became_peered":"2024-03-23T14:56:59.664268+0000","last_unstale":"2024-03-23T14:57:01.518726+0000","last_undegraded":"2024-03-23T14:57:01.518726+0000","last_fullsized":"2024-03-23T14:57:01.518726+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:53:55.152886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.517881+0000","last_change":"2024-03-23T14:56:48.722063+0000","last_active":"2024-03-23T14:57:01.517881+0000","last_peered":"2024-03-23T14:57:01.517881+0000","last_clean":"2024-03-23T14:57:01.517881+0000","last_became_active":"2024-03-23T14:56:48.720079+0000","last_became_peered":"2024-03-23T14:56:48.720079+0000","last_unstale":"2024-03-23T14:57:01.517881+0000","last_undegraded":"2024-03-23T14:57:01.517881+0000","last_fullsized":"2024-03-23T14:57:01.517881+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:47.478078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:47.478078+0000","last_clean_scrub_stamp":"2024-03-23T14:56:47.478078+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:55:51.231901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.551138+0000","last_change":"2024-03-23T14:56:58.611796+0000","last_active":"2024-03-23T14:57:01.551138+0000","last_peered":"2024-03-23T14:57:01.551138+0000","last_clean":"2024-03-23T14:57:01.551138+0000","last_became_active":"2024-03-23T14:56:58.611243+0000","last_became_peered":"2024-03-23T14:56:58.611243+0000","last_unstale":"2024-03-23T14:57:01.551138+0000","last_undegraded":"2024-03-23T14:57:01.551138+0000","last_fullsized":"2024-03-23T14:57:01.551138+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:24:18.799175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.618598+0000","last_change":"2024-03-23T14:56:59.664611+0000","last_active":"2024-03-23T14:57:00.618598+0000","last_peered":"2024-03-23T14:57:00.618598+0000","last_clean":"2024-03-23T14:57:00.618598+0000","last_became_active":"2024-03-23T14:56:59.664275+0000","last_became_peered":"2024-03-23T14:56:59.664275+0000","last_unstale":"2024-03-23T14:57:00.618598+0000","last_undegraded":"2024-03-23T14:57:00.618598+0000","last_fullsized":"2024-03-23T14:57:00.618598+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:12:59.763661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.517853+0000","last_change":"2024-03-23T14:56:58.620960+0000","last_active":"2024-03-23T14:57:01.517853+0000","last_peered":"2024-03-23T14:57:01.517853+0000","last_clean":"2024-03-23T14:57:01.517853+0000","last_became_active":"2024-03-23T14:56:58.620592+0000","last_became_peered":"2024-03-23T14:56:58.620592+0000","last_unstale":"2024-03-23T14:57:01.517853+0000","last_undegraded":"2024-03-23T14:57:01.517853+0000","last_fullsized":"2024-03-23T14:57:01.517853+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:01:25.160372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.556871+0000","last_change":"2024-03-23T14:56:59.621260+0000","last_active":"2024-03-23T14:57:01.556871+0000","last_peered":"2024-03-23T14:57:01.556871+0000","last_clean":"2024-03-23T14:57:01.556871+0000","last_became_active":"2024-03-23T14:56:59.620924+0000","last_became_peered":"2024-03-23T14:56:59.620924+0000","last_unstale":"2024-03-23T14:57:01.556871+0000","last_undegraded":"2024-03-23T14:57:01.556871+0000","last_fullsized":"2024-03-23T14:57:01.556871+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:59:19.727729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.555522+0000","last_change":"2024-03-23T14:56:58.611902+0000","last_active":"2024-03-23T14:57:01.555522+0000","last_peered":"2024-03-23T14:57:01.555522+0000","last_clean":"2024-03-23T14:57:01.555522+0000","last_became_active":"2024-03-23T14:56:58.611208+0000","last_became_peered":"2024-03-23T14:56:58.611208+0000","last_unstale":"2024-03-23T14:57:01.555522+0000","last_undegraded":"2024-03-23T14:57:01.555522+0000","last_fullsized":"2024-03-23T14:57:01.555522+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:36:10.376200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.620759+0000","last_change":"2024-03-23T14:56:59.640818+0000","last_active":"2024-03-23T14:57:00.620759+0000","last_peered":"2024-03-23T14:57:00.620759+0000","last_clean":"2024-03-23T14:57:00.620759+0000","last_became_active":"2024-03-23T14:56:59.640493+0000","last_became_peered":"2024-03-23T14:56:59.640493+0000","last_unstale":"2024-03-23T14:57:00.620759+0000","last_undegraded":"2024-03-23T14:57:00.620759+0000","last_fullsized":"2024-03-23T14:57:00.620759+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:38:24.739838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.551684+0000","last_change":"2024-03-23T14:56:58.616782+0000","last_active":"2024-03-23T14:57:01.551684+0000","last_peered":"2024-03-23T14:57:01.551684+0000","last_clean":"2024-03-23T14:57:01.551684+0000","last_became_active":"2024-03-23T14:56:58.615957+0000","last_became_peered":"2024-03-23T14:56:58.615957+0000","last_unstale":"2024-03-23T14:57:01.551684+0000","last_undegraded":"2024-03-23T14:57:01.551684+0000","last_fullsized":"2024-03-23T14:57:01.551684+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:01:25.160372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.619632+0000","last_change":"2024-03-23T14:56:59.623480+0000","last_active":"2024-03-23T14:57:00.619632+0000","last_peered":"2024-03-23T14:57:00.619632+0000","last_clean":"2024-03-23T14:57:00.619632+0000","last_became_active":"2024-03-23T14:56:59.623224+0000","last_became_peered":"2024-03-23T14:56:59.623224+0000","last_unstale":"2024-03-23T14:57:00.619632+0000","last_undegraded":"2024-03-23T14:57:00.619632+0000","last_fullsized":"2024-03-23T14:57:00.619632+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:48:13.397390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.621161+0000","last_change":"2024-03-23T14:56:58.604398+0000","last_active":"2024-03-23T14:57:00.621161+0000","last_peered":"2024-03-23T14:57:00.621161+0000","last_clean":"2024-03-23T14:57:00.621161+0000","last_became_active":"2024-03-23T14:56:58.603192+0000","last_became_peered":"2024-03-23T14:56:58.603192+0000","last_unstale":"2024-03-23T14:57:00.621161+0000","last_undegraded":"2024-03-23T14:57:00.621161+0000","last_fullsized":"2024-03-23T14:57:00.621161+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:24:18.799175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.551563+0000","last_change":"2024-03-23T14:56:59.661728+0000","last_active":"2024-03-23T14:57:01.551563+0000","last_peered":"2024-03-23T14:57:01.551563+0000","last_clean":"2024-03-23T14:57:01.551563+0000","last_became_active":"2024-03-23T14:56:59.656955+0000","last_became_peered":"2024-03-23T14:56:59.656955+0000","last_unstale":"2024-03-23T14:57:01.551563+0000","last_undegraded":"2024-03-23T14:57:01.551563+0000","last_fullsized":"2024-03-23T14:57:01.551563+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:51:59.393587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.518923+0000","last_change":"2024-03-23T14:56:58.619283+0000","last_active":"2024-03-23T14:57:01.518923+0000","last_peered":"2024-03-23T14:57:01.518923+0000","last_clean":"2024-03-23T14:57:01.518923+0000","last_became_active":"2024-03-23T14:56:58.618922+0000","last_became_peered":"2024-03-23T14:56:58.618922+0000","last_unstale":"2024-03-23T14:57:01.518923+0000","last_undegraded":"2024-03-23T14:57:01.518923+0000","last_fullsized":"2024-03-23T14:57:01.518923+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:29:07.131287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.620920+0000","last_change":"2024-03-23T14:56:59.636699+0000","last_active":"2024-03-23T14:57:00.620920+0000","last_peered":"2024-03-23T14:57:00.620920+0000","last_clean":"2024-03-23T14:57:00.620920+0000","last_became_active":"2024-03-23T14:56:59.636376+0000","last_became_peered":"2024-03-23T14:56:59.636376+0000","last_unstale":"2024-03-23T14:57:00.620920+0000","last_undegraded":"2024-03-23T14:57:00.620920+0000","last_fullsized":"2024-03-23T14:57:00.620920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:37:56.745366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.622171+0000","last_change":"2024-03-23T14:56:58.627211+0000","last_active":"2024-03-23T14:57:01.622171+0000","last_peered":"2024-03-23T14:57:01.622171+0000","last_clean":"2024-03-23T14:57:01.622171+0000","last_became_active":"2024-03-23T14:56:58.626779+0000","last_became_peered":"2024-03-23T14:56:58.626779+0000","last_unstale":"2024-03-23T14:57:01.622171+0000","last_undegraded":"2024-03-23T14:57:01.622171+0000","last_fullsized":"2024-03-23T14:57:01.622171+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:01:25.160372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.619749+0000","last_change":"2024-03-23T14:56:59.643640+0000","last_active":"2024-03-23T14:57:00.619749+0000","last_peered":"2024-03-23T14:57:00.619749+0000","last_clean":"2024-03-23T14:57:00.619749+0000","last_became_active":"2024-03-23T14:56:59.643255+0000","last_became_peered":"2024-03-23T14:56:59.643255+0000","last_unstale":"2024-03-23T14:57:00.619749+0000","last_undegraded":"2024-03-23T14:57:00.619749+0000","last_fullsized":"2024-03-23T14:57:00.619749+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:35:26.329408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.529191+0000","last_change":"2024-03-23T14:56:58.623880+0000","last_active":"2024-03-23T14:57:01.529191+0000","last_peered":"2024-03-23T14:57:01.529191+0000","last_clean":"2024-03-23T14:57:01.529191+0000","last_became_active":"2024-03-23T14:56:58.623291+0000","last_became_peered":"2024-03-23T14:56:58.623291+0000","last_unstale":"2024-03-23T14:57:01.529191+0000","last_undegraded":"2024-03-23T14:57:01.529191+0000","last_fullsized":"2024-03-23T14:57:01.529191+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:36:10.376200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.556526+0000","last_change":"2024-03-23T14:56:59.634446+0000","last_active":"2024-03-23T14:57:01.556526+0000","last_peered":"2024-03-23T14:57:01.556526+0000","last_clean":"2024-03-23T14:57:01.556526+0000","last_became_active":"2024-03-23T14:56:59.633946+0000","last_became_peered":"2024-03-23T14:56:59.633946+0000","last_unstale":"2024-03-23T14:57:01.556526+0000","last_undegraded":"2024-03-23T14:57:01.556526+0000","last_fullsized":"2024-03-23T14:57:01.556526+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:09:00.536071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.517807+0000","last_change":"2024-03-23T14:56:58.618776+0000","last_active":"2024-03-23T14:57:01.517807+0000","last_peered":"2024-03-23T14:57:01.517807+0000","last_clean":"2024-03-23T14:57:01.517807+0000","last_became_active":"2024-03-23T14:56:58.618418+0000","last_became_peered":"2024-03-23T14:56:58.618418+0000","last_unstale":"2024-03-23T14:57:01.517807+0000","last_undegraded":"2024-03-23T14:57:01.517807+0000","last_fullsized":"2024-03-23T14:57:01.517807+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:48:12.389105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.619949+0000","last_change":"2024-03-23T14:56:59.633845+0000","last_active":"2024-03-23T14:57:00.619949+0000","last_peered":"2024-03-23T14:57:00.619949+0000","last_clean":"2024-03-23T14:57:00.619949+0000","last_became_active":"2024-03-23T14:56:59.633409+0000","last_became_peered":"2024-03-23T14:56:59.633409+0000","last_unstale":"2024-03-23T14:57:00.619949+0000","last_undegraded":"2024-03-23T14:57:00.619949+0000","last_fullsized":"2024-03-23T14:57:00.619949+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:29:08.139571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.620668+0000","last_change":"2024-03-23T14:56:58.616951+0000","last_active":"2024-03-23T14:57:00.620668+0000","last_peered":"2024-03-23T14:57:00.620668+0000","last_clean":"2024-03-23T14:57:00.620668+0000","last_became_active":"2024-03-23T14:56:58.616383+0000","last_became_peered":"2024-03-23T14:56:58.616383+0000","last_unstale":"2024-03-23T14:57:00.620668+0000","last_undegraded":"2024-03-23T14:57:00.620668+0000","last_fullsized":"2024-03-23T14:57:00.620668+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:48:12.389105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.617942+0000","last_change":"2024-03-23T14:56:59.645384+0000","last_active":"2024-03-23T14:57:00.617942+0000","last_peered":"2024-03-23T14:57:00.617942+0000","last_clean":"2024-03-23T14:57:00.617942+0000","last_became_active":"2024-03-23T14:56:59.643723+0000","last_became_peered":"2024-03-23T14:56:59.643723+0000","last_unstale":"2024-03-23T14:57:00.617942+0000","last_undegraded":"2024-03-23T14:57:00.617942+0000","last_fullsized":"2024-03-23T14:57:00.617942+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:09:00.536071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.620676+0000","last_change":"2024-03-23T14:56:58.622814+0000","last_active":"2024-03-23T14:57:00.620676+0000","last_peered":"2024-03-23T14:57:00.620676+0000","last_clean":"2024-03-23T14:57:00.620676+0000","last_became_active":"2024-03-23T14:56:58.621745+0000","last_became_peered":"2024-03-23T14:56:58.621745+0000","last_unstale":"2024-03-23T14:57:00.620676+0000","last_undegraded":"2024-03-23T14:57:00.620676+0000","last_fullsized":"2024-03-23T14:57:00.620676+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:49:19.974457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.618866+0000","last_change":"2024-03-23T14:56:59.645315+0000","last_active":"2024-03-23T14:57:00.618866+0000","last_peered":"2024-03-23T14:57:00.618866+0000","last_clean":"2024-03-23T14:57:00.618866+0000","last_became_active":"2024-03-23T14:56:59.644287+0000","last_became_peered":"2024-03-23T14:56:59.644287+0000","last_unstale":"2024-03-23T14:57:00.618866+0000","last_undegraded":"2024-03-23T14:57:00.618866+0000","last_fullsized":"2024-03-23T14:57:00.618866+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:57:06.681361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.518700+0000","last_change":"2024-03-23T14:56:58.625848+0000","last_active":"2024-03-23T14:57:01.518700+0000","last_peered":"2024-03-23T14:57:01.518700+0000","last_clean":"2024-03-23T14:57:01.518700+0000","last_became_active":"2024-03-23T14:56:58.625453+0000","last_became_peered":"2024-03-23T14:56:58.625453+0000","last_unstale":"2024-03-23T14:57:01.518700+0000","last_undegraded":"2024-03-23T14:57:01.518700+0000","last_fullsized":"2024-03-23T14:57:01.518700+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:06:05.434722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.529162+0000","last_change":"2024-03-23T14:56:59.635812+0000","last_active":"2024-03-23T14:57:01.529162+0000","last_peered":"2024-03-23T14:57:01.529162+0000","last_clean":"2024-03-23T14:57:01.529162+0000","last_became_active":"2024-03-23T14:56:59.635519+0000","last_became_peered":"2024-03-23T14:56:59.635519+0000","last_unstale":"2024-03-23T14:57:01.529162+0000","last_undegraded":"2024-03-23T14:57:01.529162+0000","last_fullsized":"2024-03-23T14:57:01.529162+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:37:56.745366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.551417+0000","last_change":"2024-03-23T14:56:59.662434+0000","last_active":"2024-03-23T14:57:01.551417+0000","last_peered":"2024-03-23T14:57:01.551417+0000","last_clean":"2024-03-23T14:57:01.551417+0000","last_became_active":"2024-03-23T14:56:59.661514+0000","last_became_peered":"2024-03-23T14:56:59.661514+0000","last_unstale":"2024-03-23T14:57:01.551417+0000","last_undegraded":"2024-03-23T14:57:01.551417+0000","last_fullsized":"2024-03-23T14:57:01.551417+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:34:30.083232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.518270+0000","last_change":"2024-03-23T14:56:58.629744+0000","last_active":"2024-03-23T14:57:01.518270+0000","last_peered":"2024-03-23T14:57:01.518270+0000","last_clean":"2024-03-23T14:57:01.518270+0000","last_became_active":"2024-03-23T14:56:58.629401+0000","last_became_peered":"2024-03-23T14:56:58.629401+0000","last_unstale":"2024-03-23T14:57:01.518270+0000","last_undegraded":"2024-03-23T14:57:01.518270+0000","last_fullsized":"2024-03-23T14:57:01.518270+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:13:45.673576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.551735+0000","last_change":"2024-03-23T14:56:59.658250+0000","last_active":"2024-03-23T14:57:01.551735+0000","last_peered":"2024-03-23T14:57:01.551735+0000","last_clean":"2024-03-23T14:57:01.551735+0000","last_became_active":"2024-03-23T14:56:59.657085+0000","last_became_peered":"2024-03-23T14:56:59.657085+0000","last_unstale":"2024-03-23T14:57:01.551735+0000","last_undegraded":"2024-03-23T14:57:01.551735+0000","last_fullsized":"2024-03-23T14:57:01.551735+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:53:55.152886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.518837+0000","last_change":"2024-03-23T14:56:58.624886+0000","last_active":"2024-03-23T14:57:01.518837+0000","last_peered":"2024-03-23T14:57:01.518837+0000","last_clean":"2024-03-23T14:57:01.518837+0000","last_became_active":"2024-03-23T14:56:58.624478+0000","last_became_peered":"2024-03-23T14:56:58.624478+0000","last_unstale":"2024-03-23T14:57:01.518837+0000","last_undegraded":"2024-03-23T14:57:01.518837+0000","last_fullsized":"2024-03-23T14:57:01.518837+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:49:19.974457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.555868+0000","last_change":"2024-03-23T14:56:59.660089+0000","last_active":"2024-03-23T14:57:01.555868+0000","last_peered":"2024-03-23T14:57:01.555868+0000","last_clean":"2024-03-23T14:57:01.555868+0000","last_became_active":"2024-03-23T14:56:59.659469+0000","last_became_peered":"2024-03-23T14:56:59.659469+0000","last_unstale":"2024-03-23T14:57:01.555868+0000","last_undegraded":"2024-03-23T14:57:01.555868+0000","last_fullsized":"2024-03-23T14:57:01.555868+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:37:14.115808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.619024+0000","last_change":"2024-03-23T14:56:58.608147+0000","last_active":"2024-03-23T14:57:00.619024+0000","last_peered":"2024-03-23T14:57:00.619024+0000","last_clean":"2024-03-23T14:57:00.619024+0000","last_became_active":"2024-03-23T14:56:58.603630+0000","last_became_peered":"2024-03-23T14:56:58.603630+0000","last_unstale":"2024-03-23T14:57:00.619024+0000","last_undegraded":"2024-03-23T14:57:00.619024+0000","last_fullsized":"2024-03-23T14:57:00.619024+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:08:31.251287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.618985+0000","last_change":"2024-03-23T14:56:59.645116+0000","last_active":"2024-03-23T14:57:00.618985+0000","last_peered":"2024-03-23T14:57:00.618985+0000","last_clean":"2024-03-23T14:57:00.618985+0000","last_became_active":"2024-03-23T14:56:59.644484+0000","last_became_peered":"2024-03-23T14:56:59.644484+0000","last_unstale":"2024-03-23T14:57:00.618985+0000","last_undegraded":"2024-03-23T14:57:00.618985+0000","last_fullsized":"2024-03-23T14:57:00.618985+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:39:22.051918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.556818+0000","last_change":"2024-03-23T14:56:58.619314+0000","last_active":"2024-03-23T14:57:01.556818+0000","last_peered":"2024-03-23T14:57:01.556818+0000","last_clean":"2024-03-23T14:57:01.556818+0000","last_became_active":"2024-03-23T14:56:58.618751+0000","last_became_peered":"2024-03-23T14:56:58.618751+0000","last_unstale":"2024-03-23T14:57:01.556818+0000","last_undegraded":"2024-03-23T14:57:01.556818+0000","last_fullsized":"2024-03-23T14:57:01.556818+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:16:22.644801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574856,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":802816,"data_stored":661884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574857,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574857,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-23T14:57:09.636 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-23T14:57:09.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.777+0000 7effa64cb640 1 Processor -- start 2024-03-23T14:57:09.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.777+0000 7effa64cb640 1 -- start start 2024-03-23T14:57:09.783 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.777+0000 7effa64cb640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7effa00b4a00 0x7effa00b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:09.783 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.777+0000 7effa64cb640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7effa009e2d0 con 0x7effa009def0 2024-03-23T14:57:09.783 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.777+0000 7effa64cb640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7effa006cb70 con 0x7effa0147420 2024-03-23T14:57:09.783 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.777+0000 7effa64cb640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7effa0146de0 con 0x7effa00b4a00 2024-03-23T14:57:09.783 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.777+0000 7eff9f7fe640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7effa00b4a00 0x7effa00b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:09.783 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.777+0000 7eff9ffff640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7effa009def0 0x7effa00ae620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:42518/0 (socket says 172.21.15.164:42518) 2024-03-23T14:57:09.783 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.777+0000 7eff9ffff640 1 -- 172.21.15.164:0/43865319 learned_addr learned my addr 172.21.15.164:0/43865319 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-03-23T14:57:09.783 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.777+0000 7eff9f7fe640 1 -- 172.21.15.164:0/43865319 >> v1:172.21.15.173:6789/0 conn(0x7effa0147420 legacy=0x7effa014e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:09.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.777+0000 7eff9f7fe640 1 -- 172.21.15.164:0/43865319 >> v1:172.21.15.164:6789/0 conn(0x7effa009def0 legacy=0x7effa00ae620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:09.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.777+0000 7eff9f7fe640 1 -- 172.21.15.164:0/43865319 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effa006ce50 con 0x7effa00b4a00 2024-03-23T14:57:09.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.781+0000 7eff9f7fe640 1 --2- 172.21.15.164:0/43865319 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7effa00b4a00 0x7effa00b4e90 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7eff940097c0 tx=0x7eff9401c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ded3b67f3f997469 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:09.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.781+0000 7eff9effd640 1 -- 172.21.15.164:0/43865319 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff940ab600 con 0x7effa00b4a00 2024-03-23T14:57:09.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.781+0000 7eff9effd640 1 -- 172.21.15.164:0/43865319 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff940a7050 con 0x7effa00b4a00 2024-03-23T14:57:09.786 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.781+0000 7eff9effd640 1 -- 172.21.15.164:0/43865319 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff940ab980 con 0x7effa00b4a00 2024-03-23T14:57:09.786 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.781+0000 7effa64cb640 1 -- 172.21.15.164:0/43865319 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7effa00b4a00 msgr2=0x7effa00b4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:09.786 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.781+0000 7effa64cb640 1 --2- 172.21.15.164:0/43865319 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7effa00b4a00 0x7effa00b4e90 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7eff940097c0 tx=0x7eff9401c410 comp rx=0 tx=0).stop 2024-03-23T14:57:09.786 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.781+0000 7effa64cb640 1 -- 172.21.15.164:0/43865319 shutdown_connections 2024-03-23T14:57:09.787 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.781+0000 7effa64cb640 1 --2- 172.21.15.164:0/43865319 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7effa00b4a00 0x7effa00b4e90 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:09.787 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.781+0000 7effa64cb640 1 -- 172.21.15.164:0/43865319 >> 172.21.15.164:0/43865319 conn(0x7effa00b8ca0 msgr2=0x7effa00b90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:09.787 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.781+0000 7effa64cb640 1 -- 172.21.15.164:0/43865319 shutdown_connections 2024-03-23T14:57:09.787 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.781+0000 7effa64cb640 1 -- 172.21.15.164:0/43865319 wait complete. 2024-03-23T14:57:09.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.781+0000 7effa64cb640 1 Processor -- start 2024-03-23T14:57:09.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.781+0000 7effa64cb640 1 -- start start 2024-03-23T14:57:09.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.781+0000 7effa64cb640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7effa009def0 0x7effa013ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:09.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.781+0000 7effa64cb640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7effa00b4a00 0x7effa01434b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:09.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.781+0000 7effa64cb640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7effa0147420 0x7effa01439f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:09.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.781+0000 7effa64cb640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7effa01e2f50 con 0x7effa00b4a00 2024-03-23T14:57:09.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.781+0000 7effa64cb640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7effa01e2ac0 con 0x7effa0147420 2024-03-23T14:57:09.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.781+0000 7effa64cb640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7effa01e24f0 con 0x7effa009def0 2024-03-23T14:57:09.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.785+0000 7eff9f7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7effa00b4a00 0x7effa01434b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:09.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.785+0000 7eff9f7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7effa00b4a00 0x7effa01434b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:59344/0 (socket says 172.21.15.164:59344) 2024-03-23T14:57:09.790 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.785+0000 7eff9f7fe640 1 -- 172.21.15.164:0/2927218240 learned_addr learned my addr 172.21.15.164:0/2927218240 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:09.790 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.785+0000 7eff9ffff640 1 --2- 172.21.15.164:0/2927218240 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7effa009def0 0x7effa013ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:09.790 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.785+0000 7effa4a68640 1 --2- 172.21.15.164:0/2927218240 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7effa0147420 0x7effa01439f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:09.790 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.785+0000 7eff9f7fe640 1 -- 172.21.15.164:0/2927218240 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7effa009def0 msgr2=0x7effa013ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:09.790 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.785+0000 7eff9f7fe640 1 --2- 172.21.15.164:0/2927218240 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7effa009def0 0x7effa013ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:09.790 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.785+0000 7eff9f7fe640 1 -- 172.21.15.164:0/2927218240 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7effa0147420 msgr2=0x7effa01439f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:09.790 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.785+0000 7eff9f7fe640 1 --2- 172.21.15.164:0/2927218240 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7effa0147420 0x7effa01439f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:09.790 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.785+0000 7eff9f7fe640 1 -- 172.21.15.164:0/2927218240 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff94009420 con 0x7effa00b4a00 2024-03-23T14:57:09.790 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.785+0000 7effa4a68640 1 --2- 172.21.15.164:0/2927218240 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7effa0147420 0x7effa01439f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T14:57:09.790 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.785+0000 7eff9f7fe640 1 --2- 172.21.15.164:0/2927218240 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7effa00b4a00 0x7effa01434b0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7eff940ae040 tx=0x7eff940191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:09.790 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.785+0000 7eff9cff9640 1 -- 172.21.15.164:0/2927218240 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff940b4840 con 0x7effa00b4a00 2024-03-23T14:57:09.790 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.785+0000 7eff9cff9640 1 -- 172.21.15.164:0/2927218240 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff940b49a0 con 0x7effa00b4a00 2024-03-23T14:57:09.790 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.785+0000 7effa64cb640 1 -- 172.21.15.164:0/2927218240 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effa009b400 con 0x7effa00b4a00 2024-03-23T14:57:09.791 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.785+0000 7eff9cff9640 1 -- 172.21.15.164:0/2927218240 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff940b4d50 con 0x7effa00b4a00 2024-03-23T14:57:09.791 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.785+0000 7effa64cb640 1 -- 172.21.15.164:0/2927218240 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effa01e2750 con 0x7effa00b4a00 2024-03-23T14:57:09.791 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.785+0000 7effa64cb640 1 -- 172.21.15.164:0/2927218240 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effa0143f30 con 0x7effa00b4a00 2024-03-23T14:57:09.791 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.785+0000 7eff9cff9640 1 -- 172.21.15.164:0/2927218240 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7eff940b2040 con 0x7effa00b4a00 2024-03-23T14:57:09.791 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.785+0000 7eff9cff9640 1 --2- 172.21.15.164:0/2927218240 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7eff8c0672a0 0x7eff8c069760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:09.792 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.785+0000 7eff9ffff640 1 --2- 172.21.15.164:0/2927218240 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7eff8c0672a0 0x7eff8c069760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:09.792 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.785+0000 7eff9cff9640 1 -- 172.21.15.164:0/2927218240 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7eff9411f3e0 con 0x7effa00b4a00 2024-03-23T14:57:09.792 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.785+0000 7eff9ffff640 1 --2- 172.21.15.164:0/2927218240 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7eff8c0672a0 0x7eff8c069760 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7eff90005c10 tx=0x7eff90009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:09.794 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.789+0000 7eff9cff9640 1 -- 172.21.15.164:0/2927218240 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7eff940ec020 con 0x7effa00b4a00 2024-03-23T14:57:09.938 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.933+0000 7effa64cb640 1 -- 172.21.15.164:0/2927218240 --> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7effa00690a0 con 0x7eff8c0672a0 2024-03-23T14:57:09.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.949+0000 7eff9cff9640 1 -- 172.21.15.164:0/2927218240 <== mgr.4107 v2:172.21.15.164:6832/2139087985 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251836 (secure 0 0 0) 0x7effa00690a0 con 0x7eff8c0672a0 2024-03-23T14:57:09.955 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-23T14:57:09.959 INFO:teuthology.orchestra.run.smithi164.stderr:dumped all 2024-03-23T14:57:09.960 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.953+0000 7effa64cb640 1 -- 172.21.15.164:0/2927218240 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7eff8c0672a0 msgr2=0x7eff8c069760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:09.960 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.953+0000 7effa64cb640 1 --2- 172.21.15.164:0/2927218240 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7eff8c0672a0 0x7eff8c069760 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7eff90005c10 tx=0x7eff90009040 comp rx=0 tx=0).stop 2024-03-23T14:57:09.960 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.953+0000 7effa64cb640 1 -- 172.21.15.164:0/2927218240 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7effa00b4a00 msgr2=0x7effa01434b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:09.960 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.953+0000 7effa64cb640 1 --2- 172.21.15.164:0/2927218240 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7effa00b4a00 0x7effa01434b0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7eff940ae040 tx=0x7eff940191b0 comp rx=0 tx=0).stop 2024-03-23T14:57:09.960 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.953+0000 7effa64cb640 1 -- 172.21.15.164:0/2927218240 shutdown_connections 2024-03-23T14:57:09.960 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.953+0000 7effa64cb640 1 --2- 172.21.15.164:0/2927218240 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7eff8c0672a0 0x7eff8c069760 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:09.960 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.953+0000 7effa64cb640 1 --2- 172.21.15.164:0/2927218240 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7effa0147420 0x7effa01439f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:09.960 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.953+0000 7effa64cb640 1 --2- 172.21.15.164:0/2927218240 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7effa00b4a00 0x7effa01434b0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:09.960 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.953+0000 7effa64cb640 1 --2- 172.21.15.164:0/2927218240 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7effa009def0 0x7effa013ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:09.960 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.953+0000 7effa64cb640 1 -- 172.21.15.164:0/2927218240 >> 172.21.15.164:0/2927218240 conn(0x7effa00b8ca0 msgr2=0x7effa0151850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:09.960 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.953+0000 7effa64cb640 1 -- 172.21.15.164:0/2927218240 shutdown_connections 2024-03-23T14:57:09.960 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:09.953+0000 7effa64cb640 1 -- 172.21.15.164:0/2927218240 wait complete. 2024-03-23T14:57:09.975 INFO:teuthology.orchestra.run.smithi164.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-03-23T14:57:09.112064+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219008,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755712,"statfs":{"total":773094113280,"available":772869849088,"internally_reserved":0,"allocated":2891776,"data_stored":1753028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.499113"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.518848+0000","last_change":"2024-03-23T14:56:59.652438+0000","last_active":"2024-03-23T14:57:01.518848+0000","last_peered":"2024-03-23T14:57:01.518848+0000","last_clean":"2024-03-23T14:57:01.518848+0000","last_became_active":"2024-03-23T14:56:59.652169+0000","last_became_peered":"2024-03-23T14:56:59.652169+0000","last_unstale":"2024-03-23T14:57:01.518848+0000","last_undegraded":"2024-03-23T14:57:01.518848+0000","last_fullsized":"2024-03-23T14:57:01.518848+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:58:31.542399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.621127+0000","last_change":"2024-03-23T14:56:59.632308+0000","last_active":"2024-03-23T14:57:00.621127+0000","last_peered":"2024-03-23T14:57:00.621127+0000","last_clean":"2024-03-23T14:57:00.621127+0000","last_became_active":"2024-03-23T14:56:59.631895+0000","last_became_peered":"2024-03-23T14:56:59.631895+0000","last_unstale":"2024-03-23T14:57:00.621127+0000","last_undegraded":"2024-03-23T14:57:00.621127+0000","last_fullsized":"2024-03-23T14:57:00.621127+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:59:19.727729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.555798+0000","last_change":"2024-03-23T14:56:59.636862+0000","last_active":"2024-03-23T14:57:01.555798+0000","last_peered":"2024-03-23T14:57:01.555798+0000","last_clean":"2024-03-23T14:57:01.555798+0000","last_became_active":"2024-03-23T14:56:59.636209+0000","last_became_peered":"2024-03-23T14:56:59.636209+0000","last_unstale":"2024-03-23T14:57:01.555798+0000","last_undegraded":"2024-03-23T14:57:01.555798+0000","last_fullsized":"2024-03-23T14:57:01.555798+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:11:59.565543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.551115+0000","last_change":"2024-03-23T14:56:59.662732+0000","last_active":"2024-03-23T14:57:01.551115+0000","last_peered":"2024-03-23T14:57:01.551115+0000","last_clean":"2024-03-23T14:57:01.551115+0000","last_became_active":"2024-03-23T14:56:59.661656+0000","last_became_peered":"2024-03-23T14:56:59.661656+0000","last_unstale":"2024-03-23T14:57:01.551115+0000","last_undegraded":"2024-03-23T14:57:01.551115+0000","last_fullsized":"2024-03-23T14:57:01.551115+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:35:32.710122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.518258+0000","last_change":"2024-03-23T14:56:59.651141+0000","last_active":"2024-03-23T14:57:01.518258+0000","last_peered":"2024-03-23T14:57:01.518258+0000","last_clean":"2024-03-23T14:57:01.518258+0000","last_became_active":"2024-03-23T14:56:59.650922+0000","last_became_peered":"2024-03-23T14:56:59.650922+0000","last_unstale":"2024-03-23T14:57:01.518258+0000","last_undegraded":"2024-03-23T14:57:01.518258+0000","last_fullsized":"2024-03-23T14:57:01.518258+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:14:04.996547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.551288+0000","last_change":"2024-03-23T14:56:59.662551+0000","last_active":"2024-03-23T14:57:01.551288+0000","last_peered":"2024-03-23T14:57:01.551288+0000","last_clean":"2024-03-23T14:57:01.551288+0000","last_became_active":"2024-03-23T14:56:59.661732+0000","last_became_peered":"2024-03-23T14:56:59.661732+0000","last_unstale":"2024-03-23T14:57:01.551288+0000","last_undegraded":"2024-03-23T14:57:01.551288+0000","last_fullsized":"2024-03-23T14:57:01.551288+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:35:26.329408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.518244+0000","last_change":"2024-03-23T14:56:59.646745+0000","last_active":"2024-03-23T14:57:01.518244+0000","last_peered":"2024-03-23T14:57:01.518244+0000","last_clean":"2024-03-23T14:57:01.518244+0000","last_became_active":"2024-03-23T14:56:59.646354+0000","last_became_peered":"2024-03-23T14:56:59.646354+0000","last_unstale":"2024-03-23T14:57:01.518244+0000","last_undegraded":"2024-03-23T14:57:01.518244+0000","last_fullsized":"2024-03-23T14:57:01.518244+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:58:40.996298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.617783+0000","last_change":"2024-03-23T14:56:59.659502+0000","last_active":"2024-03-23T14:57:00.617783+0000","last_peered":"2024-03-23T14:57:00.617783+0000","last_clean":"2024-03-23T14:57:00.617783+0000","last_became_active":"2024-03-23T14:56:59.659222+0000","last_became_peered":"2024-03-23T14:56:59.659222+0000","last_unstale":"2024-03-23T14:57:00.617783+0000","last_undegraded":"2024-03-23T14:57:00.617783+0000","last_fullsized":"2024-03-23T14:57:00.617783+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:07:26.887157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.519017+0000","last_change":"2024-03-23T14:56:59.659935+0000","last_active":"2024-03-23T14:57:01.519017+0000","last_peered":"2024-03-23T14:57:01.519017+0000","last_clean":"2024-03-23T14:57:01.519017+0000","last_became_active":"2024-03-23T14:56:59.659501+0000","last_became_peered":"2024-03-23T14:56:59.659501+0000","last_unstale":"2024-03-23T14:57:01.519017+0000","last_undegraded":"2024-03-23T14:57:01.519017+0000","last_fullsized":"2024-03-23T14:57:01.519017+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:21:16.376914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.529493+0000","last_change":"2024-03-23T14:56:59.659013+0000","last_active":"2024-03-23T14:57:01.529493+0000","last_peered":"2024-03-23T14:57:01.529493+0000","last_clean":"2024-03-23T14:57:01.529493+0000","last_became_active":"2024-03-23T14:56:59.658667+0000","last_became_peered":"2024-03-23T14:56:59.658667+0000","last_unstale":"2024-03-23T14:57:01.529493+0000","last_undegraded":"2024-03-23T14:57:01.529493+0000","last_fullsized":"2024-03-23T14:57:01.529493+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:12:59.763661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.529535+0000","last_change":"2024-03-23T14:56:59.663729+0000","last_active":"2024-03-23T14:57:01.529535+0000","last_peered":"2024-03-23T14:57:01.529535+0000","last_clean":"2024-03-23T14:57:01.529535+0000","last_became_active":"2024-03-23T14:56:59.663313+0000","last_became_peered":"2024-03-23T14:56:59.663313+0000","last_unstale":"2024-03-23T14:57:01.529535+0000","last_undegraded":"2024-03-23T14:57:01.529535+0000","last_fullsized":"2024-03-23T14:57:01.529535+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:57:06.681361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.619818+0000","last_change":"2024-03-23T14:56:59.640819+0000","last_active":"2024-03-23T14:57:00.619818+0000","last_peered":"2024-03-23T14:57:00.619818+0000","last_clean":"2024-03-23T14:57:00.619818+0000","last_became_active":"2024-03-23T14:56:59.640330+0000","last_became_peered":"2024-03-23T14:56:59.640330+0000","last_unstale":"2024-03-23T14:57:00.619818+0000","last_undegraded":"2024-03-23T14:57:00.619818+0000","last_fullsized":"2024-03-23T14:57:00.619818+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:01:26.168657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.618487+0000","last_change":"2024-03-23T14:56:59.645597+0000","last_active":"2024-03-23T14:57:00.618487+0000","last_peered":"2024-03-23T14:57:00.618487+0000","last_clean":"2024-03-23T14:57:00.618487+0000","last_became_active":"2024-03-23T14:56:59.644700+0000","last_became_peered":"2024-03-23T14:56:59.644700+0000","last_unstale":"2024-03-23T14:57:00.618487+0000","last_undegraded":"2024-03-23T14:57:00.618487+0000","last_fullsized":"2024-03-23T14:57:00.618487+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:59:19.727729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.556462+0000","last_change":"2024-03-23T14:56:59.636241+0000","last_active":"2024-03-23T14:57:01.556462+0000","last_peered":"2024-03-23T14:57:01.556462+0000","last_clean":"2024-03-23T14:57:01.556462+0000","last_became_active":"2024-03-23T14:56:59.635803+0000","last_became_peered":"2024-03-23T14:56:59.635803+0000","last_unstale":"2024-03-23T14:57:01.556462+0000","last_undegraded":"2024-03-23T14:57:01.556462+0000","last_fullsized":"2024-03-23T14:57:01.556462+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:07:26.887157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.620719+0000","last_change":"2024-03-23T14:56:59.642239+0000","last_active":"2024-03-23T14:57:00.620719+0000","last_peered":"2024-03-23T14:57:00.620719+0000","last_clean":"2024-03-23T14:57:00.620719+0000","last_became_active":"2024-03-23T14:56:59.641872+0000","last_became_peered":"2024-03-23T14:56:59.641872+0000","last_unstale":"2024-03-23T14:57:00.620719+0000","last_undegraded":"2024-03-23T14:57:00.620719+0000","last_fullsized":"2024-03-23T14:57:00.620719+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:09:00.536071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.551044+0000","last_change":"2024-03-23T14:56:59.634551+0000","last_active":"2024-03-23T14:57:01.551044+0000","last_peered":"2024-03-23T14:57:01.551044+0000","last_clean":"2024-03-23T14:57:01.551044+0000","last_became_active":"2024-03-23T14:56:59.634255+0000","last_became_peered":"2024-03-23T14:56:59.634255+0000","last_unstale":"2024-03-23T14:57:01.551044+0000","last_undegraded":"2024-03-23T14:57:01.551044+0000","last_fullsized":"2024-03-23T14:57:01.551044+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:39:22.051918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.552128+0000","last_change":"2024-03-23T14:56:59.657535+0000","last_active":"2024-03-23T14:57:01.552128+0000","last_peered":"2024-03-23T14:57:01.552128+0000","last_clean":"2024-03-23T14:57:01.552128+0000","last_became_active":"2024-03-23T14:56:59.656959+0000","last_became_peered":"2024-03-23T14:56:59.656959+0000","last_unstale":"2024-03-23T14:57:01.552128+0000","last_undegraded":"2024-03-23T14:57:01.552128+0000","last_fullsized":"2024-03-23T14:57:01.552128+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:58:31.542399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.555866+0000","last_change":"2024-03-23T14:56:59.639104+0000","last_active":"2024-03-23T14:57:01.555866+0000","last_peered":"2024-03-23T14:57:01.555866+0000","last_clean":"2024-03-23T14:57:01.555866+0000","last_became_active":"2024-03-23T14:56:59.638744+0000","last_became_peered":"2024-03-23T14:56:59.638744+0000","last_unstale":"2024-03-23T14:57:01.555866+0000","last_undegraded":"2024-03-23T14:57:01.555866+0000","last_fullsized":"2024-03-23T14:57:01.555866+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:29:08.139571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.518933+0000","last_change":"2024-03-23T14:56:59.647334+0000","last_active":"2024-03-23T14:57:01.518933+0000","last_peered":"2024-03-23T14:57:01.518933+0000","last_clean":"2024-03-23T14:57:01.518933+0000","last_became_active":"2024-03-23T14:56:59.646801+0000","last_became_peered":"2024-03-23T14:56:59.646801+0000","last_unstale":"2024-03-23T14:57:01.518933+0000","last_undegraded":"2024-03-23T14:57:01.518933+0000","last_fullsized":"2024-03-23T14:57:01.518933+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:02:12.733995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.529380+0000","last_change":"2024-03-23T14:56:59.639076+0000","last_active":"2024-03-23T14:57:01.529380+0000","last_peered":"2024-03-23T14:57:01.529380+0000","last_clean":"2024-03-23T14:57:01.529380+0000","last_became_active":"2024-03-23T14:56:59.638267+0000","last_became_peered":"2024-03-23T14:56:59.638267+0000","last_unstale":"2024-03-23T14:57:01.529380+0000","last_undegraded":"2024-03-23T14:57:01.529380+0000","last_fullsized":"2024-03-23T14:57:01.529380+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:59:19.727729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.556655+0000","last_change":"2024-03-23T14:56:59.647220+0000","last_active":"2024-03-23T14:57:01.556655+0000","last_peered":"2024-03-23T14:57:01.556655+0000","last_clean":"2024-03-23T14:57:01.556655+0000","last_became_active":"2024-03-23T14:56:59.646833+0000","last_became_peered":"2024-03-23T14:56:59.646833+0000","last_unstale":"2024-03-23T14:57:01.556655+0000","last_undegraded":"2024-03-23T14:57:01.556655+0000","last_fullsized":"2024-03-23T14:57:01.556655+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:38:24.739838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.551049+0000","last_change":"2024-03-23T14:56:59.662969+0000","last_active":"2024-03-23T14:57:01.551049+0000","last_peered":"2024-03-23T14:57:01.551049+0000","last_clean":"2024-03-23T14:57:01.551049+0000","last_became_active":"2024-03-23T14:56:59.661554+0000","last_became_peered":"2024-03-23T14:56:59.661554+0000","last_unstale":"2024-03-23T14:57:01.551049+0000","last_undegraded":"2024-03-23T14:57:01.551049+0000","last_fullsized":"2024-03-23T14:57:01.551049+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:52:36.323039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.618010+0000","last_change":"2024-03-23T14:56:59.643866+0000","last_active":"2024-03-23T14:57:00.618010+0000","last_peered":"2024-03-23T14:57:00.618010+0000","last_clean":"2024-03-23T14:57:00.618010+0000","last_became_active":"2024-03-23T14:56:59.643577+0000","last_became_peered":"2024-03-23T14:56:59.643577+0000","last_unstale":"2024-03-23T14:57:00.618010+0000","last_undegraded":"2024-03-23T14:57:00.618010+0000","last_fullsized":"2024-03-23T14:57:00.618010+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:37:56.745366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.550754+0000","last_change":"2024-03-23T14:56:59.651798+0000","last_active":"2024-03-23T14:57:01.550754+0000","last_peered":"2024-03-23T14:57:01.550754+0000","last_clean":"2024-03-23T14:57:01.550754+0000","last_became_active":"2024-03-23T14:56:59.651534+0000","last_became_peered":"2024-03-23T14:56:59.651534+0000","last_unstale":"2024-03-23T14:57:01.550754+0000","last_undegraded":"2024-03-23T14:57:01.550754+0000","last_fullsized":"2024-03-23T14:57:01.550754+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:57:06.681361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.619373+0000","last_change":"2024-03-23T14:56:59.645160+0000","last_active":"2024-03-23T14:57:00.619373+0000","last_peered":"2024-03-23T14:57:00.619373+0000","last_clean":"2024-03-23T14:57:00.619373+0000","last_became_active":"2024-03-23T14:56:59.643402+0000","last_became_peered":"2024-03-23T14:56:59.643402+0000","last_unstale":"2024-03-23T14:57:00.619373+0000","last_undegraded":"2024-03-23T14:57:00.619373+0000","last_fullsized":"2024-03-23T14:57:00.619373+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:35:32.710122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.529573+0000","last_change":"2024-03-23T14:56:59.640718+0000","last_active":"2024-03-23T14:57:01.529573+0000","last_peered":"2024-03-23T14:57:01.529573+0000","last_clean":"2024-03-23T14:57:01.529573+0000","last_became_active":"2024-03-23T14:56:59.640386+0000","last_became_peered":"2024-03-23T14:56:59.640386+0000","last_unstale":"2024-03-23T14:57:01.529573+0000","last_undegraded":"2024-03-23T14:57:01.529573+0000","last_fullsized":"2024-03-23T14:57:01.529573+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:39:22.051918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.621420+0000","last_change":"2024-03-23T14:56:59.650572+0000","last_active":"2024-03-23T14:57:00.621420+0000","last_peered":"2024-03-23T14:57:00.621420+0000","last_clean":"2024-03-23T14:57:00.621420+0000","last_became_active":"2024-03-23T14:56:59.650260+0000","last_became_peered":"2024-03-23T14:56:59.650260+0000","last_unstale":"2024-03-23T14:57:00.621420+0000","last_undegraded":"2024-03-23T14:57:00.621420+0000","last_fullsized":"2024-03-23T14:57:00.621420+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:12:59.763661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.619035+0000","last_change":"2024-03-23T14:56:59.646883+0000","last_active":"2024-03-23T14:57:00.619035+0000","last_peered":"2024-03-23T14:57:00.619035+0000","last_clean":"2024-03-23T14:57:00.619035+0000","last_became_active":"2024-03-23T14:56:59.643427+0000","last_became_peered":"2024-03-23T14:56:59.643427+0000","last_unstale":"2024-03-23T14:57:00.619035+0000","last_undegraded":"2024-03-23T14:57:00.619035+0000","last_fullsized":"2024-03-23T14:57:00.619035+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:52:36.323039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.557036+0000","last_change":"2024-03-23T14:56:59.646524+0000","last_active":"2024-03-23T14:57:01.557036+0000","last_peered":"2024-03-23T14:57:01.557036+0000","last_clean":"2024-03-23T14:57:01.557036+0000","last_became_active":"2024-03-23T14:56:59.646037+0000","last_became_peered":"2024-03-23T14:56:59.646037+0000","last_unstale":"2024-03-23T14:57:01.557036+0000","last_undegraded":"2024-03-23T14:57:01.557036+0000","last_fullsized":"2024-03-23T14:57:01.557036+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:57:06.681361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.551413+0000","last_change":"2024-03-23T14:56:59.658015+0000","last_active":"2024-03-23T14:57:01.551413+0000","last_peered":"2024-03-23T14:57:01.551413+0000","last_clean":"2024-03-23T14:57:01.551413+0000","last_became_active":"2024-03-23T14:56:59.657623+0000","last_became_peered":"2024-03-23T14:56:59.657623+0000","last_unstale":"2024-03-23T14:57:01.551413+0000","last_undegraded":"2024-03-23T14:57:01.551413+0000","last_fullsized":"2024-03-23T14:57:01.551413+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:14:04.996547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.555261+0000","last_change":"2024-03-23T14:56:59.660146+0000","last_active":"2024-03-23T14:57:01.555261+0000","last_peered":"2024-03-23T14:57:01.555261+0000","last_clean":"2024-03-23T14:57:01.555261+0000","last_became_active":"2024-03-23T14:56:59.659765+0000","last_became_peered":"2024-03-23T14:56:59.659765+0000","last_unstale":"2024-03-23T14:57:01.555261+0000","last_undegraded":"2024-03-23T14:57:01.555261+0000","last_fullsized":"2024-03-23T14:57:01.555261+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:49:20.982742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.529104+0000","last_change":"2024-03-23T14:56:59.637937+0000","last_active":"2024-03-23T14:57:01.529104+0000","last_peered":"2024-03-23T14:57:01.529104+0000","last_clean":"2024-03-23T14:57:01.529104+0000","last_became_active":"2024-03-23T14:56:59.637465+0000","last_became_peered":"2024-03-23T14:56:59.637465+0000","last_unstale":"2024-03-23T14:57:01.529104+0000","last_undegraded":"2024-03-23T14:57:01.529104+0000","last_fullsized":"2024-03-23T14:57:01.529104+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:38:24.739838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.619136+0000","last_change":"2024-03-23T14:56:59.633433+0000","last_active":"2024-03-23T14:57:00.619136+0000","last_peered":"2024-03-23T14:57:00.619136+0000","last_clean":"2024-03-23T14:57:00.619136+0000","last_became_active":"2024-03-23T14:56:59.633171+0000","last_became_peered":"2024-03-23T14:56:59.633171+0000","last_unstale":"2024-03-23T14:57:00.619136+0000","last_undegraded":"2024-03-23T14:57:00.619136+0000","last_fullsized":"2024-03-23T14:57:00.619136+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:36:11.384485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.618779+0000","last_change":"2024-03-23T14:56:58.621210+0000","last_active":"2024-03-23T14:57:00.618779+0000","last_peered":"2024-03-23T14:57:00.618779+0000","last_clean":"2024-03-23T14:57:00.618779+0000","last_became_active":"2024-03-23T14:56:58.620750+0000","last_became_peered":"2024-03-23T14:56:58.620750+0000","last_unstale":"2024-03-23T14:57:00.618779+0000","last_undegraded":"2024-03-23T14:57:00.618779+0000","last_fullsized":"2024-03-23T14:57:00.618779+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:24:18.799175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.556969+0000","last_change":"2024-03-23T14:56:59.633767+0000","last_active":"2024-03-23T14:57:01.556969+0000","last_peered":"2024-03-23T14:57:01.556969+0000","last_clean":"2024-03-23T14:57:01.556969+0000","last_became_active":"2024-03-23T14:56:59.633390+0000","last_became_peered":"2024-03-23T14:56:59.633390+0000","last_unstale":"2024-03-23T14:57:01.556969+0000","last_undegraded":"2024-03-23T14:57:01.556969+0000","last_fullsized":"2024-03-23T14:57:01.556969+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:12:59.763661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.529138+0000","last_change":"2024-03-23T14:56:58.613611+0000","last_active":"2024-03-23T14:57:01.529138+0000","last_peered":"2024-03-23T14:57:01.529138+0000","last_clean":"2024-03-23T14:57:01.529138+0000","last_became_active":"2024-03-23T14:56:58.608339+0000","last_became_peered":"2024-03-23T14:56:58.608339+0000","last_unstale":"2024-03-23T14:57:01.529138+0000","last_undegraded":"2024-03-23T14:57:01.529138+0000","last_fullsized":"2024-03-23T14:57:01.529138+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:48:12.389105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.618094+0000","last_change":"2024-03-23T14:56:58.621716+0000","last_active":"2024-03-23T14:57:00.618094+0000","last_peered":"2024-03-23T14:57:00.618094+0000","last_clean":"2024-03-23T14:57:00.618094+0000","last_became_active":"2024-03-23T14:56:58.620675+0000","last_became_peered":"2024-03-23T14:56:58.620675+0000","last_unstale":"2024-03-23T14:57:00.618094+0000","last_undegraded":"2024-03-23T14:57:00.618094+0000","last_fullsized":"2024-03-23T14:57:00.618094+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:07:10.915534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.617954+0000","last_change":"2024-03-23T14:56:59.644784+0000","last_active":"2024-03-23T14:57:00.617954+0000","last_peered":"2024-03-23T14:57:00.617954+0000","last_clean":"2024-03-23T14:57:00.617954+0000","last_became_active":"2024-03-23T14:56:59.644312+0000","last_became_peered":"2024-03-23T14:56:59.644312+0000","last_unstale":"2024-03-23T14:57:00.617954+0000","last_undegraded":"2024-03-23T14:57:00.617954+0000","last_fullsized":"2024-03-23T14:57:00.617954+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:38:24.739838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.550888+0000","last_change":"2024-03-23T14:56:58.611793+0000","last_active":"2024-03-23T14:57:01.550888+0000","last_peered":"2024-03-23T14:57:01.550888+0000","last_clean":"2024-03-23T14:57:01.550888+0000","last_became_active":"2024-03-23T14:56:58.611131+0000","last_became_peered":"2024-03-23T14:56:58.611131+0000","last_unstale":"2024-03-23T14:57:01.550888+0000","last_undegraded":"2024-03-23T14:57:01.550888+0000","last_fullsized":"2024-03-23T14:57:01.550888+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:36:10.376200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.620461+0000","last_change":"2024-03-23T14:56:59.649256+0000","last_active":"2024-03-23T14:57:00.620461+0000","last_peered":"2024-03-23T14:57:00.620461+0000","last_clean":"2024-03-23T14:57:00.620461+0000","last_became_active":"2024-03-23T14:56:59.648957+0000","last_became_peered":"2024-03-23T14:56:59.648957+0000","last_unstale":"2024-03-23T14:57:00.620461+0000","last_undegraded":"2024-03-23T14:57:00.620461+0000","last_fullsized":"2024-03-23T14:57:00.620461+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:07:26.887157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.529168+0000","last_change":"2024-03-23T14:56:58.630337+0000","last_active":"2024-03-23T14:57:01.529168+0000","last_peered":"2024-03-23T14:57:01.529168+0000","last_clean":"2024-03-23T14:57:01.529168+0000","last_became_active":"2024-03-23T14:56:58.629899+0000","last_became_peered":"2024-03-23T14:56:58.629899+0000","last_unstale":"2024-03-23T14:57:01.529168+0000","last_undegraded":"2024-03-23T14:57:01.529168+0000","last_fullsized":"2024-03-23T14:57:01.529168+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:24:18.799175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.551507+0000","last_change":"2024-03-23T14:56:59.657632+0000","last_active":"2024-03-23T14:57:01.551507+0000","last_peered":"2024-03-23T14:57:01.551507+0000","last_clean":"2024-03-23T14:57:01.551507+0000","last_became_active":"2024-03-23T14:56:59.655136+0000","last_became_peered":"2024-03-23T14:56:59.655136+0000","last_unstale":"2024-03-23T14:57:01.551507+0000","last_undegraded":"2024-03-23T14:57:01.551507+0000","last_fullsized":"2024-03-23T14:57:01.551507+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:46:57.866240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.551534+0000","last_change":"2024-03-23T14:56:58.627359+0000","last_active":"2024-03-23T14:57:01.551534+0000","last_peered":"2024-03-23T14:57:01.551534+0000","last_clean":"2024-03-23T14:57:01.551534+0000","last_became_active":"2024-03-23T14:56:58.626614+0000","last_became_peered":"2024-03-23T14:56:58.626614+0000","last_unstale":"2024-03-23T14:57:01.551534+0000","last_undegraded":"2024-03-23T14:57:01.551534+0000","last_fullsized":"2024-03-23T14:57:01.551534+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:05:00.726517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.556310+0000","last_change":"2024-03-23T14:56:59.659806+0000","last_active":"2024-03-23T14:57:01.556310+0000","last_peered":"2024-03-23T14:57:01.556310+0000","last_clean":"2024-03-23T14:57:01.556310+0000","last_became_active":"2024-03-23T14:56:59.658983+0000","last_became_peered":"2024-03-23T14:56:59.658983+0000","last_unstale":"2024-03-23T14:57:01.556310+0000","last_undegraded":"2024-03-23T14:57:01.556310+0000","last_fullsized":"2024-03-23T14:57:01.556310+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:58:29.591659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.621558+0000","last_change":"2024-03-23T14:56:58.625877+0000","last_active":"2024-03-23T14:57:00.621558+0000","last_peered":"2024-03-23T14:57:00.621558+0000","last_clean":"2024-03-23T14:57:00.621558+0000","last_became_active":"2024-03-23T14:56:58.625410+0000","last_became_peered":"2024-03-23T14:56:58.625410+0000","last_unstale":"2024-03-23T14:57:00.621558+0000","last_undegraded":"2024-03-23T14:57:00.621558+0000","last_fullsized":"2024-03-23T14:57:00.621558+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:05:00.726517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.619847+0000","last_change":"2024-03-23T14:56:59.647811+0000","last_active":"2024-03-23T14:57:00.619847+0000","last_peered":"2024-03-23T14:57:00.619847+0000","last_clean":"2024-03-23T14:57:00.619847+0000","last_became_active":"2024-03-23T14:56:59.647138+0000","last_became_peered":"2024-03-23T14:56:59.647138+0000","last_unstale":"2024-03-23T14:57:00.619847+0000","last_undegraded":"2024-03-23T14:57:00.619847+0000","last_fullsized":"2024-03-23T14:57:00.619847+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:49:20.982742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.529463+0000","last_change":"2024-03-23T14:56:58.630507+0000","last_active":"2024-03-23T14:57:01.529463+0000","last_peered":"2024-03-23T14:57:01.529463+0000","last_clean":"2024-03-23T14:57:01.529463+0000","last_became_active":"2024-03-23T14:56:58.629991+0000","last_became_peered":"2024-03-23T14:56:58.629991+0000","last_unstale":"2024-03-23T14:57:01.529463+0000","last_undegraded":"2024-03-23T14:57:01.529463+0000","last_fullsized":"2024-03-23T14:57:01.529463+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:01:25.160372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.518629+0000","last_change":"2024-03-23T14:56:59.649839+0000","last_active":"2024-03-23T14:57:01.518629+0000","last_peered":"2024-03-23T14:57:01.518629+0000","last_clean":"2024-03-23T14:57:01.518629+0000","last_became_active":"2024-03-23T14:56:59.648333+0000","last_became_peered":"2024-03-23T14:56:59.648333+0000","last_unstale":"2024-03-23T14:57:01.518629+0000","last_undegraded":"2024-03-23T14:57:01.518629+0000","last_fullsized":"2024-03-23T14:57:01.518629+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:46:57.866240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.557119+0000","last_change":"2024-03-23T14:56:58.609669+0000","last_active":"2024-03-23T14:57:01.557119+0000","last_peered":"2024-03-23T14:57:01.557119+0000","last_clean":"2024-03-23T14:57:01.557119+0000","last_became_active":"2024-03-23T14:56:58.608906+0000","last_became_peered":"2024-03-23T14:56:58.608906+0000","last_unstale":"2024-03-23T14:57:01.557119+0000","last_undegraded":"2024-03-23T14:57:01.557119+0000","last_fullsized":"2024-03-23T14:57:01.557119+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:34:51.972168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.556854+0000","last_change":"2024-03-23T14:56:59.648070+0000","last_active":"2024-03-23T14:57:01.556854+0000","last_peered":"2024-03-23T14:57:01.556854+0000","last_clean":"2024-03-23T14:57:01.556854+0000","last_became_active":"2024-03-23T14:56:59.647803+0000","last_became_peered":"2024-03-23T14:56:59.647803+0000","last_unstale":"2024-03-23T14:57:01.556854+0000","last_undegraded":"2024-03-23T14:57:01.556854+0000","last_fullsized":"2024-03-23T14:57:01.556854+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:37:56.745366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.551552+0000","last_change":"2024-03-23T14:56:58.626829+0000","last_active":"2024-03-23T14:57:01.551552+0000","last_peered":"2024-03-23T14:57:01.551552+0000","last_clean":"2024-03-23T14:57:01.551552+0000","last_became_active":"2024-03-23T14:56:58.626375+0000","last_became_peered":"2024-03-23T14:56:58.626375+0000","last_unstale":"2024-03-23T14:57:01.551552+0000","last_undegraded":"2024-03-23T14:57:01.551552+0000","last_fullsized":"2024-03-23T14:57:01.551552+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:48:12.389105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.518496+0000","last_change":"2024-03-23T14:56:59.653570+0000","last_active":"2024-03-23T14:57:01.518496+0000","last_peered":"2024-03-23T14:57:01.518496+0000","last_clean":"2024-03-23T14:57:01.518496+0000","last_became_active":"2024-03-23T14:56:59.653284+0000","last_became_peered":"2024-03-23T14:56:59.653284+0000","last_unstale":"2024-03-23T14:57:01.518496+0000","last_undegraded":"2024-03-23T14:57:01.518496+0000","last_fullsized":"2024-03-23T14:57:01.518496+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:34:30.083232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.528879+0000","last_change":"2024-03-23T14:56:58.614950+0000","last_active":"2024-03-23T14:57:01.528879+0000","last_peered":"2024-03-23T14:57:01.528879+0000","last_clean":"2024-03-23T14:57:01.528879+0000","last_became_active":"2024-03-23T14:56:58.614441+0000","last_became_peered":"2024-03-23T14:56:58.614441+0000","last_unstale":"2024-03-23T14:57:01.528879+0000","last_undegraded":"2024-03-23T14:57:01.528879+0000","last_fullsized":"2024-03-23T14:57:01.528879+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:05:00.726517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.528950+0000","last_change":"2024-03-23T14:56:59.636998+0000","last_active":"2024-03-23T14:57:01.528950+0000","last_peered":"2024-03-23T14:57:01.528950+0000","last_clean":"2024-03-23T14:57:01.528950+0000","last_became_active":"2024-03-23T14:56:59.636444+0000","last_became_peered":"2024-03-23T14:56:59.636444+0000","last_unstale":"2024-03-23T14:57:01.528950+0000","last_undegraded":"2024-03-23T14:57:01.528950+0000","last_fullsized":"2024-03-23T14:57:01.528950+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:09:00.536071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.555118+0000","last_change":"2024-03-23T14:56:58.604409+0000","last_active":"2024-03-23T14:57:01.555118+0000","last_peered":"2024-03-23T14:57:01.555118+0000","last_clean":"2024-03-23T14:57:01.555118+0000","last_became_active":"2024-03-23T14:56:58.604175+0000","last_became_peered":"2024-03-23T14:56:58.604175+0000","last_unstale":"2024-03-23T14:57:01.555118+0000","last_undegraded":"2024-03-23T14:57:01.555118+0000","last_fullsized":"2024-03-23T14:57:01.555118+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:07:10.915534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.518264+0000","last_change":"2024-03-23T14:56:59.651120+0000","last_active":"2024-03-23T14:57:01.518264+0000","last_peered":"2024-03-23T14:57:01.518264+0000","last_clean":"2024-03-23T14:57:01.518264+0000","last_became_active":"2024-03-23T14:56:59.650822+0000","last_became_peered":"2024-03-23T14:56:59.650822+0000","last_unstale":"2024-03-23T14:57:01.518264+0000","last_undegraded":"2024-03-23T14:57:01.518264+0000","last_fullsized":"2024-03-23T14:57:01.518264+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:35:26.329408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.619806+0000","last_change":"2024-03-23T14:56:58.606372+0000","last_active":"2024-03-23T14:57:00.619806+0000","last_peered":"2024-03-23T14:57:00.619806+0000","last_clean":"2024-03-23T14:57:00.619806+0000","last_became_active":"2024-03-23T14:56:58.606011+0000","last_became_peered":"2024-03-23T14:56:58.606011+0000","last_unstale":"2024-03-23T14:57:00.619806+0000","last_undegraded":"2024-03-23T14:57:00.619806+0000","last_fullsized":"2024-03-23T14:57:00.619806+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:34:51.972168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.518669+0000","last_change":"2024-03-23T14:56:59.652521+0000","last_active":"2024-03-23T14:57:01.518669+0000","last_peered":"2024-03-23T14:57:01.518669+0000","last_clean":"2024-03-23T14:57:01.518669+0000","last_became_active":"2024-03-23T14:56:59.652270+0000","last_became_peered":"2024-03-23T14:56:59.652270+0000","last_unstale":"2024-03-23T14:57:01.518669+0000","last_undegraded":"2024-03-23T14:57:01.518669+0000","last_fullsized":"2024-03-23T14:57:01.518669+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:51:59.393587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.556006+0000","last_change":"2024-03-23T14:56:58.610985+0000","last_active":"2024-03-23T14:57:01.556006+0000","last_peered":"2024-03-23T14:57:01.556006+0000","last_clean":"2024-03-23T14:57:01.556006+0000","last_became_active":"2024-03-23T14:56:58.610717+0000","last_became_peered":"2024-03-23T14:56:58.610717+0000","last_unstale":"2024-03-23T14:57:01.556006+0000","last_undegraded":"2024-03-23T14:57:01.556006+0000","last_fullsized":"2024-03-23T14:57:01.556006+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:24:18.799175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.519128+0000","last_change":"2024-03-23T14:56:59.652993+0000","last_active":"2024-03-23T14:57:01.519128+0000","last_peered":"2024-03-23T14:57:01.519128+0000","last_clean":"2024-03-23T14:57:01.519128+0000","last_became_active":"2024-03-23T14:56:59.652718+0000","last_became_peered":"2024-03-23T14:56:59.652718+0000","last_unstale":"2024-03-23T14:57:01.519128+0000","last_undegraded":"2024-03-23T14:57:01.519128+0000","last_fullsized":"2024-03-23T14:57:01.519128+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:06:34.693789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.621685+0000","last_change":"2024-03-23T14:56:58.608418+0000","last_active":"2024-03-23T14:57:00.621685+0000","last_peered":"2024-03-23T14:57:00.621685+0000","last_clean":"2024-03-23T14:57:00.621685+0000","last_became_active":"2024-03-23T14:56:58.607992+0000","last_became_peered":"2024-03-23T14:56:58.607992+0000","last_unstale":"2024-03-23T14:57:00.621685+0000","last_undegraded":"2024-03-23T14:57:00.621685+0000","last_fullsized":"2024-03-23T14:57:00.621685+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:36:10.376200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.551352+0000","last_change":"2024-03-23T14:56:59.657882+0000","last_active":"2024-03-23T14:57:01.551352+0000","last_peered":"2024-03-23T14:57:01.551352+0000","last_clean":"2024-03-23T14:57:01.551352+0000","last_became_active":"2024-03-23T14:56:59.657004+0000","last_became_peered":"2024-03-23T14:56:59.657004+0000","last_unstale":"2024-03-23T14:57:01.551352+0000","last_undegraded":"2024-03-23T14:57:01.551352+0000","last_fullsized":"2024-03-23T14:57:01.551352+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:58:40.996298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.557066+0000","last_change":"2024-03-23T14:56:58.617074+0000","last_active":"2024-03-23T14:57:01.557066+0000","last_peered":"2024-03-23T14:57:01.557066+0000","last_clean":"2024-03-23T14:57:01.557066+0000","last_became_active":"2024-03-23T14:56:58.616732+0000","last_became_peered":"2024-03-23T14:56:58.616732+0000","last_unstale":"2024-03-23T14:57:01.557066+0000","last_undegraded":"2024-03-23T14:57:01.557066+0000","last_fullsized":"2024-03-23T14:57:01.557066+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:05:00.726517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.518726+0000","last_change":"2024-03-23T14:56:59.664625+0000","last_active":"2024-03-23T14:57:01.518726+0000","last_peered":"2024-03-23T14:57:01.518726+0000","last_clean":"2024-03-23T14:57:01.518726+0000","last_became_active":"2024-03-23T14:56:59.664268+0000","last_became_peered":"2024-03-23T14:56:59.664268+0000","last_unstale":"2024-03-23T14:57:01.518726+0000","last_undegraded":"2024-03-23T14:57:01.518726+0000","last_fullsized":"2024-03-23T14:57:01.518726+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:53:55.152886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.517881+0000","last_change":"2024-03-23T14:56:48.722063+0000","last_active":"2024-03-23T14:57:01.517881+0000","last_peered":"2024-03-23T14:57:01.517881+0000","last_clean":"2024-03-23T14:57:01.517881+0000","last_became_active":"2024-03-23T14:56:48.720079+0000","last_became_peered":"2024-03-23T14:56:48.720079+0000","last_unstale":"2024-03-23T14:57:01.517881+0000","last_undegraded":"2024-03-23T14:57:01.517881+0000","last_fullsized":"2024-03-23T14:57:01.517881+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:47.478078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:47.478078+0000","last_clean_scrub_stamp":"2024-03-23T14:56:47.478078+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:55:51.231901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.551138+0000","last_change":"2024-03-23T14:56:58.611796+0000","last_active":"2024-03-23T14:57:01.551138+0000","last_peered":"2024-03-23T14:57:01.551138+0000","last_clean":"2024-03-23T14:57:01.551138+0000","last_became_active":"2024-03-23T14:56:58.611243+0000","last_became_peered":"2024-03-23T14:56:58.611243+0000","last_unstale":"2024-03-23T14:57:01.551138+0000","last_undegraded":"2024-03-23T14:57:01.551138+0000","last_fullsized":"2024-03-23T14:57:01.551138+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:24:18.799175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.618598+0000","last_change":"2024-03-23T14:56:59.664611+0000","last_active":"2024-03-23T14:57:00.618598+0000","last_peered":"2024-03-23T14:57:00.618598+0000","last_clean":"2024-03-23T14:57:00.618598+0000","last_became_active":"2024-03-23T14:56:59.664275+0000","last_became_peered":"2024-03-23T14:56:59.664275+0000","last_unstale":"2024-03-23T14:57:00.618598+0000","last_undegraded":"2024-03-23T14:57:00.618598+0000","last_fullsized":"2024-03-23T14:57:00.618598+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:12:59.763661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.517853+0000","last_change":"2024-03-23T14:56:58.620960+0000","last_active":"2024-03-23T14:57:01.517853+0000","last_peered":"2024-03-23T14:57:01.517853+0000","last_clean":"2024-03-23T14:57:01.517853+0000","last_became_active":"2024-03-23T14:56:58.620592+0000","last_became_peered":"2024-03-23T14:56:58.620592+0000","last_unstale":"2024-03-23T14:57:01.517853+0000","last_undegraded":"2024-03-23T14:57:01.517853+0000","last_fullsized":"2024-03-23T14:57:01.517853+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:01:25.160372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.556871+0000","last_change":"2024-03-23T14:56:59.621260+0000","last_active":"2024-03-23T14:57:01.556871+0000","last_peered":"2024-03-23T14:57:01.556871+0000","last_clean":"2024-03-23T14:57:01.556871+0000","last_became_active":"2024-03-23T14:56:59.620924+0000","last_became_peered":"2024-03-23T14:56:59.620924+0000","last_unstale":"2024-03-23T14:57:01.556871+0000","last_undegraded":"2024-03-23T14:57:01.556871+0000","last_fullsized":"2024-03-23T14:57:01.556871+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:59:19.727729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.555522+0000","last_change":"2024-03-23T14:56:58.611902+0000","last_active":"2024-03-23T14:57:01.555522+0000","last_peered":"2024-03-23T14:57:01.555522+0000","last_clean":"2024-03-23T14:57:01.555522+0000","last_became_active":"2024-03-23T14:56:58.611208+0000","last_became_peered":"2024-03-23T14:56:58.611208+0000","last_unstale":"2024-03-23T14:57:01.555522+0000","last_undegraded":"2024-03-23T14:57:01.555522+0000","last_fullsized":"2024-03-23T14:57:01.555522+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:36:10.376200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.620759+0000","last_change":"2024-03-23T14:56:59.640818+0000","last_active":"2024-03-23T14:57:00.620759+0000","last_peered":"2024-03-23T14:57:00.620759+0000","last_clean":"2024-03-23T14:57:00.620759+0000","last_became_active":"2024-03-23T14:56:59.640493+0000","last_became_peered":"2024-03-23T14:56:59.640493+0000","last_unstale":"2024-03-23T14:57:00.620759+0000","last_undegraded":"2024-03-23T14:57:00.620759+0000","last_fullsized":"2024-03-23T14:57:00.620759+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:38:24.739838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.551684+0000","last_change":"2024-03-23T14:56:58.616782+0000","last_active":"2024-03-23T14:57:01.551684+0000","last_peered":"2024-03-23T14:57:01.551684+0000","last_clean":"2024-03-23T14:57:01.551684+0000","last_became_active":"2024-03-23T14:56:58.615957+0000","last_became_peered":"2024-03-23T14:56:58.615957+0000","last_unstale":"2024-03-23T14:57:01.551684+0000","last_undegraded":"2024-03-23T14:57:01.551684+0000","last_fullsized":"2024-03-23T14:57:01.551684+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:01:25.160372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.619632+0000","last_change":"2024-03-23T14:56:59.623480+0000","last_active":"2024-03-23T14:57:00.619632+0000","last_peered":"2024-03-23T14:57:00.619632+0000","last_clean":"2024-03-23T14:57:00.619632+0000","last_became_active":"2024-03-23T14:56:59.623224+0000","last_became_peered":"2024-03-23T14:56:59.623224+0000","last_unstale":"2024-03-23T14:57:00.619632+0000","last_undegraded":"2024-03-23T14:57:00.619632+0000","last_fullsized":"2024-03-23T14:57:00.619632+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:48:13.397390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.621161+0000","last_change":"2024-03-23T14:56:58.604398+0000","last_active":"2024-03-23T14:57:00.621161+0000","last_peered":"2024-03-23T14:57:00.621161+0000","last_clean":"2024-03-23T14:57:00.621161+0000","last_became_active":"2024-03-23T14:56:58.603192+0000","last_became_peered":"2024-03-23T14:56:58.603192+0000","last_unstale":"2024-03-23T14:57:00.621161+0000","last_undegraded":"2024-03-23T14:57:00.621161+0000","last_fullsized":"2024-03-23T14:57:00.621161+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:24:18.799175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.551563+0000","last_change":"2024-03-23T14:56:59.661728+0000","last_active":"2024-03-23T14:57:01.551563+0000","last_peered":"2024-03-23T14:57:01.551563+0000","last_clean":"2024-03-23T14:57:01.551563+0000","last_became_active":"2024-03-23T14:56:59.656955+0000","last_became_peered":"2024-03-23T14:56:59.656955+0000","last_unstale":"2024-03-23T14:57:01.551563+0000","last_undegraded":"2024-03-23T14:57:01.551563+0000","last_fullsized":"2024-03-23T14:57:01.551563+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:51:59.393587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.518923+0000","last_change":"2024-03-23T14:56:58.619283+0000","last_active":"2024-03-23T14:57:01.518923+0000","last_peered":"2024-03-23T14:57:01.518923+0000","last_clean":"2024-03-23T14:57:01.518923+0000","last_became_active":"2024-03-23T14:56:58.618922+0000","last_became_peered":"2024-03-23T14:56:58.618922+0000","last_unstale":"2024-03-23T14:57:01.518923+0000","last_undegraded":"2024-03-23T14:57:01.518923+0000","last_fullsized":"2024-03-23T14:57:01.518923+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:29:07.131287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.620920+0000","last_change":"2024-03-23T14:56:59.636699+0000","last_active":"2024-03-23T14:57:00.620920+0000","last_peered":"2024-03-23T14:57:00.620920+0000","last_clean":"2024-03-23T14:57:00.620920+0000","last_became_active":"2024-03-23T14:56:59.636376+0000","last_became_peered":"2024-03-23T14:56:59.636376+0000","last_unstale":"2024-03-23T14:57:00.620920+0000","last_undegraded":"2024-03-23T14:57:00.620920+0000","last_fullsized":"2024-03-23T14:57:00.620920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:37:56.745366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.622171+0000","last_change":"2024-03-23T14:56:58.627211+0000","last_active":"2024-03-23T14:57:01.622171+0000","last_peered":"2024-03-23T14:57:01.622171+0000","last_clean":"2024-03-23T14:57:01.622171+0000","last_became_active":"2024-03-23T14:56:58.626779+0000","last_became_peered":"2024-03-23T14:56:58.626779+0000","last_unstale":"2024-03-23T14:57:01.622171+0000","last_undegraded":"2024-03-23T14:57:01.622171+0000","last_fullsized":"2024-03-23T14:57:01.622171+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:01:25.160372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.619749+0000","last_change":"2024-03-23T14:56:59.643640+0000","last_active":"2024-03-23T14:57:00.619749+0000","last_peered":"2024-03-23T14:57:00.619749+0000","last_clean":"2024-03-23T14:57:00.619749+0000","last_became_active":"2024-03-23T14:56:59.643255+0000","last_became_peered":"2024-03-23T14:56:59.643255+0000","last_unstale":"2024-03-23T14:57:00.619749+0000","last_undegraded":"2024-03-23T14:57:00.619749+0000","last_fullsized":"2024-03-23T14:57:00.619749+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:35:26.329408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.529191+0000","last_change":"2024-03-23T14:56:58.623880+0000","last_active":"2024-03-23T14:57:01.529191+0000","last_peered":"2024-03-23T14:57:01.529191+0000","last_clean":"2024-03-23T14:57:01.529191+0000","last_became_active":"2024-03-23T14:56:58.623291+0000","last_became_peered":"2024-03-23T14:56:58.623291+0000","last_unstale":"2024-03-23T14:57:01.529191+0000","last_undegraded":"2024-03-23T14:57:01.529191+0000","last_fullsized":"2024-03-23T14:57:01.529191+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:36:10.376200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.556526+0000","last_change":"2024-03-23T14:56:59.634446+0000","last_active":"2024-03-23T14:57:01.556526+0000","last_peered":"2024-03-23T14:57:01.556526+0000","last_clean":"2024-03-23T14:57:01.556526+0000","last_became_active":"2024-03-23T14:56:59.633946+0000","last_became_peered":"2024-03-23T14:56:59.633946+0000","last_unstale":"2024-03-23T14:57:01.556526+0000","last_undegraded":"2024-03-23T14:57:01.556526+0000","last_fullsized":"2024-03-23T14:57:01.556526+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:09:00.536071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.517807+0000","last_change":"2024-03-23T14:56:58.618776+0000","last_active":"2024-03-23T14:57:01.517807+0000","last_peered":"2024-03-23T14:57:01.517807+0000","last_clean":"2024-03-23T14:57:01.517807+0000","last_became_active":"2024-03-23T14:56:58.618418+0000","last_became_peered":"2024-03-23T14:56:58.618418+0000","last_unstale":"2024-03-23T14:57:01.517807+0000","last_undegraded":"2024-03-23T14:57:01.517807+0000","last_fullsized":"2024-03-23T14:57:01.517807+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:48:12.389105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.619949+0000","last_change":"2024-03-23T14:56:59.633845+0000","last_active":"2024-03-23T14:57:00.619949+0000","last_peered":"2024-03-23T14:57:00.619949+0000","last_clean":"2024-03-23T14:57:00.619949+0000","last_became_active":"2024-03-23T14:56:59.633409+0000","last_became_peered":"2024-03-23T14:56:59.633409+0000","last_unstale":"2024-03-23T14:57:00.619949+0000","last_undegraded":"2024-03-23T14:57:00.619949+0000","last_fullsized":"2024-03-23T14:57:00.619949+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:29:08.139571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.620668+0000","last_change":"2024-03-23T14:56:58.616951+0000","last_active":"2024-03-23T14:57:00.620668+0000","last_peered":"2024-03-23T14:57:00.620668+0000","last_clean":"2024-03-23T14:57:00.620668+0000","last_became_active":"2024-03-23T14:56:58.616383+0000","last_became_peered":"2024-03-23T14:56:58.616383+0000","last_unstale":"2024-03-23T14:57:00.620668+0000","last_undegraded":"2024-03-23T14:57:00.620668+0000","last_fullsized":"2024-03-23T14:57:00.620668+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:48:12.389105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.617942+0000","last_change":"2024-03-23T14:56:59.645384+0000","last_active":"2024-03-23T14:57:00.617942+0000","last_peered":"2024-03-23T14:57:00.617942+0000","last_clean":"2024-03-23T14:57:00.617942+0000","last_became_active":"2024-03-23T14:56:59.643723+0000","last_became_peered":"2024-03-23T14:56:59.643723+0000","last_unstale":"2024-03-23T14:57:00.617942+0000","last_undegraded":"2024-03-23T14:57:00.617942+0000","last_fullsized":"2024-03-23T14:57:00.617942+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:09:00.536071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.620676+0000","last_change":"2024-03-23T14:56:58.622814+0000","last_active":"2024-03-23T14:57:00.620676+0000","last_peered":"2024-03-23T14:57:00.620676+0000","last_clean":"2024-03-23T14:57:00.620676+0000","last_became_active":"2024-03-23T14:56:58.621745+0000","last_became_peered":"2024-03-23T14:56:58.621745+0000","last_unstale":"2024-03-23T14:57:00.620676+0000","last_undegraded":"2024-03-23T14:57:00.620676+0000","last_fullsized":"2024-03-23T14:57:00.620676+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:49:19.974457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.618866+0000","last_change":"2024-03-23T14:56:59.645315+0000","last_active":"2024-03-23T14:57:00.618866+0000","last_peered":"2024-03-23T14:57:00.618866+0000","last_clean":"2024-03-23T14:57:00.618866+0000","last_became_active":"2024-03-23T14:56:59.644287+0000","last_became_peered":"2024-03-23T14:56:59.644287+0000","last_unstale":"2024-03-23T14:57:00.618866+0000","last_undegraded":"2024-03-23T14:57:00.618866+0000","last_fullsized":"2024-03-23T14:57:00.618866+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:57:06.681361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.518700+0000","last_change":"2024-03-23T14:56:58.625848+0000","last_active":"2024-03-23T14:57:01.518700+0000","last_peered":"2024-03-23T14:57:01.518700+0000","last_clean":"2024-03-23T14:57:01.518700+0000","last_became_active":"2024-03-23T14:56:58.625453+0000","last_became_peered":"2024-03-23T14:56:58.625453+0000","last_unstale":"2024-03-23T14:57:01.518700+0000","last_undegraded":"2024-03-23T14:57:01.518700+0000","last_fullsized":"2024-03-23T14:57:01.518700+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:06:05.434722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.529162+0000","last_change":"2024-03-23T14:56:59.635812+0000","last_active":"2024-03-23T14:57:01.529162+0000","last_peered":"2024-03-23T14:57:01.529162+0000","last_clean":"2024-03-23T14:57:01.529162+0000","last_became_active":"2024-03-23T14:56:59.635519+0000","last_became_peered":"2024-03-23T14:56:59.635519+0000","last_unstale":"2024-03-23T14:57:01.529162+0000","last_undegraded":"2024-03-23T14:57:01.529162+0000","last_fullsized":"2024-03-23T14:57:01.529162+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:37:56.745366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.551417+0000","last_change":"2024-03-23T14:56:59.662434+0000","last_active":"2024-03-23T14:57:01.551417+0000","last_peered":"2024-03-23T14:57:01.551417+0000","last_clean":"2024-03-23T14:57:01.551417+0000","last_became_active":"2024-03-23T14:56:59.661514+0000","last_became_peered":"2024-03-23T14:56:59.661514+0000","last_unstale":"2024-03-23T14:57:01.551417+0000","last_undegraded":"2024-03-23T14:57:01.551417+0000","last_fullsized":"2024-03-23T14:57:01.551417+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:34:30.083232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.518270+0000","last_change":"2024-03-23T14:56:58.629744+0000","last_active":"2024-03-23T14:57:01.518270+0000","last_peered":"2024-03-23T14:57:01.518270+0000","last_clean":"2024-03-23T14:57:01.518270+0000","last_became_active":"2024-03-23T14:56:58.629401+0000","last_became_peered":"2024-03-23T14:56:58.629401+0000","last_unstale":"2024-03-23T14:57:01.518270+0000","last_undegraded":"2024-03-23T14:57:01.518270+0000","last_fullsized":"2024-03-23T14:57:01.518270+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:13:45.673576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.551735+0000","last_change":"2024-03-23T14:56:59.658250+0000","last_active":"2024-03-23T14:57:01.551735+0000","last_peered":"2024-03-23T14:57:01.551735+0000","last_clean":"2024-03-23T14:57:01.551735+0000","last_became_active":"2024-03-23T14:56:59.657085+0000","last_became_peered":"2024-03-23T14:56:59.657085+0000","last_unstale":"2024-03-23T14:57:01.551735+0000","last_undegraded":"2024-03-23T14:57:01.551735+0000","last_fullsized":"2024-03-23T14:57:01.551735+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:53:55.152886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.518837+0000","last_change":"2024-03-23T14:56:58.624886+0000","last_active":"2024-03-23T14:57:01.518837+0000","last_peered":"2024-03-23T14:57:01.518837+0000","last_clean":"2024-03-23T14:57:01.518837+0000","last_became_active":"2024-03-23T14:56:58.624478+0000","last_became_peered":"2024-03-23T14:56:58.624478+0000","last_unstale":"2024-03-23T14:57:01.518837+0000","last_undegraded":"2024-03-23T14:57:01.518837+0000","last_fullsized":"2024-03-23T14:57:01.518837+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:49:19.974457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.555868+0000","last_change":"2024-03-23T14:56:59.660089+0000","last_active":"2024-03-23T14:57:01.555868+0000","last_peered":"2024-03-23T14:57:01.555868+0000","last_clean":"2024-03-23T14:57:01.555868+0000","last_became_active":"2024-03-23T14:56:59.659469+0000","last_became_peered":"2024-03-23T14:56:59.659469+0000","last_unstale":"2024-03-23T14:57:01.555868+0000","last_undegraded":"2024-03-23T14:57:01.555868+0000","last_fullsized":"2024-03-23T14:57:01.555868+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:37:14.115808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.619024+0000","last_change":"2024-03-23T14:56:58.608147+0000","last_active":"2024-03-23T14:57:00.619024+0000","last_peered":"2024-03-23T14:57:00.619024+0000","last_clean":"2024-03-23T14:57:00.619024+0000","last_became_active":"2024-03-23T14:56:58.603630+0000","last_became_peered":"2024-03-23T14:56:58.603630+0000","last_unstale":"2024-03-23T14:57:00.619024+0000","last_undegraded":"2024-03-23T14:57:00.619024+0000","last_fullsized":"2024-03-23T14:57:00.619024+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:08:31.251287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:00.618985+0000","last_change":"2024-03-23T14:56:59.645116+0000","last_active":"2024-03-23T14:57:00.618985+0000","last_peered":"2024-03-23T14:57:00.618985+0000","last_clean":"2024-03-23T14:57:00.618985+0000","last_became_active":"2024-03-23T14:56:59.644484+0000","last_became_peered":"2024-03-23T14:56:59.644484+0000","last_unstale":"2024-03-23T14:57:00.618985+0000","last_undegraded":"2024-03-23T14:57:00.618985+0000","last_fullsized":"2024-03-23T14:57:00.618985+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:58.583412+0000","last_clean_scrub_stamp":"2024-03-23T14:56:58.583412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:39:22.051918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-23T14:57:01.556818+0000","last_change":"2024-03-23T14:56:58.619314+0000","last_active":"2024-03-23T14:57:01.556818+0000","last_peered":"2024-03-23T14:57:01.556818+0000","last_clean":"2024-03-23T14:57:01.556818+0000","last_became_active":"2024-03-23T14:56:58.618751+0000","last_became_peered":"2024-03-23T14:56:58.618751+0000","last_unstale":"2024-03-23T14:57:01.556818+0000","last_undegraded":"2024-03-23T14:57:01.556818+0000","last_fullsized":"2024-03-23T14:57:01.556818+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:57.575127+0000","last_clean_scrub_stamp":"2024-03-23T14:56:57.575127+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:16:22.644801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574856,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":802816,"data_stored":661884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574857,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574857,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-23T14:57:09.979 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-03-23T14:57:09.979 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-03-23T14:57:09.979 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-03-23T14:57:09.979 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-03-23T14:57:10.134 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.129+0000 7f0575c75640 1 Processor -- start 2024-03-23T14:57:10.134 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.129+0000 7f0575c75640 1 -- start start 2024-03-23T14:57:10.135 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.129+0000 7f0575c75640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f057009def0 0x7f05700b45e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:10.135 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.129+0000 7f0575c75640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f05700add50 0x7f057006bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:10.135 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.129+0000 7f0575c75640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f05700b4b20 0x7f057006c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:10.135 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.129+0000 7f0575c75640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f057006d650 con 0x7f05700add50 2024-03-23T14:57:10.135 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.129+0000 7f0575c75640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 -- 0x7f05700af4a0 con 0x7f05700b4b20 2024-03-23T14:57:10.135 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.129+0000 7f0575c75640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f057009e2d0 con 0x7f057009def0 2024-03-23T14:57:10.135 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.129+0000 7f056effd640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f05700add50 0x7f057006bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:10.136 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.129+0000 7f056effd640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f05700add50 0x7f057006bf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:59346/0 (socket says 172.21.15.164:59346) 2024-03-23T14:57:10.136 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.129+0000 7f056effd640 1 -- 172.21.15.164:0/1160210152 learned_addr learned my addr 172.21.15.164:0/1160210152 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:10.136 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.129+0000 7f056ffff640 1 --2- 172.21.15.164:0/1160210152 >> v2:172.21.15.173:3300/0 conn(0x7f05700b4b20 0x7f057006c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:10.136 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.129+0000 7f056f7fe640 1 --2- 172.21.15.164:0/1160210152 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f057009def0 0x7f05700b45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:10.136 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.129+0000 7f056effd640 1 -- 172.21.15.164:0/1160210152 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f057009def0 msgr2=0x7f05700b45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:10.136 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.129+0000 7f056effd640 1 --2- 172.21.15.164:0/1160210152 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f057009def0 0x7f05700b45e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:10.136 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.129+0000 7f056effd640 1 -- 172.21.15.164:0/1160210152 >> v2:172.21.15.173:3300/0 conn(0x7f05700b4b20 msgr2=0x7f057006c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:10.136 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.129+0000 7f056effd640 1 --2- 172.21.15.164:0/1160210152 >> v2:172.21.15.173:3300/0 conn(0x7f05700b4b20 0x7f057006c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:10.136 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.129+0000 7f056effd640 1 -- 172.21.15.164:0/1160210152 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05700aeca0 con 0x7f05700add50 2024-03-23T14:57:10.136 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.129+0000 7f056f7fe640 1 --2- 172.21.15.164:0/1160210152 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f057009def0 0x7f05700b45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T14:57:10.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.133+0000 7f056effd640 1 --2- 172.21.15.164:0/1160210152 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f05700add50 0x7f057006bf70 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f05700729e0 tx=0x7f055801ead0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=513af2c5a5469782 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:10.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.133+0000 7f056e7fc640 1 -- 172.21.15.164:0/1160210152 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f055800b2d0 con 0x7f05700add50 2024-03-23T14:57:10.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.133+0000 7f056e7fc640 1 -- 172.21.15.164:0/1160210152 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f055800b430 con 0x7f05700add50 2024-03-23T14:57:10.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.133+0000 7f056e7fc640 1 -- 172.21.15.164:0/1160210152 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05580a7a90 con 0x7f05700add50 2024-03-23T14:57:10.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.133+0000 7f0575c75640 1 -- 172.21.15.164:0/1160210152 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f05700add50 msgr2=0x7f057006bf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:10.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.133+0000 7f0575c75640 1 --2- 172.21.15.164:0/1160210152 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f05700add50 0x7f057006bf70 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f05700729e0 tx=0x7f055801ead0 comp rx=0 tx=0).stop 2024-03-23T14:57:10.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.133+0000 7f0575c75640 1 -- 172.21.15.164:0/1160210152 shutdown_connections 2024-03-23T14:57:10.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.133+0000 7f0575c75640 1 --2- 172.21.15.164:0/1160210152 >> v2:172.21.15.173:3300/0 conn(0x7f05700b4b20 0x7f057006c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:10.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.133+0000 7f0575c75640 1 --2- 172.21.15.164:0/1160210152 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f05700add50 0x7f057006bf70 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:10.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.133+0000 7f0575c75640 1 --2- 172.21.15.164:0/1160210152 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f057009def0 0x7f05700b45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:10.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.133+0000 7f0575c75640 1 -- 172.21.15.164:0/1160210152 >> 172.21.15.164:0/1160210152 conn(0x7f05700b8ca0 msgr2=0x7f05700b90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:10.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.133+0000 7f0575c75640 1 -- 172.21.15.164:0/1160210152 shutdown_connections 2024-03-23T14:57:10.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.133+0000 7f0575c75640 1 -- 172.21.15.164:0/1160210152 wait complete. 2024-03-23T14:57:10.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.133+0000 7f0575c75640 1 Processor -- start 2024-03-23T14:57:10.139 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.133+0000 7f0575c75640 1 -- start start 2024-03-23T14:57:10.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.133+0000 7f0575c75640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f057009def0 0x7f057009c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:10.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.133+0000 7f0575c75640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f05700add50 0x7f05701e0460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:10.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.133+0000 7f0575c75640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f05700b4b20 0x7f05701e09a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:10.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.133+0000 7f0575c75640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f057009bd90 con 0x7f05700b4b20 2024-03-23T14:57:10.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.133+0000 7f0575c75640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f057009b1f0 con 0x7f05700add50 2024-03-23T14:57:10.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.133+0000 7f0575c75640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f05701dff70 con 0x7f057009def0 2024-03-23T14:57:10.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.133+0000 7f056ffff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f05700b4b20 0x7f05701e09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:10.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.133+0000 7f056ffff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f05700b4b20 0x7f05701e09a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:59358/0 (socket says 172.21.15.164:59358) 2024-03-23T14:57:10.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.133+0000 7f056ffff640 1 -- 172.21.15.164:0/3236439069 learned_addr learned my addr 172.21.15.164:0/3236439069 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:10.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.137+0000 7f056f7fe640 1 --2- 172.21.15.164:0/3236439069 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f057009def0 0x7f057009c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:10.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.137+0000 7f056effd640 1 --2- 172.21.15.164:0/3236439069 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f05700add50 0x7f05701e0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:10.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.137+0000 7f056ffff640 1 -- 172.21.15.164:0/3236439069 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f057009def0 msgr2=0x7f057009c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:10.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.137+0000 7f056ffff640 1 --2- 172.21.15.164:0/3236439069 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f057009def0 0x7f057009c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:10.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.137+0000 7f056ffff640 1 -- 172.21.15.164:0/3236439069 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f05700add50 msgr2=0x7f05701e0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:10.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.137+0000 7f056ffff640 1 --2- 172.21.15.164:0/3236439069 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f05700add50 0x7f05701e0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:10.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.137+0000 7f056ffff640 1 -- 172.21.15.164:0/3236439069 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f055800bd40 con 0x7f05700b4b20 2024-03-23T14:57:10.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.137+0000 7f056ffff640 1 --2- 172.21.15.164:0/3236439069 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f05700b4b20 0x7f05701e09a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f056800d560 tx=0x7f056800da20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:10.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.137+0000 7f0567fff640 1 -- 172.21.15.164:0/3236439069 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05680125f0 con 0x7f05700b4b20 2024-03-23T14:57:10.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.137+0000 7f0567fff640 1 -- 172.21.15.164:0/3236439069 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0568012770 con 0x7f05700b4b20 2024-03-23T14:57:10.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.137+0000 7f0575c75640 1 -- 172.21.15.164:0/3236439069 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f057014e680 con 0x7f05700b4b20 2024-03-23T14:57:10.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.137+0000 7f0567fff640 1 -- 172.21.15.164:0/3236439069 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0568012b20 con 0x7f05700b4b20 2024-03-23T14:57:10.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.137+0000 7f0575c75640 1 -- 172.21.15.164:0/3236439069 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f057009d690 con 0x7f05700b4b20 2024-03-23T14:57:10.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.137+0000 7f0575c75640 1 -- 172.21.15.164:0/3236439069 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f057009afc0 con 0x7f05700b4b20 2024-03-23T14:57:10.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.137+0000 7f0567fff640 1 -- 172.21.15.164:0/3236439069 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7f0568012c80 con 0x7f05700b4b20 2024-03-23T14:57:10.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.137+0000 7f0567fff640 1 --2- 172.21.15.164:0/3236439069 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f05500672d0 0x7f0550069790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:10.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.137+0000 7f0567fff640 1 -- 172.21.15.164:0/3236439069 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f05680853c0 con 0x7f05700b4b20 2024-03-23T14:57:10.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.137+0000 7f056f7fe640 1 --2- 172.21.15.164:0/3236439069 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f05500672d0 0x7f0550069790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:10.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.137+0000 7f056f7fe640 1 --2- 172.21.15.164:0/3236439069 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f05500672d0 0x7f0550069790 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f056000ab60 tx=0x7f0560008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:10.146 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.141+0000 7f0567fff640 1 -- 172.21.15.164:0/3236439069 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7f0568052090 con 0x7f05700b4b20 2024-03-23T14:57:10.324 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.321+0000 7f0575c75640 1 -- 172.21.15.164:0/3236439069 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f057009afc0 con 0x7f05700b4b20 2024-03-23T14:57:10.324 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.321+0000 7f0567fff640 1 -- 172.21.15.164:0/3236439069 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f057009afc0 con 0x7f05700b4b20 2024-03-23T14:57:10.325 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-23T14:57:10.325 INFO:teuthology.orchestra.run.smithi164.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-23T14:57:10.327 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.321+0000 7f0575c75640 1 -- 172.21.15.164:0/3236439069 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f05500672d0 msgr2=0x7f0550069790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:10.327 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.321+0000 7f0575c75640 1 --2- 172.21.15.164:0/3236439069 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f05500672d0 0x7f0550069790 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f056000ab60 tx=0x7f0560008040 comp rx=0 tx=0).stop 2024-03-23T14:57:10.327 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.321+0000 7f0575c75640 1 -- 172.21.15.164:0/3236439069 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f05700b4b20 msgr2=0x7f05701e09a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:10.328 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.321+0000 7f0575c75640 1 --2- 172.21.15.164:0/3236439069 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f05700b4b20 0x7f05701e09a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f056800d560 tx=0x7f056800da20 comp rx=0 tx=0).stop 2024-03-23T14:57:10.328 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.321+0000 7f0575c75640 1 -- 172.21.15.164:0/3236439069 shutdown_connections 2024-03-23T14:57:10.328 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.321+0000 7f0575c75640 1 --2- 172.21.15.164:0/3236439069 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f05700b4b20 0x7f05701e09a0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:10.328 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.321+0000 7f0575c75640 1 --2- 172.21.15.164:0/3236439069 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f05500672d0 0x7f0550069790 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:10.328 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.321+0000 7f0575c75640 1 --2- 172.21.15.164:0/3236439069 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f05700add50 0x7f05701e0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:10.328 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.321+0000 7f0575c75640 1 --2- 172.21.15.164:0/3236439069 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f057009def0 0x7f057009c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:10.328 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.321+0000 7f0575c75640 1 -- 172.21.15.164:0/3236439069 >> 172.21.15.164:0/3236439069 conn(0x7f05700b8ca0 msgr2=0x7f0570145470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:10.328 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.321+0000 7f0575c75640 1 -- 172.21.15.164:0/3236439069 shutdown_connections 2024-03-23T14:57:10.328 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.321+0000 7f0575c75640 1 -- 172.21.15.164:0/3236439069 wait complete. 2024-03-23T14:57:10.342 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-03-23T14:57:10.343 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T14:57:10.457 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.453+0000 7f90acb7c640 1 Processor -- start 2024-03-23T14:57:10.458 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.453+0000 7f90acb7c640 1 -- start start 2024-03-23T14:57:10.460 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.453+0000 7f90acb7c640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f90a8148cf0 0x7f90a8149180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:10.460 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.453+0000 7f90acb7c640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f90a814db70 0x7f90a814ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:10.460 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.453+0000 7f90acb7c640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90a80b51d0 con 0x7f90a80b0230 2024-03-23T14:57:10.461 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.453+0000 7f90acb7c640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 -- 0x7f90a80b0610 con 0x7f90a814db70 2024-03-23T14:57:10.461 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.453+0000 7f90acb7c640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f90a80b1500 con 0x7f90a8148cf0 2024-03-23T14:57:10.461 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.453+0000 7f90a659c640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f90a80b0230 0x7f90a80b34d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:42520/0 (socket says 172.21.15.164:42520) 2024-03-23T14:57:10.461 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.453+0000 7f90a659c640 1 -- 172.21.15.164:0/450568220 learned_addr learned my addr 172.21.15.164:0/450568220 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-03-23T14:57:10.461 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.453+0000 7f90a5d9b640 1 --2- 172.21.15.164:0/450568220 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f90a8148cf0 0x7f90a8149180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:10.461 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.453+0000 7f90a6d9d640 1 --2- 172.21.15.164:0/450568220 >> v2:172.21.15.173:3300/0 conn(0x7f90a814db70 0x7f90a814ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:10.461 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.453+0000 7f90a559a640 1 -- 172.21.15.164:0/450568220 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 992642979 0 0) 0x7f90a80b51d0 con 0x7f90a80b0230 2024-03-23T14:57:10.461 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.453+0000 7f90a559a640 1 -- 172.21.15.164:0/450568220 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9090003530 con 0x7f90a80b0230 2024-03-23T14:57:10.461 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.453+0000 7f90a559a640 1 -- 172.21.15.164:0/450568220 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1217209475 0 0) 0x7f9090003530 con 0x7f90a80b0230 2024-03-23T14:57:10.461 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.453+0000 7f90a559a640 1 -- 172.21.15.164:0/450568220 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f90a8148cf0 msgr2=0x7f90a8149180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:10.461 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.453+0000 7f90a559a640 1 --2- 172.21.15.164:0/450568220 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f90a8148cf0 0x7f90a8149180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:10.461 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.453+0000 7f90a559a640 1 -- 172.21.15.164:0/450568220 >> v2:172.21.15.173:3300/0 conn(0x7f90a814db70 msgr2=0x7f90a814ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:10.461 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.453+0000 7f90a559a640 1 --2- 172.21.15.164:0/450568220 >> v2:172.21.15.173:3300/0 conn(0x7f90a814db70 0x7f90a814ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:10.462 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.453+0000 7f90a559a640 1 -- 172.21.15.164:0/450568220 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90a806d250 con 0x7f90a80b0230 2024-03-23T14:57:10.462 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.453+0000 7f90a559a640 1 -- 172.21.15.164:0/450568220 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2526390548 0 0) 0x7f9098001680 con 0x7f90a80b0230 2024-03-23T14:57:10.462 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.453+0000 7f90a559a640 1 -- 172.21.15.164:0/450568220 >> v1:172.21.15.164:6789/0 conn(0x7f90a80b0230 legacy=0x7f90a80b34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:10.462 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.453+0000 7f90a559a640 1 --2- 172.21.15.164:0/450568220 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f90a8148cf0 0x7f9090003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:10.462 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.453+0000 7f90a559a640 1 --2- 172.21.15.164:0/450568220 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9090003f00 0x7f90900047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:10.462 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.453+0000 7f90a559a640 1 --2- 172.21.15.164:0/450568220 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9090004d10 0x7f9090005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:10.462 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.453+0000 7f90a559a640 1 -- 172.21.15.164:0/450568220 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f90a80b0610 con 0x7f9090003f00 2024-03-23T14:57:10.462 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.453+0000 7f90a559a640 1 -- 172.21.15.164:0/450568220 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f90a80b1500 con 0x7f9090004d10 2024-03-23T14:57:10.462 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.453+0000 7f90a559a640 1 -- 172.21.15.164:0/450568220 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f909000ba90 con 0x7f90a8148cf0 2024-03-23T14:57:10.462 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.453+0000 7f90a6d9d640 1 --2- 172.21.15.164:0/450568220 >> v2:172.21.15.173:3300/0 conn(0x7f90a814db70 0x7f90a814ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T14:57:10.462 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.453+0000 7f90a5d9b640 1 --2- 172.21.15.164:0/450568220 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9090003f00 0x7f90900047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:10.462 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.453+0000 7f90a659c640 1 --2- 172.21.15.164:0/450568220 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f90a8148cf0 0x7f9090003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:10.462 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.453+0000 7f90a5d9b640 1 -- 172.21.15.164:0/450568220 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f90a8148cf0 msgr2=0x7f9090003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:10.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.453+0000 7f90a5d9b640 1 --2- 172.21.15.164:0/450568220 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f90a8148cf0 0x7f9090003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:10.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.453+0000 7f90a5d9b640 1 -- 172.21.15.164:0/450568220 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9090004d10 msgr2=0x7f9090005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:10.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.453+0000 7f90a5d9b640 1 --2- 172.21.15.164:0/450568220 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9090004d10 0x7f9090005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:10.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.453+0000 7f90a5d9b640 1 -- 172.21.15.164:0/450568220 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9090003530 con 0x7f9090003f00 2024-03-23T14:57:10.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.453+0000 7f90a659c640 1 --2- 172.21.15.164:0/450568220 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f90a8148cf0 0x7f9090003980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T14:57:10.466 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.457+0000 7f90a5d9b640 1 --2- 172.21.15.164:0/450568220 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9090003f00 0x7f90900047d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f909c009b10 tx=0x7f909c01c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89ee76be65b42798 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:10.466 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.457+0000 7f90a559a640 1 -- 172.21.15.164:0/450568220 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f909c0a55b0 con 0x7f9090003f00 2024-03-23T14:57:10.466 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.457+0000 7f90a559a640 1 -- 172.21.15.164:0/450568220 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f909c0a5710 con 0x7f9090003f00 2024-03-23T14:57:10.466 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.457+0000 7f90a559a640 1 -- 172.21.15.164:0/450568220 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f909c0a5ac0 con 0x7f9090003f00 2024-03-23T14:57:10.467 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.457+0000 7f90acb7c640 1 -- 172.21.15.164:0/450568220 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9090003f00 msgr2=0x7f90900047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:10.467 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.457+0000 7f90acb7c640 1 --2- 172.21.15.164:0/450568220 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9090003f00 0x7f90900047d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f909c009b10 tx=0x7f909c01c9e0 comp rx=0 tx=0).stop 2024-03-23T14:57:10.467 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.457+0000 7f90a6d9d640 1 -- 172.21.15.164:0/450568220 reap_dead start 2024-03-23T14:57:10.467 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.457+0000 7f90acb7c640 1 -- 172.21.15.164:0/450568220 shutdown_connections 2024-03-23T14:57:10.467 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.457+0000 7f90acb7c640 1 -- 172.21.15.164:0/450568220 >> 172.21.15.164:0/450568220 conn(0x7f90a80b8ca0 msgr2=0x7f90a80b90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:10.467 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.457+0000 7f90acb7c640 1 -- 172.21.15.164:0/450568220 shutdown_connections 2024-03-23T14:57:10.467 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.457+0000 7f90acb7c640 1 -- 172.21.15.164:0/450568220 wait complete. 2024-03-23T14:57:10.467 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.457+0000 7f90acb7c640 1 Processor -- start 2024-03-23T14:57:10.467 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.457+0000 7f90acb7c640 1 -- start start 2024-03-23T14:57:10.467 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.461+0000 7f90acb7c640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9090003f00 0x7f90a809ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:10.467 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.461+0000 7f90acb7c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f90a809b860 0x7f90a8149940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:10.467 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.461+0000 7f90acb7c640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f90a8149e80 0x7f90a814a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:10.467 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.461+0000 7f90acb7c640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f90a809b160 con 0x7f90a809b860 2024-03-23T14:57:10.467 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.461+0000 7f90acb7c640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f90a809b360 con 0x7f9090003f00 2024-03-23T14:57:10.467 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.461+0000 7f90acb7c640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f90a8148190 con 0x7f90a8149e80 2024-03-23T14:57:10.467 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.461+0000 7f90a5d9b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f90a809b860 0x7f90a8149940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:10.468 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.461+0000 7f90a5d9b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f90a809b860 0x7f90a8149940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:59378/0 (socket says 172.21.15.164:59378) 2024-03-23T14:57:10.468 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.461+0000 7f90a5d9b640 1 -- 172.21.15.164:0/3147068790 learned_addr learned my addr 172.21.15.164:0/3147068790 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:10.468 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.461+0000 7f90a659c640 1 --2- 172.21.15.164:0/3147068790 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9090003f00 0x7f90a809ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:10.468 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.461+0000 7f90a6d9d640 1 --2- 172.21.15.164:0/3147068790 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f90a8149e80 0x7f90a814a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:10.468 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.461+0000 7f90a5d9b640 1 -- 172.21.15.164:0/3147068790 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f90a8149e80 msgr2=0x7f90a814a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:10.468 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.461+0000 7f90a5d9b640 1 --2- 172.21.15.164:0/3147068790 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f90a8149e80 0x7f90a814a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:10.468 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.461+0000 7f90a5d9b640 1 -- 172.21.15.164:0/3147068790 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9090003f00 msgr2=0x7f90a809ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:10.468 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.461+0000 7f90a5d9b640 1 --2- 172.21.15.164:0/3147068790 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9090003f00 0x7f90a809ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:10.468 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.461+0000 7f90a5d9b640 1 -- 172.21.15.164:0/3147068790 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90a80b0f70 con 0x7f90a809b860 2024-03-23T14:57:10.468 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.461+0000 7f90a5d9b640 1 --2- 172.21.15.164:0/3147068790 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f90a809b860 0x7f90a8149940 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f909c01c810 tx=0x7f9090005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:10.468 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.461+0000 7f9096ffd640 1 -- 172.21.15.164:0/3147068790 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f909c0a5e20 con 0x7f90a809b860 2024-03-23T14:57:10.468 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.461+0000 7f9096ffd640 1 -- 172.21.15.164:0/3147068790 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f909c004020 con 0x7f90a809b860 2024-03-23T14:57:10.468 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.461+0000 7f90acb7c640 1 -- 172.21.15.164:0/3147068790 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90a8148470 con 0x7f90a809b860 2024-03-23T14:57:10.468 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.461+0000 7f9096ffd640 1 -- 172.21.15.164:0/3147068790 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f909c0043d0 con 0x7f90a809b860 2024-03-23T14:57:10.468 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.461+0000 7f90acb7c640 1 -- 172.21.15.164:0/3147068790 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90a814ac00 con 0x7f90a809b860 2024-03-23T14:57:10.468 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.461+0000 7f90acb7c640 1 -- 172.21.15.164:0/3147068790 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90a80690a0 con 0x7f90a809b860 2024-03-23T14:57:10.469 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.461+0000 7f9096ffd640 1 -- 172.21.15.164:0/3147068790 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7f909c0b4040 con 0x7f90a809b860 2024-03-23T14:57:10.469 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.461+0000 7f9096ffd640 1 --2- 172.21.15.164:0/3147068790 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f90880672d0 0x7f9088069790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:10.469 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.461+0000 7f9096ffd640 1 -- 172.21.15.164:0/3147068790 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f909c11f210 con 0x7f90a809b860 2024-03-23T14:57:10.469 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.461+0000 7f90a659c640 1 --2- 172.21.15.164:0/3147068790 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f90880672d0 0x7f9088069790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:10.469 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.461+0000 7f90a659c640 1 --2- 172.21.15.164:0/3147068790 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f90880672d0 0x7f9088069790 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9098001830 tx=0x7f9098009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:10.470 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.465+0000 7f9096ffd640 1 -- 172.21.15.164:0/3147068790 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7f909c0ec020 con 0x7f90a809b860 2024-03-23T14:57:10.616 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.613+0000 7f90acb7c640 1 -- 172.21.15.164:0/3147068790 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f90a80b2fa0 con 0x7f90a809b860 2024-03-23T14:57:10.618 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.613+0000 7f9096ffd640 1 -- 172.21.15.164:0/3147068790 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16629 (secure 0 0 0) 0x7f909c0ef6e0 con 0x7f90a809b860 2024-03-23T14:57:10.619 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-23T14:57:10.619 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":20,"fsid":"45c7911b-b69d-4e68-907b-fbfeb780aa3b","created":"2024-03-23T14:56:34.367633+0000","modified":"2024-03-23T14:57:00.611265+0000","last_up_change":"2024-03-23T14:56:46.297403+0000","last_in_change":"2024-03-23T14:56:36.629884+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T14:56:47.186157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-23T14:56:56.994057+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-23T14:56:57.953862+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"80abdc62-bd0d-4e60-a554-7ed6fd67cf54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":984716690},{"type":"v1","addr":"172.21.15.164:6801","nonce":984716690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":984716690},{"type":"v1","addr":"172.21.15.164:6803","nonce":984716690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":984716690},{"type":"v1","addr":"172.21.15.164:6807","nonce":984716690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":984716690},{"type":"v1","addr":"172.21.15.164:6805","nonce":984716690}]},"public_addr":"172.21.15.164:6801/984716690","cluster_addr":"172.21.15.164:6803/984716690","heartbeat_back_addr":"172.21.15.164:6807/984716690","heartbeat_front_addr":"172.21.15.164:6805/984716690","state":["exists","up"]},{"osd":1,"uuid":"f9001b06-be4d-4f42-a0e6-bcef2afa0d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":1349657239},{"type":"v1","addr":"172.21.15.164:6817","nonce":1349657239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":1349657239},{"type":"v1","addr":"172.21.15.164:6819","nonce":1349657239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":1349657239},{"type":"v1","addr":"172.21.15.164:6823","nonce":1349657239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":1349657239},{"type":"v1","addr":"172.21.15.164:6821","nonce":1349657239}]},"public_addr":"172.21.15.164:6817/1349657239","cluster_addr":"172.21.15.164:6819/1349657239","heartbeat_back_addr":"172.21.15.164:6823/1349657239","heartbeat_front_addr":"172.21.15.164:6821/1349657239","state":["exists","up"]},{"osd":2,"uuid":"fa0aeb9f-8b54-4831-a188-3e361c7980d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":887861252},{"type":"v1","addr":"172.21.15.164:6809","nonce":887861252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":887861252},{"type":"v1","addr":"172.21.15.164:6811","nonce":887861252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":887861252},{"type":"v1","addr":"172.21.15.164:6815","nonce":887861252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":887861252},{"type":"v1","addr":"172.21.15.164:6813","nonce":887861252}]},"public_addr":"172.21.15.164:6809/887861252","cluster_addr":"172.21.15.164:6811/887861252","heartbeat_back_addr":"172.21.15.164:6815/887861252","heartbeat_front_addr":"172.21.15.164:6813/887861252","state":["exists","up"]},{"osd":3,"uuid":"7e18c957-3091-46d5-8e17-8f3354c1726c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":812540271},{"type":"v1","addr":"172.21.15.164:6825","nonce":812540271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":812540271},{"type":"v1","addr":"172.21.15.164:6827","nonce":812540271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":812540271},{"type":"v1","addr":"172.21.15.164:6831","nonce":812540271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":812540271},{"type":"v1","addr":"172.21.15.164:6829","nonce":812540271}]},"public_addr":"172.21.15.164:6825/812540271","cluster_addr":"172.21.15.164:6827/812540271","heartbeat_back_addr":"172.21.15.164:6831/812540271","heartbeat_front_addr":"172.21.15.164:6829/812540271","state":["exists","up"]},{"osd":4,"uuid":"8e83f315-42bf-4471-9ce8-d81948da0f9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":2656857421},{"type":"v1","addr":"172.21.15.173:6825","nonce":2656857421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2656857421},{"type":"v1","addr":"172.21.15.173:6827","nonce":2656857421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2656857421},{"type":"v1","addr":"172.21.15.173:6831","nonce":2656857421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":2656857421},{"type":"v1","addr":"172.21.15.173:6829","nonce":2656857421}]},"public_addr":"172.21.15.173:6825/2656857421","cluster_addr":"172.21.15.173:6827/2656857421","heartbeat_back_addr":"172.21.15.173:6831/2656857421","heartbeat_front_addr":"172.21.15.173:6829/2656857421","state":["exists","up"]},{"osd":5,"uuid":"a230c872-71f9-40ab-8999-defd0162d4ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":3676483185},{"type":"v1","addr":"172.21.15.173:6801","nonce":3676483185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":3676483185},{"type":"v1","addr":"172.21.15.173:6803","nonce":3676483185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":3676483185},{"type":"v1","addr":"172.21.15.173:6807","nonce":3676483185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":3676483185},{"type":"v1","addr":"172.21.15.173:6805","nonce":3676483185}]},"public_addr":"172.21.15.173:6801/3676483185","cluster_addr":"172.21.15.173:6803/3676483185","heartbeat_back_addr":"172.21.15.173:6807/3676483185","heartbeat_front_addr":"172.21.15.173:6805/3676483185","state":["exists","up"]},{"osd":6,"uuid":"6193bfd7-8432-4f7f-a117-a3fcee21af99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":3631684087},{"type":"v1","addr":"172.21.15.173:6809","nonce":3631684087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":3631684087},{"type":"v1","addr":"172.21.15.173:6811","nonce":3631684087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":3631684087},{"type":"v1","addr":"172.21.15.173:6815","nonce":3631684087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":3631684087},{"type":"v1","addr":"172.21.15.173:6813","nonce":3631684087}]},"public_addr":"172.21.15.173:6809/3631684087","cluster_addr":"172.21.15.173:6811/3631684087","heartbeat_back_addr":"172.21.15.173:6815/3631684087","heartbeat_front_addr":"172.21.15.173:6813/3631684087","state":["exists","up"]},{"osd":7,"uuid":"b0f1b80a-afc4-4538-83b9-f081d2dcb8cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":3261114948},{"type":"v1","addr":"172.21.15.173:6817","nonce":3261114948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":3261114948},{"type":"v1","addr":"172.21.15.173:6819","nonce":3261114948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":3261114948},{"type":"v1","addr":"172.21.15.173:6823","nonce":3261114948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":3261114948},{"type":"v1","addr":"172.21.15.173:6821","nonce":3261114948}]},"public_addr":"172.21.15.173:6817/3261114948","cluster_addr":"172.21.15.173:6819/3261114948","heartbeat_back_addr":"172.21.15.173:6823/3261114948","heartbeat_front_addr":"172.21.15.173:6821/3261114948","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.188390+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.159932+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.191101+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.181776+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.224365+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.177515+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.153631+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.139972+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T14:57:10.621 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.617+0000 7f90acb7c640 1 -- 172.21.15.164:0/3147068790 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f90880672d0 msgr2=0x7f9088069790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:10.621 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.617+0000 7f90acb7c640 1 --2- 172.21.15.164:0/3147068790 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f90880672d0 0x7f9088069790 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9098001830 tx=0x7f9098009040 comp rx=0 tx=0).stop 2024-03-23T14:57:10.622 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.617+0000 7f90acb7c640 1 -- 172.21.15.164:0/3147068790 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f90a809b860 msgr2=0x7f90a8149940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:10.622 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.617+0000 7f90acb7c640 1 --2- 172.21.15.164:0/3147068790 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f90a809b860 0x7f90a8149940 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f909c01c810 tx=0x7f9090005800 comp rx=0 tx=0).stop 2024-03-23T14:57:10.622 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.617+0000 7f90acb7c640 1 -- 172.21.15.164:0/3147068790 shutdown_connections 2024-03-23T14:57:10.622 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.617+0000 7f90acb7c640 1 --2- 172.21.15.164:0/3147068790 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f90a8149e80 0x7f90a814a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:10.622 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.617+0000 7f90acb7c640 1 --2- 172.21.15.164:0/3147068790 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f90a809b860 0x7f90a8149940 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:10.622 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.617+0000 7f90acb7c640 1 --2- 172.21.15.164:0/3147068790 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f90880672d0 0x7f9088069790 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:10.622 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.617+0000 7f90acb7c640 1 --2- 172.21.15.164:0/3147068790 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9090003f00 0x7f90a809ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:10.622 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.617+0000 7f90acb7c640 1 -- 172.21.15.164:0/3147068790 >> 172.21.15.164:0/3147068790 conn(0x7f90a80b8ca0 msgr2=0x7f90a80b4510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:10.622 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.617+0000 7f90acb7c640 1 -- 172.21.15.164:0/3147068790 shutdown_connections 2024-03-23T14:57:10.622 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.617+0000 7f90acb7c640 1 -- 172.21.15.164:0/3147068790 wait complete. 2024-03-23T14:57:10.637 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T14:56:47.186157+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-23T14:56:56.994057+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-23T14:56:57.953862+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-23T14:57:10.637 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-23T14:57:10.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.749+0000 7f7513751640 1 Processor -- start 2024-03-23T14:57:10.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.749+0000 7f7513751640 1 -- start start 2024-03-23T14:57:10.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.749+0000 7f7513751640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f750c148cf0 0x7f750c149180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:10.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.749+0000 7f7513751640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f750c0b51d0 con 0x7f750c0b0230 2024-03-23T14:57:10.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.749+0000 7f7513751640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f750c0b0610 con 0x7f750c14db70 2024-03-23T14:57:10.756 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.749+0000 7f7513751640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 -- 0x7f750c0b1500 con 0x7f750c148cf0 2024-03-23T14:57:10.756 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.749+0000 7f75114ed640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f750c0b0230 0x7f750c0b34d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:42526/0 (socket says 172.21.15.164:42526) 2024-03-23T14:57:10.756 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.749+0000 7f75114ed640 1 -- 172.21.15.164:0/528864256 learned_addr learned my addr 172.21.15.164:0/528864256 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-03-23T14:57:10.756 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.749+0000 7f7510cec640 1 --2- 172.21.15.164:0/528864256 >> v2:172.21.15.173:3300/0 conn(0x7f750c148cf0 0x7f750c149180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:10.756 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.749+0000 7f7503fff640 1 -- 172.21.15.164:0/528864256 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4277673013 0 0) 0x7f750c0b51d0 con 0x7f750c0b0230 2024-03-23T14:57:10.756 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.749+0000 7f7503fff640 1 -- 172.21.15.164:0/528864256 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f74f8003530 con 0x7f750c0b0230 2024-03-23T14:57:10.756 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.749+0000 7f7503fff640 1 -- 172.21.15.164:0/528864256 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1439578096 0 0) 0x7f74f8003530 con 0x7f750c0b0230 2024-03-23T14:57:10.756 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.749+0000 7f7503fff640 1 -- 172.21.15.164:0/528864256 >> v2:172.21.15.173:3300/0 conn(0x7f750c148cf0 msgr2=0x7f750c149180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:10.756 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.749+0000 7f7503fff640 1 --2- 172.21.15.164:0/528864256 >> v2:172.21.15.173:3300/0 conn(0x7f750c148cf0 0x7f750c149180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:10.756 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.749+0000 7f7503fff640 1 -- 172.21.15.164:0/528864256 >> v1:172.21.15.173:6789/0 conn(0x7f750c14db70 legacy=0x7f750c14ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:10.756 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.749+0000 7f7503fff640 1 -- 172.21.15.164:0/528864256 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f750c06d250 con 0x7f750c0b0230 2024-03-23T14:57:10.756 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.749+0000 7f7503fff640 1 -- 172.21.15.164:0/528864256 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2526390548 0 0) 0x7f74fc001680 con 0x7f750c0b0230 2024-03-23T14:57:10.756 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.749+0000 7f7503fff640 1 -- 172.21.15.164:0/528864256 >> v1:172.21.15.164:6789/0 conn(0x7f750c0b0230 legacy=0x7f750c0b34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:10.756 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.749+0000 7f7503fff640 1 --2- 172.21.15.164:0/528864256 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f74f8003980 0x7f74f8003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:10.756 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.749+0000 7f7503fff640 1 --2- 172.21.15.164:0/528864256 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f74f80047d0 0x7f74f8004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:10.757 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.749+0000 7f7503fff640 1 --2- 172.21.15.164:0/528864256 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f74f80052b0 0x7f74f800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:10.757 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.749+0000 7f7503fff640 1 -- 172.21.15.164:0/528864256 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f750c0b1500 con 0x7f74f80052b0 2024-03-23T14:57:10.757 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.749+0000 7f7503fff640 1 -- 172.21.15.164:0/528864256 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f74f800c180 con 0x7f74f8003980 2024-03-23T14:57:10.757 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.749+0000 7f7503fff640 1 -- 172.21.15.164:0/528864256 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f74f800c340 con 0x7f74f80047d0 2024-03-23T14:57:10.757 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.749+0000 7f7511cee640 1 --2- 172.21.15.164:0/528864256 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f74f80052b0 0x7f74f800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:10.757 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.749+0000 7f75114ed640 1 --2- 172.21.15.164:0/528864256 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f74f8003980 0x7f74f8003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:10.757 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.749+0000 7f7510cec640 1 --2- 172.21.15.164:0/528864256 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f74f80047d0 0x7f74f8004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:10.757 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.749+0000 7f7511cee640 1 -- 172.21.15.164:0/528864256 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f74f80047d0 msgr2=0x7f74f8004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:10.757 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.749+0000 7f7511cee640 1 --2- 172.21.15.164:0/528864256 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f74f80047d0 0x7f74f8004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:10.757 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.749+0000 7f7511cee640 1 -- 172.21.15.164:0/528864256 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f74f8003980 msgr2=0x7f74f8003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:10.757 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.749+0000 7f7511cee640 1 --2- 172.21.15.164:0/528864256 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f74f8003980 0x7f74f8003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:10.757 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.749+0000 7f7511cee640 1 -- 172.21.15.164:0/528864256 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74f8003530 con 0x7f74f80052b0 2024-03-23T14:57:10.760 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.753+0000 7f7511cee640 1 -- 172.21.15.164:0/528864256 reap_dead start 2024-03-23T14:57:10.760 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.753+0000 7f7511cee640 1 --2- 172.21.15.164:0/528864256 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f74f80052b0 0x7f74f800b990 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f7504001e40 tx=0x7f750401c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f3b569ee47bcf3e server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:10.760 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.753+0000 7f7503fff640 1 -- 172.21.15.164:0/528864256 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7504003b80 con 0x7f74f80052b0 2024-03-23T14:57:10.760 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.753+0000 7f7503fff640 1 -- 172.21.15.164:0/528864256 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7504003ce0 con 0x7f74f80052b0 2024-03-23T14:57:10.760 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.753+0000 7f7503fff640 1 -- 172.21.15.164:0/528864256 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7504005240 con 0x7f74f80052b0 2024-03-23T14:57:10.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.753+0000 7f7513751640 1 -- 172.21.15.164:0/528864256 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f74f80052b0 msgr2=0x7f74f800b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:10.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.753+0000 7f7513751640 1 --2- 172.21.15.164:0/528864256 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f74f80052b0 0x7f74f800b990 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f7504001e40 tx=0x7f750401c730 comp rx=0 tx=0).stop 2024-03-23T14:57:10.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.753+0000 7f7513751640 1 -- 172.21.15.164:0/528864256 shutdown_connections 2024-03-23T14:57:10.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.753+0000 7f7513751640 1 --2- 172.21.15.164:0/528864256 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f74f80052b0 0x7f74f800b990 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:10.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.753+0000 7f7513751640 1 -- 172.21.15.164:0/528864256 >> 172.21.15.164:0/528864256 conn(0x7f750c0b8ca0 msgr2=0x7f750c0b90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:10.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.753+0000 7f7513751640 1 -- 172.21.15.164:0/528864256 shutdown_connections 2024-03-23T14:57:10.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.753+0000 7f7513751640 1 -- 172.21.15.164:0/528864256 wait complete. 2024-03-23T14:57:10.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.753+0000 7f7513751640 1 Processor -- start 2024-03-23T14:57:10.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.753+0000 7f7513751640 1 -- start start 2024-03-23T14:57:10.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.753+0000 7f7513751640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f74f80052b0 0x7f750c1509d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:10.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.753+0000 7f7513751640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f750c070e30 0x7f750c14e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:10.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.753+0000 7f7513751640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f750c14e950 0x7f750c136810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:10.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.753+0000 7f7513751640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f750c072160 con 0x7f74f80052b0 2024-03-23T14:57:10.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.753+0000 7f7513751640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f750c14f700 con 0x7f750c14e950 2024-03-23T14:57:10.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.753+0000 7f7513751640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f750c14fbb0 con 0x7f750c070e30 2024-03-23T14:57:10.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.753+0000 7f75114ed640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f74f80052b0 0x7f750c1509d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:10.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.753+0000 7f75114ed640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f74f80052b0 0x7f750c1509d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:59390/0 (socket says 172.21.15.164:59390) 2024-03-23T14:57:10.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.753+0000 7f75114ed640 1 -- 172.21.15.164:0/88274236 learned_addr learned my addr 172.21.15.164:0/88274236 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:10.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.753+0000 7f7510cec640 1 --2- 172.21.15.164:0/88274236 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f750c070e30 0x7f750c14e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:10.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.753+0000 7f7511cee640 1 --2- 172.21.15.164:0/88274236 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f750c14e950 0x7f750c136810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:10.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.753+0000 7f75114ed640 1 -- 172.21.15.164:0/88274236 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f750c070e30 msgr2=0x7f750c14e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:10.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.753+0000 7f75114ed640 1 --2- 172.21.15.164:0/88274236 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f750c070e30 0x7f750c14e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:10.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.753+0000 7f75114ed640 1 -- 172.21.15.164:0/88274236 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f750c14e950 msgr2=0x7f750c136810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:10.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.753+0000 7f75114ed640 1 --2- 172.21.15.164:0/88274236 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f750c14e950 0x7f750c136810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:10.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.753+0000 7f75114ed640 1 -- 172.21.15.164:0/88274236 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f750c0715f0 con 0x7f74f80052b0 2024-03-23T14:57:10.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.757+0000 7f75114ed640 1 --2- 172.21.15.164:0/88274236 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f74f80052b0 0x7f750c1509d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f74fc0072f0 tx=0x7f74fc0077b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:10.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.757+0000 7f7501ffb640 1 -- 172.21.15.164:0/88274236 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74fc001650 con 0x7f74f80052b0 2024-03-23T14:57:10.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.757+0000 7f7501ffb640 1 -- 172.21.15.164:0/88274236 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74fc00f9c0 con 0x7f74f80052b0 2024-03-23T14:57:10.763 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.757+0000 7f7513751640 1 -- 172.21.15.164:0/88274236 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f750c0718d0 con 0x7f74f80052b0 2024-03-23T14:57:10.763 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.757+0000 7f7501ffb640 1 -- 172.21.15.164:0/88274236 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74fc00fd70 con 0x7f74f80052b0 2024-03-23T14:57:10.763 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.757+0000 7f7513751640 1 -- 172.21.15.164:0/88274236 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f750c0712b0 con 0x7f74f80052b0 2024-03-23T14:57:10.764 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.757+0000 7f7513751640 1 -- 172.21.15.164:0/88274236 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f750c02d370 con 0x7f74f80052b0 2024-03-23T14:57:10.764 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.757+0000 7f7501ffb640 1 -- 172.21.15.164:0/88274236 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7f74fc01d460 con 0x7f74f80052b0 2024-03-23T14:57:10.764 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.757+0000 7f7501ffb640 1 --2- 172.21.15.164:0/88274236 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f74f00672a0 0x7f74f0069760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:10.764 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.757+0000 7f7510cec640 1 --2- 172.21.15.164:0/88274236 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f74f00672a0 0x7f74f0069760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:10.764 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.757+0000 7f7501ffb640 1 -- 172.21.15.164:0/88274236 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f74fc0850f0 con 0x7f74f80052b0 2024-03-23T14:57:10.764 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.757+0000 7f7510cec640 1 --2- 172.21.15.164:0/88274236 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f74f00672a0 0x7f74f0069760 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f750c06cfb0 tx=0x7f7508008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:10.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.761+0000 7f7501ffb640 1 -- 172.21.15.164:0/88274236 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7f74fc051d30 con 0x7f74f80052b0 2024-03-23T14:57:10.911 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.905+0000 7f7513751640 1 -- 172.21.15.164:0/88274236 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f750c0690a0 con 0x7f74f80052b0 2024-03-23T14:57:10.911 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.905+0000 7f7501ffb640 1 -- 172.21.15.164:0/88274236 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f74fc0553f0 con 0x7f74f80052b0 2024-03-23T14:57:10.912 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 1 2024-03-23T14:57:10.914 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.909+0000 7f7513751640 1 -- 172.21.15.164:0/88274236 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f74f00672a0 msgr2=0x7f74f0069760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:10.914 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.909+0000 7f7513751640 1 --2- 172.21.15.164:0/88274236 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f74f00672a0 0x7f74f0069760 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f750c06cfb0 tx=0x7f7508008040 comp rx=0 tx=0).stop 2024-03-23T14:57:10.914 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.909+0000 7f7513751640 1 -- 172.21.15.164:0/88274236 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f74f80052b0 msgr2=0x7f750c1509d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:10.914 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.909+0000 7f7513751640 1 --2- 172.21.15.164:0/88274236 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f74f80052b0 0x7f750c1509d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f74fc0072f0 tx=0x7f74fc0077b0 comp rx=0 tx=0).stop 2024-03-23T14:57:10.914 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.909+0000 7f7513751640 1 -- 172.21.15.164:0/88274236 shutdown_connections 2024-03-23T14:57:10.915 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.909+0000 7f7513751640 1 --2- 172.21.15.164:0/88274236 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f74f00672a0 0x7f74f0069760 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:10.915 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.909+0000 7f7513751640 1 --2- 172.21.15.164:0/88274236 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f750c14e950 0x7f750c136810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:10.915 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.909+0000 7f7513751640 1 --2- 172.21.15.164:0/88274236 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f750c070e30 0x7f750c14e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:10.915 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.909+0000 7f7513751640 1 --2- 172.21.15.164:0/88274236 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f74f80052b0 0x7f750c1509d0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:10.915 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.909+0000 7f7513751640 1 -- 172.21.15.164:0/88274236 >> 172.21.15.164:0/88274236 conn(0x7f750c0b8ca0 msgr2=0x7f750c0b43f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:10.915 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.909+0000 7f7513751640 1 -- 172.21.15.164:0/88274236 shutdown_connections 2024-03-23T14:57:10.915 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:10.909+0000 7f7513751640 1 -- 172.21.15.164:0/88274236 wait complete. 2024-03-23T14:57:10.929 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-23T14:57:11.039 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.033+0000 7f567bffd640 1 Processor -- start 2024-03-23T14:57:11.040 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.033+0000 7f567bffd640 1 -- start start 2024-03-23T14:57:11.041 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.037+0000 7f567bffd640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5674148cf0 0x7f5674149180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:11.041 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.037+0000 7f567bffd640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f567414db70 0x7f567414ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:11.041 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.037+0000 7f567bffd640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56740b51d0 con 0x7f56740b0230 2024-03-23T14:57:11.041 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.037+0000 7f567bffd640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 -- 0x7f56740b0610 con 0x7f567414db70 2024-03-23T14:57:11.041 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.037+0000 7f567bffd640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f56740b1500 con 0x7f5674148cf0 2024-03-23T14:57:11.041 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.037+0000 7f5679598640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5674148cf0 0x7f5674149180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:11.041 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.037+0000 7f5679598640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5674148cf0 0x7f5674149180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.164:60810/0 (socket says 172.21.15.164:60810) 2024-03-23T14:57:11.041 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.037+0000 7f5679598640 1 -- 172.21.15.164:0/4171566607 learned_addr learned my addr 172.21.15.164:0/4171566607 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:11.041 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.037+0000 7f567a59a640 1 --2- 172.21.15.164:0/4171566607 >> v2:172.21.15.173:3300/0 conn(0x7f567414db70 0x7f567414ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:11.041 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.037+0000 7f5679598640 1 -- 172.21.15.164:0/4171566607 >> v2:172.21.15.173:3300/0 conn(0x7f567414db70 msgr2=0x7f567414ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:11.041 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.037+0000 7f5679598640 1 --2- 172.21.15.164:0/4171566607 >> v2:172.21.15.173:3300/0 conn(0x7f567414db70 0x7f567414ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:11.042 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.037+0000 7f5679598640 1 -- 172.21.15.164:0/4171566607 >> v1:172.21.15.173:6789/0 conn(0x7f56740b0230 legacy=0x7f56740b34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:11.042 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.037+0000 7f5679598640 1 -- 172.21.15.164:0/4171566607 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f567406d250 con 0x7f5674148cf0 2024-03-23T14:57:11.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.037+0000 7f5679598640 1 --2- 172.21.15.164:0/4171566607 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5674148cf0 0x7f5674149180 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f5664009870 tx=0x7f566401c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ed9899b9c4eb785 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:11.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.037+0000 7f5678d97640 1 -- 172.21.15.164:0/4171566607 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56640a6040 con 0x7f5674148cf0 2024-03-23T14:57:11.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.037+0000 7f5678d97640 1 -- 172.21.15.164:0/4171566607 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56640aa600 con 0x7f5674148cf0 2024-03-23T14:57:11.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.037+0000 7f5678d97640 1 -- 172.21.15.164:0/4171566607 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56640aa990 con 0x7f5674148cf0 2024-03-23T14:57:11.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.041+0000 7f567bffd640 1 -- 172.21.15.164:0/4171566607 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5674148cf0 msgr2=0x7f5674149180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:11.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.041+0000 7f567bffd640 1 --2- 172.21.15.164:0/4171566607 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5674148cf0 0x7f5674149180 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f5664009870 tx=0x7f566401c4a0 comp rx=0 tx=0).stop 2024-03-23T14:57:11.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.041+0000 7f567bffd640 1 -- 172.21.15.164:0/4171566607 shutdown_connections 2024-03-23T14:57:11.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.041+0000 7f567bffd640 1 --2- 172.21.15.164:0/4171566607 >> v2:172.21.15.173:3300/0 conn(0x7f567414db70 0x7f567414ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:11.045 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.041+0000 7f567bffd640 1 --2- 172.21.15.164:0/4171566607 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5674148cf0 0x7f5674149180 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:11.045 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.041+0000 7f567bffd640 1 -- 172.21.15.164:0/4171566607 >> 172.21.15.164:0/4171566607 conn(0x7f56740b8ca0 msgr2=0x7f56740b90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:11.045 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.041+0000 7f567bffd640 1 -- 172.21.15.164:0/4171566607 shutdown_connections 2024-03-23T14:57:11.045 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.041+0000 7f567bffd640 1 -- 172.21.15.164:0/4171566607 wait complete. 2024-03-23T14:57:11.048 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.041+0000 7f567bffd640 1 Processor -- start 2024-03-23T14:57:11.048 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.041+0000 7f567bffd640 1 -- start start 2024-03-23T14:57:11.048 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.041+0000 7f567bffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f56740b0230 0x7f567414f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:11.048 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.041+0000 7f567bffd640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5674148cf0 0x7f567413bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:11.048 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.041+0000 7f567bffd640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f567414db70 0x7f56740ada30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:11.048 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.041+0000 7f567bffd640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f56740af570 con 0x7f56740b0230 2024-03-23T14:57:11.048 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.041+0000 7f567bffd640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f56740af320 con 0x7f567414db70 2024-03-23T14:57:11.048 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.041+0000 7f567bffd640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f5674138660 con 0x7f5674148cf0 2024-03-23T14:57:11.048 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.041+0000 7f5679d99640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f56740b0230 0x7f567414f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:11.048 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.041+0000 7f5679d99640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f56740b0230 0x7f567414f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:59400/0 (socket says 172.21.15.164:59400) 2024-03-23T14:57:11.049 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.041+0000 7f5679d99640 1 -- 172.21.15.164:0/1875742325 learned_addr learned my addr 172.21.15.164:0/1875742325 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:11.049 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.041+0000 7f5679598640 1 --2- 172.21.15.164:0/1875742325 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5674148cf0 0x7f567413bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:11.049 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.041+0000 7f567a59a640 1 --2- 172.21.15.164:0/1875742325 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f567414db70 0x7f56740ada30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:11.049 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.041+0000 7f5679d99640 1 -- 172.21.15.164:0/1875742325 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5674148cf0 msgr2=0x7f567413bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:11.050 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.041+0000 7f5679d99640 1 --2- 172.21.15.164:0/1875742325 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5674148cf0 0x7f567413bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:11.050 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.041+0000 7f5679d99640 1 -- 172.21.15.164:0/1875742325 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f567414db70 msgr2=0x7f56740ada30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:11.050 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.041+0000 7f5679d99640 1 --2- 172.21.15.164:0/1875742325 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f567414db70 0x7f56740ada30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:11.050 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.041+0000 7f5679d99640 1 -- 172.21.15.164:0/1875742325 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56640094d0 con 0x7f56740b0230 2024-03-23T14:57:11.050 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.041+0000 7f5679598640 1 --2- 172.21.15.164:0/1875742325 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5674148cf0 0x7f567413bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T14:57:11.050 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.041+0000 7f567a59a640 1 --2- 172.21.15.164:0/1875742325 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f567414db70 0x7f56740ada30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T14:57:11.050 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.041+0000 7f5679d99640 1 --2- 172.21.15.164:0/1875742325 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f56740b0230 0x7f567414f0f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f567000c480 tx=0x7f567000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:11.050 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.041+0000 7f566a7fc640 1 -- 172.21.15.164:0/1875742325 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56700052d0 con 0x7f56740b0230 2024-03-23T14:57:11.050 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.041+0000 7f566a7fc640 1 -- 172.21.15.164:0/1875742325 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5670005430 con 0x7f56740b0230 2024-03-23T14:57:11.050 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.041+0000 7f567bffd640 1 -- 172.21.15.164:0/1875742325 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f567413d540 con 0x7f56740b0230 2024-03-23T14:57:11.050 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.041+0000 7f566a7fc640 1 -- 172.21.15.164:0/1875742325 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5670007260 con 0x7f56740b0230 2024-03-23T14:57:11.050 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.041+0000 7f567bffd640 1 -- 172.21.15.164:0/1875742325 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5674150000 con 0x7f56740b0230 2024-03-23T14:57:11.050 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.041+0000 7f567bffd640 1 -- 172.21.15.164:0/1875742325 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f567413ead0 con 0x7f56740b0230 2024-03-23T14:57:11.050 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.045+0000 7f566a7fc640 1 -- 172.21.15.164:0/1875742325 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7f56700073c0 con 0x7f56740b0230 2024-03-23T14:57:11.051 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.045+0000 7f566a7fc640 1 --2- 172.21.15.164:0/1875742325 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f56580672a0 0x7f5658069760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:11.051 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.045+0000 7f5679598640 1 --2- 172.21.15.164:0/1875742325 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f56580672a0 0x7f5658069760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:11.051 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.045+0000 7f566a7fc640 1 -- 172.21.15.164:0/1875742325 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f5670084900 con 0x7f56740b0230 2024-03-23T14:57:11.051 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.045+0000 7f5679598640 1 --2- 172.21.15.164:0/1875742325 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f56580672a0 0x7f5658069760 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f56640ae000 tx=0x7f56640a7000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:11.053 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.049+0000 7f566a7fc640 1 -- 172.21.15.164:0/1875742325 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7f5670051540 con 0x7f56740b0230 2024-03-23T14:57:11.210 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.205+0000 7f567bffd640 1 -- 172.21.15.164:0/1875742325 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f56740690a0 con 0x7f56740b0230 2024-03-23T14:57:11.210 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.205+0000 7f566a7fc640 1 -- 172.21.15.164:0/1875742325 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f5670054c00 con 0x7f56740b0230 2024-03-23T14:57:11.211 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 32 2024-03-23T14:57:11.214 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.209+0000 7f567bffd640 1 -- 172.21.15.164:0/1875742325 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f56580672a0 msgr2=0x7f5658069760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:11.214 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.209+0000 7f567bffd640 1 --2- 172.21.15.164:0/1875742325 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f56580672a0 0x7f5658069760 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f56640ae000 tx=0x7f56640a7000 comp rx=0 tx=0).stop 2024-03-23T14:57:11.214 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.209+0000 7f567bffd640 1 -- 172.21.15.164:0/1875742325 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f56740b0230 msgr2=0x7f567414f0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:11.214 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.209+0000 7f567bffd640 1 --2- 172.21.15.164:0/1875742325 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f56740b0230 0x7f567414f0f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f567000c480 tx=0x7f567000c940 comp rx=0 tx=0).stop 2024-03-23T14:57:11.214 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.209+0000 7f567bffd640 1 -- 172.21.15.164:0/1875742325 shutdown_connections 2024-03-23T14:57:11.214 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.209+0000 7f567bffd640 1 --2- 172.21.15.164:0/1875742325 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f56580672a0 0x7f5658069760 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:11.215 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.209+0000 7f567bffd640 1 --2- 172.21.15.164:0/1875742325 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f567414db70 0x7f56740ada30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:11.215 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.209+0000 7f567bffd640 1 --2- 172.21.15.164:0/1875742325 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5674148cf0 0x7f567413bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:11.215 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.209+0000 7f567bffd640 1 --2- 172.21.15.164:0/1875742325 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f56740b0230 0x7f567414f0f0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:11.215 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.209+0000 7f567bffd640 1 -- 172.21.15.164:0/1875742325 >> 172.21.15.164:0/1875742325 conn(0x7f56740b8ca0 msgr2=0x7f56740b43f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:11.215 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.209+0000 7f567bffd640 1 -- 172.21.15.164:0/1875742325 shutdown_connections 2024-03-23T14:57:11.215 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.209+0000 7f567bffd640 1 -- 172.21.15.164:0/1875742325 wait complete. 2024-03-23T14:57:11.229 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-23T14:57:11.344 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.341+0000 7fca6ebae640 1 Processor -- start 2024-03-23T14:57:11.345 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.341+0000 7fca6ebae640 1 -- start start 2024-03-23T14:57:11.346 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.341+0000 7fca6ebae640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fca680b0230 0x7fca680b34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:11.346 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.341+0000 7fca6ebae640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca680b51d0 con 0x7fca6814db70 2024-03-23T14:57:11.346 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.341+0000 7fca6ebae640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca680b0610 con 0x7fca68148cf0 2024-03-23T14:57:11.346 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.341+0000 7fca6ebae640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fca680b1500 con 0x7fca680b0230 2024-03-23T14:57:11.346 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.341+0000 7fca6c94a640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fca680b0230 0x7fca680b34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:11.346 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.341+0000 7fca6c94a640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fca680b0230 0x7fca680b34d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:59410/0 (socket says 172.21.15.164:59410) 2024-03-23T14:57:11.346 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.341+0000 7fca6d14b640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fca6814db70 0x7fca6814ff60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:42534/0 (socket says 172.21.15.164:42534) 2024-03-23T14:57:11.346 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.341+0000 7fca6c94a640 1 -- 172.21.15.164:0/3595161821 learned_addr learned my addr 172.21.15.164:0/3595161821 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:11.346 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.341+0000 7fca6c94a640 1 -- 172.21.15.164:0/3595161821 >> v1:172.21.15.173:6789/0 conn(0x7fca68148cf0 legacy=0x7fca68149180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:11.346 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.341+0000 7fca6c94a640 1 -- 172.21.15.164:0/3595161821 >> v1:172.21.15.164:6789/0 conn(0x7fca6814db70 legacy=0x7fca6814ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:11.346 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.341+0000 7fca677fe640 1 -- 172.21.15.164:0/3595161821 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 49563010 0 0) 0x7fca680b51d0 con 0x7fca6814db70 2024-03-23T14:57:11.346 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.341+0000 7fca6c94a640 1 -- 172.21.15.164:0/3595161821 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca6806d250 con 0x7fca680b0230 2024-03-23T14:57:11.348 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.345+0000 7fca6c94a640 1 --2- 172.21.15.164:0/3595161821 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fca680b0230 0x7fca680b34d0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fca58009870 tx=0x7fca5801c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17dc1e6889309dd5 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:11.349 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.345+0000 7fca677fe640 1 -- 172.21.15.164:0/3595161821 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca580ac630 con 0x7fca680b0230 2024-03-23T14:57:11.349 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.345+0000 7fca677fe640 1 -- 172.21.15.164:0/3595161821 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca580ac790 con 0x7fca680b0230 2024-03-23T14:57:11.349 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.345+0000 7fca677fe640 1 -- 172.21.15.164:0/3595161821 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca580acb20 con 0x7fca680b0230 2024-03-23T14:57:11.349 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.345+0000 7fca6ebae640 1 -- 172.21.15.164:0/3595161821 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fca680b0230 msgr2=0x7fca680b34d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:11.349 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.345+0000 7fca6ebae640 1 --2- 172.21.15.164:0/3595161821 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fca680b0230 0x7fca680b34d0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fca58009870 tx=0x7fca5801c3d0 comp rx=0 tx=0).stop 2024-03-23T14:57:11.349 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.345+0000 7fca6ebae640 1 -- 172.21.15.164:0/3595161821 shutdown_connections 2024-03-23T14:57:11.349 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.345+0000 7fca6ebae640 1 --2- 172.21.15.164:0/3595161821 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fca680b0230 0x7fca680b34d0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:11.349 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.345+0000 7fca6ebae640 1 -- 172.21.15.164:0/3595161821 >> 172.21.15.164:0/3595161821 conn(0x7fca680b8ca0 msgr2=0x7fca680b90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:11.349 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.345+0000 7fca6ebae640 1 -- 172.21.15.164:0/3595161821 shutdown_connections 2024-03-23T14:57:11.349 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.345+0000 7fca6ebae640 1 -- 172.21.15.164:0/3595161821 wait complete. 2024-03-23T14:57:11.350 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.345+0000 7fca6ebae640 1 Processor -- start 2024-03-23T14:57:11.350 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.345+0000 7fca6ebae640 1 -- start start 2024-03-23T14:57:11.352 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.345+0000 7fca6ebae640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fca680b0230 0x7fca6809c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:11.352 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.345+0000 7fca6ebae640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fca68148cf0 0x7fca6813fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:11.352 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.345+0000 7fca6ebae640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fca6814db70 0x7fca68142030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:11.352 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.345+0000 7fca6ebae640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fca6813ea40 con 0x7fca680b0230 2024-03-23T14:57:11.352 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.345+0000 7fca6ebae640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fca68141870 con 0x7fca68148cf0 2024-03-23T14:57:11.352 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.345+0000 7fca6ebae640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fca68140ac0 con 0x7fca6814db70 2024-03-23T14:57:11.352 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.345+0000 7fca6c94a640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fca680b0230 0x7fca6809c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:11.352 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.345+0000 7fca6c94a640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fca680b0230 0x7fca6809c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:59426/0 (socket says 172.21.15.164:59426) 2024-03-23T14:57:11.353 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.345+0000 7fca6c94a640 1 -- 172.21.15.164:0/488059039 learned_addr learned my addr 172.21.15.164:0/488059039 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:11.353 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.345+0000 7fca6d14b640 1 --2- 172.21.15.164:0/488059039 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fca6814db70 0x7fca68142030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:11.353 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.345+0000 7fca6c94a640 1 -- 172.21.15.164:0/488059039 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fca6814db70 msgr2=0x7fca68142030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:11.353 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.345+0000 7fca6c94a640 1 --2- 172.21.15.164:0/488059039 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fca6814db70 0x7fca68142030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:11.353 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.345+0000 7fca6c94a640 1 -- 172.21.15.164:0/488059039 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fca68148cf0 msgr2=0x7fca6813fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:11.353 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.345+0000 7fca6c94a640 1 --2- 172.21.15.164:0/488059039 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fca68148cf0 0x7fca6813fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:11.353 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.345+0000 7fca6c94a640 1 -- 172.21.15.164:0/488059039 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca580094d0 con 0x7fca680b0230 2024-03-23T14:57:11.353 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.345+0000 7fca6d14b640 1 --2- 172.21.15.164:0/488059039 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fca6814db70 0x7fca68142030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T14:57:11.353 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.345+0000 7fca6c94a640 1 --2- 172.21.15.164:0/488059039 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fca680b0230 0x7fca6809c530 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fca580af040 tx=0x7fca5801ac10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:11.353 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.345+0000 7fca657fa640 1 -- 172.21.15.164:0/488059039 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca58004220 con 0x7fca680b0230 2024-03-23T14:57:11.353 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.345+0000 7fca657fa640 1 -- 172.21.15.164:0/488059039 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca58004380 con 0x7fca680b0230 2024-03-23T14:57:11.353 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.345+0000 7fca657fa640 1 -- 172.21.15.164:0/488059039 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca58003820 con 0x7fca680b0230 2024-03-23T14:57:11.353 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.349+0000 7fca6ebae640 1 -- 172.21.15.164:0/488059039 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca6809d180 con 0x7fca680b0230 2024-03-23T14:57:11.353 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.349+0000 7fca6ebae640 1 -- 172.21.15.164:0/488059039 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca680725b0 con 0x7fca680b0230 2024-03-23T14:57:11.354 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.349+0000 7fca6ebae640 1 -- 172.21.15.164:0/488059039 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca680690a0 con 0x7fca680b0230 2024-03-23T14:57:11.354 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.349+0000 7fca657fa640 1 -- 172.21.15.164:0/488059039 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7fca580b3040 con 0x7fca680b0230 2024-03-23T14:57:11.354 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.349+0000 7fca657fa640 1 --2- 172.21.15.164:0/488059039 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fca540672a0 0x7fca54069760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:11.355 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.349+0000 7fca657fa640 1 -- 172.21.15.164:0/488059039 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fca5811f2f0 con 0x7fca680b0230 2024-03-23T14:57:11.355 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.349+0000 7fca67fff640 1 --2- 172.21.15.164:0/488059039 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fca540672a0 0x7fca54069760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:11.355 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.349+0000 7fca67fff640 1 --2- 172.21.15.164:0/488059039 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fca540672a0 0x7fca54069760 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fca68150fc0 tx=0x7fca5c0079c0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:11.357 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.353+0000 7fca657fa640 1 -- 172.21.15.164:0/488059039 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7fca580ebfb0 con 0x7fca680b0230 2024-03-23T14:57:11.517 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.513+0000 7fca6ebae640 1 -- 172.21.15.164:0/488059039 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fca680b2fa0 con 0x7fca680b0230 2024-03-23T14:57:11.518 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.513+0000 7fca657fa640 1 -- 172.21.15.164:0/488059039 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7fca580bc020 con 0x7fca680b0230 2024-03-23T14:57:11.518 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 64 2024-03-23T14:57:11.520 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.517+0000 7fca6ebae640 1 -- 172.21.15.164:0/488059039 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fca540672a0 msgr2=0x7fca54069760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:11.520 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.517+0000 7fca6ebae640 1 --2- 172.21.15.164:0/488059039 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fca540672a0 0x7fca54069760 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fca68150fc0 tx=0x7fca5c0079c0 comp rx=0 tx=0).stop 2024-03-23T14:57:11.520 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.517+0000 7fca6ebae640 1 -- 172.21.15.164:0/488059039 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fca680b0230 msgr2=0x7fca6809c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:11.520 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.517+0000 7fca6ebae640 1 --2- 172.21.15.164:0/488059039 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fca680b0230 0x7fca6809c530 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fca580af040 tx=0x7fca5801ac10 comp rx=0 tx=0).stop 2024-03-23T14:57:11.520 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.517+0000 7fca6ebae640 1 -- 172.21.15.164:0/488059039 shutdown_connections 2024-03-23T14:57:11.520 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.517+0000 7fca6ebae640 1 --2- 172.21.15.164:0/488059039 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fca6814db70 0x7fca68142030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:11.521 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.517+0000 7fca6ebae640 1 --2- 172.21.15.164:0/488059039 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fca540672a0 0x7fca54069760 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:11.521 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.517+0000 7fca6ebae640 1 --2- 172.21.15.164:0/488059039 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fca68148cf0 0x7fca6813fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:11.521 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.517+0000 7fca6ebae640 1 --2- 172.21.15.164:0/488059039 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fca680b0230 0x7fca6809c530 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:11.521 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.517+0000 7fca6ebae640 1 -- 172.21.15.164:0/488059039 >> 172.21.15.164:0/488059039 conn(0x7fca680b8ca0 msgr2=0x7fca680b2350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:11.521 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.517+0000 7fca6ebae640 1 -- 172.21.15.164:0/488059039 shutdown_connections 2024-03-23T14:57:11.521 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.517+0000 7fca6ebae640 1 -- 172.21.15.164:0/488059039 wait complete. 2024-03-23T14:57:11.537 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T14:57:11.656 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.649+0000 7fa42b578640 1 Processor -- start 2024-03-23T14:57:11.656 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.649+0000 7fa42b578640 1 -- start start 2024-03-23T14:57:11.658 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.653+0000 7fa42b578640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa4240b0230 0x7fa4240b34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:11.658 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.653+0000 7fa42b578640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fa42414db70 0x7fa42414ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:11.658 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.653+0000 7fa42b578640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4240b51d0 con 0x7fa424148cf0 2024-03-23T14:57:11.658 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.653+0000 7fa42b578640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 -- 0x7fa4240b0610 con 0x7fa42414db70 2024-03-23T14:57:11.658 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.653+0000 7fa42b578640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fa4240b1500 con 0x7fa4240b0230 2024-03-23T14:57:11.658 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.653+0000 7fa428b13640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fa424148cf0 0x7fa424149150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:42548/0 (socket says 172.21.15.164:42548) 2024-03-23T14:57:11.658 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.653+0000 7fa428b13640 1 -- 172.21.15.164:0/1337141231 learned_addr learned my addr 172.21.15.164:0/1337141231 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-03-23T14:57:11.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.653+0000 7fa429b15640 1 --2- 172.21.15.164:0/1337141231 >> v2:172.21.15.173:3300/0 conn(0x7fa42414db70 0x7fa42414ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:11.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.653+0000 7fa429314640 1 --2- 172.21.15.164:0/1337141231 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa4240b0230 0x7fa4240b34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:11.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.653+0000 7fa41bfff640 1 -- 172.21.15.164:0/1337141231 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 98004698 0 0) 0x7fa4240b51d0 con 0x7fa424148cf0 2024-03-23T14:57:11.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.653+0000 7fa41bfff640 1 -- 172.21.15.164:0/1337141231 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa410003530 con 0x7fa424148cf0 2024-03-23T14:57:11.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.653+0000 7fa41bfff640 1 -- 172.21.15.164:0/1337141231 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 876251364 0 0) 0x7fa410003530 con 0x7fa424148cf0 2024-03-23T14:57:11.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.653+0000 7fa41bfff640 1 -- 172.21.15.164:0/1337141231 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa4240b0230 msgr2=0x7fa4240b34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:11.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.653+0000 7fa41bfff640 1 --2- 172.21.15.164:0/1337141231 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa4240b0230 0x7fa4240b34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:11.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.653+0000 7fa41bfff640 1 -- 172.21.15.164:0/1337141231 >> v2:172.21.15.173:3300/0 conn(0x7fa42414db70 msgr2=0x7fa42414ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:11.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.653+0000 7fa41bfff640 1 --2- 172.21.15.164:0/1337141231 >> v2:172.21.15.173:3300/0 conn(0x7fa42414db70 0x7fa42414ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:11.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.653+0000 7fa41bfff640 1 -- 172.21.15.164:0/1337141231 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa42406d250 con 0x7fa424148cf0 2024-03-23T14:57:11.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.653+0000 7fa41bfff640 1 -- 172.21.15.164:0/1337141231 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2526390548 0 0) 0x7fa420001680 con 0x7fa424148cf0 2024-03-23T14:57:11.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.653+0000 7fa41bfff640 1 -- 172.21.15.164:0/1337141231 >> v1:172.21.15.164:6789/0 conn(0x7fa424148cf0 legacy=0x7fa424149150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:11.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.653+0000 7fa41bfff640 1 --2- 172.21.15.164:0/1337141231 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa4240b0230 0x7fa410003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:11.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.653+0000 7fa41bfff640 1 --2- 172.21.15.164:0/1337141231 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa410003f00 0x7fa4100047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:11.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.653+0000 7fa41bfff640 1 --2- 172.21.15.164:0/1337141231 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa410004d10 0x7fa410005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:11.660 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.653+0000 7fa41bfff640 1 -- 172.21.15.164:0/1337141231 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa4240b0610 con 0x7fa410003f00 2024-03-23T14:57:11.660 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.653+0000 7fa41bfff640 1 -- 172.21.15.164:0/1337141231 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa4240b1500 con 0x7fa410004d10 2024-03-23T14:57:11.660 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.653+0000 7fa41bfff640 1 -- 172.21.15.164:0/1337141231 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fa41000b9e0 con 0x7fa4240b0230 2024-03-23T14:57:11.660 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.653+0000 7fa428b13640 1 --2- 172.21.15.164:0/1337141231 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa410003f00 0x7fa4100047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:11.660 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.653+0000 7fa429b15640 1 --2- 172.21.15.164:0/1337141231 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa410004d10 0x7fa410005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:11.660 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.653+0000 7fa428b13640 1 -- 172.21.15.164:0/1337141231 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa4240b0230 msgr2=0x7fa410003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:11.660 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.653+0000 7fa428b13640 1 --2- 172.21.15.164:0/1337141231 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa4240b0230 0x7fa410003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:11.660 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.653+0000 7fa428b13640 1 -- 172.21.15.164:0/1337141231 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa410004d10 msgr2=0x7fa410005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:11.660 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.653+0000 7fa428b13640 1 --2- 172.21.15.164:0/1337141231 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa410004d10 0x7fa410005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:11.660 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.653+0000 7fa428b13640 1 -- 172.21.15.164:0/1337141231 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa410003530 con 0x7fa410003f00 2024-03-23T14:57:11.662 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.657+0000 7fa428b13640 1 --2- 172.21.15.164:0/1337141231 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa410003f00 0x7fa4100047d0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa42000c2c0 tx=0x7fa42001ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8bb9821c515d873 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:11.662 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.657+0000 7fa41bfff640 1 -- 172.21.15.164:0/1337141231 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4200a5900 con 0x7fa410003f00 2024-03-23T14:57:11.662 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.657+0000 7fa41bfff640 1 -- 172.21.15.164:0/1337141231 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4200a5a60 con 0x7fa410003f00 2024-03-23T14:57:11.662 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.657+0000 7fa41bfff640 1 -- 172.21.15.164:0/1337141231 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4200a5e10 con 0x7fa410003f00 2024-03-23T14:57:11.662 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.657+0000 7fa42b578640 1 -- 172.21.15.164:0/1337141231 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa410003f00 msgr2=0x7fa4100047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:11.662 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.657+0000 7fa42b578640 1 --2- 172.21.15.164:0/1337141231 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa410003f00 0x7fa4100047d0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa42000c2c0 tx=0x7fa42001ccb0 comp rx=0 tx=0).stop 2024-03-23T14:57:11.662 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.657+0000 7fa429b15640 1 -- 172.21.15.164:0/1337141231 reap_dead start 2024-03-23T14:57:11.662 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.657+0000 7fa42b578640 1 -- 172.21.15.164:0/1337141231 shutdown_connections 2024-03-23T14:57:11.662 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.657+0000 7fa42b578640 1 -- 172.21.15.164:0/1337141231 >> 172.21.15.164:0/1337141231 conn(0x7fa4240b8ca0 msgr2=0x7fa4240b90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:11.662 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.657+0000 7fa42b578640 1 -- 172.21.15.164:0/1337141231 shutdown_connections 2024-03-23T14:57:11.662 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.657+0000 7fa42b578640 1 -- 172.21.15.164:0/1337141231 wait complete. 2024-03-23T14:57:11.663 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.657+0000 7fa42b578640 1 Processor -- start 2024-03-23T14:57:11.663 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.657+0000 7fa42b578640 1 -- start start 2024-03-23T14:57:11.665 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.657+0000 7fa42b578640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa410003f00 0x7fa42409d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:11.665 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.657+0000 7fa42b578640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa42409b700 0x7fa42409c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:11.665 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.657+0000 7fa42b578640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa424145f50 0x7fa42413e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:11.665 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.657+0000 7fa42b578640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa42409d670 con 0x7fa424145f50 2024-03-23T14:57:11.665 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.657+0000 7fa42b578640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa42409c6d0 con 0x7fa42409b700 2024-03-23T14:57:11.665 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.657+0000 7fa42b578640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fa42409d880 con 0x7fa410003f00 2024-03-23T14:57:11.665 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.657+0000 7fa429b15640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa424145f50 0x7fa42413e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:11.665 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.657+0000 7fa429b15640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa424145f50 0x7fa42413e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:59436/0 (socket says 172.21.15.164:59436) 2024-03-23T14:57:11.665 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.657+0000 7fa429b15640 1 -- 172.21.15.164:0/2917055477 learned_addr learned my addr 172.21.15.164:0/2917055477 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:11.665 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.657+0000 7fa428b13640 1 --2- 172.21.15.164:0/2917055477 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa42409b700 0x7fa42409c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:11.666 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.657+0000 7fa429314640 1 --2- 172.21.15.164:0/2917055477 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa410003f00 0x7fa42409d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:11.666 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.657+0000 7fa429b15640 1 -- 172.21.15.164:0/2917055477 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa410003f00 msgr2=0x7fa42409d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:11.666 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.657+0000 7fa429b15640 1 --2- 172.21.15.164:0/2917055477 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa410003f00 0x7fa42409d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:11.666 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.657+0000 7fa429b15640 1 -- 172.21.15.164:0/2917055477 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa42409b700 msgr2=0x7fa42409c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:11.666 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.657+0000 7fa429b15640 1 --2- 172.21.15.164:0/2917055477 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa42409b700 0x7fa42409c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:11.666 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.657+0000 7fa429b15640 1 -- 172.21.15.164:0/2917055477 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4240b0f70 con 0x7fa424145f50 2024-03-23T14:57:11.666 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.657+0000 7fa428b13640 1 --2- 172.21.15.164:0/2917055477 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa42409b700 0x7fa42409c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T14:57:11.666 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.657+0000 7fa429b15640 1 --2- 172.21.15.164:0/2917055477 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa424145f50 0x7fa42413e000 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa42409e2a0 tx=0x7fa41c00f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:11.666 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.657+0000 7fa419ffb640 1 -- 172.21.15.164:0/2917055477 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa41c00e040 con 0x7fa424145f50 2024-03-23T14:57:11.666 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.657+0000 7fa419ffb640 1 -- 172.21.15.164:0/2917055477 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa41c00f040 con 0x7fa424145f50 2024-03-23T14:57:11.666 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.657+0000 7fa419ffb640 1 -- 172.21.15.164:0/2917055477 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa41c015430 con 0x7fa424145f50 2024-03-23T14:57:11.666 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.657+0000 7fa42b578640 1 -- 172.21.15.164:0/2917055477 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa424147d70 con 0x7fa424145f50 2024-03-23T14:57:11.666 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.657+0000 7fa42b578640 1 -- 172.21.15.164:0/2917055477 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa42409b1b0 con 0x7fa424145f50 2024-03-23T14:57:11.666 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.661+0000 7fa42b578640 1 -- 172.21.15.164:0/2917055477 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4240690a0 con 0x7fa424145f50 2024-03-23T14:57:11.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.661+0000 7fa419ffb640 1 -- 172.21.15.164:0/2917055477 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7fa41c015590 con 0x7fa424145f50 2024-03-23T14:57:11.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.661+0000 7fa419ffb640 1 --2- 172.21.15.164:0/2917055477 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fa4080672a0 0x7fa408069760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:11.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.661+0000 7fa429314640 1 --2- 172.21.15.164:0/2917055477 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fa4080672a0 0x7fa408069760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:11.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.661+0000 7fa419ffb640 1 -- 172.21.15.164:0/2917055477 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa41c085ae0 con 0x7fa424145f50 2024-03-23T14:57:11.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.661+0000 7fa429314640 1 --2- 172.21.15.164:0/2917055477 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fa4080672a0 0x7fa408069760 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa4140047c0 tx=0x7fa414008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:11.669 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.665+0000 7fa419ffb640 1 -- 172.21.15.164:0/2917055477 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7fa41c052720 con 0x7fa424145f50 2024-03-23T14:57:11.872 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.865+0000 7fa42b578640 1 -- 172.21.15.164:0/2917055477 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa4240b2fa0 con 0x7fa424145f50 2024-03-23T14:57:11.872 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.869+0000 7fa419ffb640 1 -- 172.21.15.164:0/2917055477 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+3533 (secure 0 0 0) 0x7fa41c022de0 con 0x7fa424145f50 2024-03-23T14:57:11.873 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-23T14:57:11.873 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/3352372536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":3352372536},{"type":"v1","addr":"172.21.15.164:6835","nonce":3352372536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-23T14:56:58.953326+0000","modified":"2024-03-23T14:57:00.999623+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4279":{"gid":4279,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.173:6833/3506402771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":3506402771},{"type":"v1","addr":"172.21.15.173:6833","nonce":3506402771}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4279,"qdb_cluster":[4279]},"id":1}]} 2024-03-23T14:57:11.873 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 6 2024-03-23T14:57:11.875 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.869+0000 7fa42b578640 1 -- 172.21.15.164:0/2917055477 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fa4080672a0 msgr2=0x7fa408069760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:11.875 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.869+0000 7fa42b578640 1 --2- 172.21.15.164:0/2917055477 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fa4080672a0 0x7fa408069760 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa4140047c0 tx=0x7fa414008040 comp rx=0 tx=0).stop 2024-03-23T14:57:11.875 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.869+0000 7fa42b578640 1 -- 172.21.15.164:0/2917055477 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa424145f50 msgr2=0x7fa42413e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:11.875 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.869+0000 7fa42b578640 1 --2- 172.21.15.164:0/2917055477 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa424145f50 0x7fa42413e000 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa42409e2a0 tx=0x7fa41c00f580 comp rx=0 tx=0).stop 2024-03-23T14:57:11.875 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.869+0000 7fa42b578640 1 -- 172.21.15.164:0/2917055477 shutdown_connections 2024-03-23T14:57:11.875 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.869+0000 7fa42b578640 1 --2- 172.21.15.164:0/2917055477 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa424145f50 0x7fa42413e000 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:11.875 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.869+0000 7fa42b578640 1 --2- 172.21.15.164:0/2917055477 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fa4080672a0 0x7fa408069760 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:11.875 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.869+0000 7fa42b578640 1 --2- 172.21.15.164:0/2917055477 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa42409b700 0x7fa42409c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:11.875 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.869+0000 7fa42b578640 1 --2- 172.21.15.164:0/2917055477 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa410003f00 0x7fa42409d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:11.875 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.869+0000 7fa42b578640 1 -- 172.21.15.164:0/2917055477 >> 172.21.15.164:0/2917055477 conn(0x7fa4240b8ca0 msgr2=0x7fa4240b2310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:11.875 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.869+0000 7fa42b578640 1 -- 172.21.15.164:0/2917055477 shutdown_connections 2024-03-23T14:57:11.875 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:11.869+0000 7fa42b578640 1 -- 172.21.15.164:0/2917055477 wait complete. 2024-03-23T14:57:11.891 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T14:57:12.004 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.001+0000 7fc09e346640 1 Processor -- start 2024-03-23T14:57:12.004 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.001+0000 7fc09e346640 1 -- start start 2024-03-23T14:57:12.005 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.001+0000 7fc09e346640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fc0980ae070 0x7fc09814b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:12.005 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.001+0000 7fc09e346640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc09806c9c0 con 0x7fc09806bf70 2024-03-23T14:57:12.005 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.001+0000 7fc09e346640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0980af460 con 0x7fc0980adc90 2024-03-23T14:57:12.006 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.001+0000 7fc09e346640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fc0980ae810 con 0x7fc0980ae070 2024-03-23T14:57:12.006 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.001+0000 7fc09c8e3640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fc0980ae070 0x7fc09814b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:12.006 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.001+0000 7fc0977fe640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fc09806bf70 0x7fc09814ad20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:42560/0 (socket says 172.21.15.164:42560) 2024-03-23T14:57:12.006 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.001+0000 7fc0977fe640 1 -- 172.21.15.164:0/2624963536 learned_addr learned my addr 172.21.15.164:0/2624963536 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-03-23T14:57:12.006 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.001+0000 7fc09c8e3640 1 -- 172.21.15.164:0/2624963536 >> v1:172.21.15.173:6789/0 conn(0x7fc0980adc90 legacy=0x7fc0980b4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:12.006 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.001+0000 7fc09c8e3640 1 -- 172.21.15.164:0/2624963536 >> v1:172.21.15.164:6789/0 conn(0x7fc09806bf70 legacy=0x7fc09814ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:12.006 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.001+0000 7fc09c8e3640 1 -- 172.21.15.164:0/2624963536 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0980b5170 con 0x7fc0980ae070 2024-03-23T14:57:12.006 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.001+0000 7fc096ffd640 1 -- 172.21.15.164:0/2624963536 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2646470479 0 0) 0x7fc09806c9c0 con 0x7fc09806bf70 2024-03-23T14:57:12.008 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.001+0000 7fc09c8e3640 1 --2- 172.21.15.164:0/2624963536 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc0980ae070 0x7fc09814b430 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc090009dc0 tx=0x7fc09001c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=682b71f4d7e0c145 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:12.008 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.001+0000 7fc096ffd640 1 -- 172.21.15.164:0/2624963536 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0900a5800 con 0x7fc0980ae070 2024-03-23T14:57:12.008 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.001+0000 7fc096ffd640 1 -- 172.21.15.164:0/2624963536 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0900a5960 con 0x7fc0980ae070 2024-03-23T14:57:12.009 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.001+0000 7fc096ffd640 1 -- 172.21.15.164:0/2624963536 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0900a5cf0 con 0x7fc0980ae070 2024-03-23T14:57:12.009 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.005+0000 7fc09e346640 1 -- 172.21.15.164:0/2624963536 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc0980ae070 msgr2=0x7fc09814b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:12.009 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.005+0000 7fc09e346640 1 --2- 172.21.15.164:0/2624963536 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc0980ae070 0x7fc09814b430 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc090009dc0 tx=0x7fc09001c850 comp rx=0 tx=0).stop 2024-03-23T14:57:12.009 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.005+0000 7fc09e346640 1 -- 172.21.15.164:0/2624963536 shutdown_connections 2024-03-23T14:57:12.009 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.005+0000 7fc09e346640 1 --2- 172.21.15.164:0/2624963536 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc0980ae070 0x7fc09814b430 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:12.009 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.005+0000 7fc09e346640 1 -- 172.21.15.164:0/2624963536 >> 172.21.15.164:0/2624963536 conn(0x7fc0980b8ca0 msgr2=0x7fc0980b90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:12.009 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.005+0000 7fc09e346640 1 -- 172.21.15.164:0/2624963536 shutdown_connections 2024-03-23T14:57:12.009 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.005+0000 7fc09e346640 1 -- 172.21.15.164:0/2624963536 wait complete. 2024-03-23T14:57:12.009 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.005+0000 7fc09e346640 1 Processor -- start 2024-03-23T14:57:12.010 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.005+0000 7fc09e346640 1 -- start start 2024-03-23T14:57:12.012 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.005+0000 7fc09e346640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc0980adc90 0x7fc09814a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:12.012 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.005+0000 7fc09e346640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc0980ae070 0x7fc098146f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:12.012 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.005+0000 7fc09e346640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc0981474c0 0x7fc0981e2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:12.012 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.005+0000 7fc09e346640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc098149b40 con 0x7fc0981474c0 2024-03-23T14:57:12.012 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.005+0000 7fc09e346640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fc09814acc0 con 0x7fc0980ae070 2024-03-23T14:57:12.012 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.005+0000 7fc09e346640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fc098149fd0 con 0x7fc0980adc90 2024-03-23T14:57:12.012 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.005+0000 7fc09c8e3640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc0981474c0 0x7fc0981e2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:12.012 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.005+0000 7fc09c8e3640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc0981474c0 0x7fc0981e2ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:59454/0 (socket says 172.21.15.164:59454) 2024-03-23T14:57:12.012 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.005+0000 7fc09c8e3640 1 -- 172.21.15.164:0/3894333786 learned_addr learned my addr 172.21.15.164:0/3894333786 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:12.012 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.005+0000 7fc0977fe640 1 --2- 172.21.15.164:0/3894333786 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc0980ae070 0x7fc098146f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:12.012 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.005+0000 7fc097fff640 1 --2- 172.21.15.164:0/3894333786 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc0980adc90 0x7fc09814a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:12.012 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.005+0000 7fc09c8e3640 1 -- 172.21.15.164:0/3894333786 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc0980adc90 msgr2=0x7fc09814a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:12.013 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.005+0000 7fc09c8e3640 1 --2- 172.21.15.164:0/3894333786 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc0980adc90 0x7fc09814a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:12.013 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.005+0000 7fc09c8e3640 1 -- 172.21.15.164:0/3894333786 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc0980ae070 msgr2=0x7fc098146f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:12.013 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.005+0000 7fc09c8e3640 1 --2- 172.21.15.164:0/3894333786 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc0980ae070 0x7fc098146f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:12.013 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.005+0000 7fc09c8e3640 1 -- 172.21.15.164:0/3894333786 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc090009a70 con 0x7fc0981474c0 2024-03-23T14:57:12.013 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.005+0000 7fc09c8e3640 1 --2- 172.21.15.164:0/3894333786 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc0981474c0 0x7fc0981e2ac0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc09001a480 tx=0x7fc0900195f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:12.013 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.005+0000 7fc094ff9640 1 -- 172.21.15.164:0/3894333786 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0900b9860 con 0x7fc0981474c0 2024-03-23T14:57:12.013 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.005+0000 7fc094ff9640 1 -- 172.21.15.164:0/3894333786 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0900b99c0 con 0x7fc0981474c0 2024-03-23T14:57:12.013 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.005+0000 7fc094ff9640 1 -- 172.21.15.164:0/3894333786 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0900b9d30 con 0x7fc0981474c0 2024-03-23T14:57:12.013 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.005+0000 7fc09e346640 1 -- 172.21.15.164:0/3894333786 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc09814c730 con 0x7fc0981474c0 2024-03-23T14:57:12.013 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.005+0000 7fc09e346640 1 -- 172.21.15.164:0/3894333786 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc09814af20 con 0x7fc0981474c0 2024-03-23T14:57:12.013 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.009+0000 7fc09e346640 1 -- 172.21.15.164:0/3894333786 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc09814b110 con 0x7fc0981474c0 2024-03-23T14:57:12.014 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.009+0000 7fc094ff9640 1 -- 172.21.15.164:0/3894333786 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7fc0900ad440 con 0x7fc0981474c0 2024-03-23T14:57:12.014 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.009+0000 7fc094ff9640 1 --2- 172.21.15.164:0/3894333786 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fc0840672a0 0x7fc084069760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:12.014 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.009+0000 7fc094ff9640 1 -- 172.21.15.164:0/3894333786 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fc0901202f0 con 0x7fc0981474c0 2024-03-23T14:57:12.014 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.009+0000 7fc097fff640 1 --2- 172.21.15.164:0/3894333786 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fc0840672a0 0x7fc084069760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:12.014 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.009+0000 7fc097fff640 1 --2- 172.21.15.164:0/3894333786 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fc0840672a0 0x7fc084069760 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc09809da70 tx=0x7fc08c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:12.017 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.013+0000 7fc094ff9640 1 -- 172.21.15.164:0/3894333786 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7fc0900ecf30 con 0x7fc0981474c0 2024-03-23T14:57:12.166 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.161+0000 7fc09e346640 1 -- 172.21.15.164:0/3894333786 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc0980690a0 con 0x7fc0981474c0 2024-03-23T14:57:12.167 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.161+0000 7fc094ff9640 1 -- 172.21.15.164:0/3894333786 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16629 (secure 0 0 0) 0x7fc0900f05f0 con 0x7fc0981474c0 2024-03-23T14:57:12.167 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-23T14:57:12.168 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":20,"fsid":"45c7911b-b69d-4e68-907b-fbfeb780aa3b","created":"2024-03-23T14:56:34.367633+0000","modified":"2024-03-23T14:57:00.611265+0000","last_up_change":"2024-03-23T14:56:46.297403+0000","last_in_change":"2024-03-23T14:56:36.629884+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T14:56:47.186157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-23T14:56:56.994057+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-23T14:56:57.953862+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"80abdc62-bd0d-4e60-a554-7ed6fd67cf54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":984716690},{"type":"v1","addr":"172.21.15.164:6801","nonce":984716690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":984716690},{"type":"v1","addr":"172.21.15.164:6803","nonce":984716690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":984716690},{"type":"v1","addr":"172.21.15.164:6807","nonce":984716690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":984716690},{"type":"v1","addr":"172.21.15.164:6805","nonce":984716690}]},"public_addr":"172.21.15.164:6801/984716690","cluster_addr":"172.21.15.164:6803/984716690","heartbeat_back_addr":"172.21.15.164:6807/984716690","heartbeat_front_addr":"172.21.15.164:6805/984716690","state":["exists","up"]},{"osd":1,"uuid":"f9001b06-be4d-4f42-a0e6-bcef2afa0d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":1349657239},{"type":"v1","addr":"172.21.15.164:6817","nonce":1349657239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":1349657239},{"type":"v1","addr":"172.21.15.164:6819","nonce":1349657239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":1349657239},{"type":"v1","addr":"172.21.15.164:6823","nonce":1349657239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":1349657239},{"type":"v1","addr":"172.21.15.164:6821","nonce":1349657239}]},"public_addr":"172.21.15.164:6817/1349657239","cluster_addr":"172.21.15.164:6819/1349657239","heartbeat_back_addr":"172.21.15.164:6823/1349657239","heartbeat_front_addr":"172.21.15.164:6821/1349657239","state":["exists","up"]},{"osd":2,"uuid":"fa0aeb9f-8b54-4831-a188-3e361c7980d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":887861252},{"type":"v1","addr":"172.21.15.164:6809","nonce":887861252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":887861252},{"type":"v1","addr":"172.21.15.164:6811","nonce":887861252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":887861252},{"type":"v1","addr":"172.21.15.164:6815","nonce":887861252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":887861252},{"type":"v1","addr":"172.21.15.164:6813","nonce":887861252}]},"public_addr":"172.21.15.164:6809/887861252","cluster_addr":"172.21.15.164:6811/887861252","heartbeat_back_addr":"172.21.15.164:6815/887861252","heartbeat_front_addr":"172.21.15.164:6813/887861252","state":["exists","up"]},{"osd":3,"uuid":"7e18c957-3091-46d5-8e17-8f3354c1726c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":812540271},{"type":"v1","addr":"172.21.15.164:6825","nonce":812540271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":812540271},{"type":"v1","addr":"172.21.15.164:6827","nonce":812540271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":812540271},{"type":"v1","addr":"172.21.15.164:6831","nonce":812540271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":812540271},{"type":"v1","addr":"172.21.15.164:6829","nonce":812540271}]},"public_addr":"172.21.15.164:6825/812540271","cluster_addr":"172.21.15.164:6827/812540271","heartbeat_back_addr":"172.21.15.164:6831/812540271","heartbeat_front_addr":"172.21.15.164:6829/812540271","state":["exists","up"]},{"osd":4,"uuid":"8e83f315-42bf-4471-9ce8-d81948da0f9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":2656857421},{"type":"v1","addr":"172.21.15.173:6825","nonce":2656857421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2656857421},{"type":"v1","addr":"172.21.15.173:6827","nonce":2656857421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2656857421},{"type":"v1","addr":"172.21.15.173:6831","nonce":2656857421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":2656857421},{"type":"v1","addr":"172.21.15.173:6829","nonce":2656857421}]},"public_addr":"172.21.15.173:6825/2656857421","cluster_addr":"172.21.15.173:6827/2656857421","heartbeat_back_addr":"172.21.15.173:6831/2656857421","heartbeat_front_addr":"172.21.15.173:6829/2656857421","state":["exists","up"]},{"osd":5,"uuid":"a230c872-71f9-40ab-8999-defd0162d4ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":3676483185},{"type":"v1","addr":"172.21.15.173:6801","nonce":3676483185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":3676483185},{"type":"v1","addr":"172.21.15.173:6803","nonce":3676483185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":3676483185},{"type":"v1","addr":"172.21.15.173:6807","nonce":3676483185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":3676483185},{"type":"v1","addr":"172.21.15.173:6805","nonce":3676483185}]},"public_addr":"172.21.15.173:6801/3676483185","cluster_addr":"172.21.15.173:6803/3676483185","heartbeat_back_addr":"172.21.15.173:6807/3676483185","heartbeat_front_addr":"172.21.15.173:6805/3676483185","state":["exists","up"]},{"osd":6,"uuid":"6193bfd7-8432-4f7f-a117-a3fcee21af99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":3631684087},{"type":"v1","addr":"172.21.15.173:6809","nonce":3631684087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":3631684087},{"type":"v1","addr":"172.21.15.173:6811","nonce":3631684087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":3631684087},{"type":"v1","addr":"172.21.15.173:6815","nonce":3631684087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":3631684087},{"type":"v1","addr":"172.21.15.173:6813","nonce":3631684087}]},"public_addr":"172.21.15.173:6809/3631684087","cluster_addr":"172.21.15.173:6811/3631684087","heartbeat_back_addr":"172.21.15.173:6815/3631684087","heartbeat_front_addr":"172.21.15.173:6813/3631684087","state":["exists","up"]},{"osd":7,"uuid":"b0f1b80a-afc4-4538-83b9-f081d2dcb8cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":3261114948},{"type":"v1","addr":"172.21.15.173:6817","nonce":3261114948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":3261114948},{"type":"v1","addr":"172.21.15.173:6819","nonce":3261114948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":3261114948},{"type":"v1","addr":"172.21.15.173:6823","nonce":3261114948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":3261114948},{"type":"v1","addr":"172.21.15.173:6821","nonce":3261114948}]},"public_addr":"172.21.15.173:6817/3261114948","cluster_addr":"172.21.15.173:6819/3261114948","heartbeat_back_addr":"172.21.15.173:6823/3261114948","heartbeat_front_addr":"172.21.15.173:6821/3261114948","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.188390+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.159932+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.191101+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.181776+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.224365+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.177515+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.153631+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.139972+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T14:57:12.170 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.165+0000 7fc09e346640 1 -- 172.21.15.164:0/3894333786 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fc0840672a0 msgr2=0x7fc084069760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:12.170 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.165+0000 7fc09e346640 1 --2- 172.21.15.164:0/3894333786 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fc0840672a0 0x7fc084069760 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc09809da70 tx=0x7fc08c009040 comp rx=0 tx=0).stop 2024-03-23T14:57:12.170 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.165+0000 7fc09e346640 1 -- 172.21.15.164:0/3894333786 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc0981474c0 msgr2=0x7fc0981e2ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:12.170 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.165+0000 7fc09e346640 1 --2- 172.21.15.164:0/3894333786 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc0981474c0 0x7fc0981e2ac0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc09001a480 tx=0x7fc0900195f0 comp rx=0 tx=0).stop 2024-03-23T14:57:12.170 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.165+0000 7fc09e346640 1 -- 172.21.15.164:0/3894333786 shutdown_connections 2024-03-23T14:57:12.170 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.165+0000 7fc09e346640 1 --2- 172.21.15.164:0/3894333786 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc0981474c0 0x7fc0981e2ac0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:12.170 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.165+0000 7fc09e346640 1 --2- 172.21.15.164:0/3894333786 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fc0840672a0 0x7fc084069760 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:12.170 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.165+0000 7fc09e346640 1 --2- 172.21.15.164:0/3894333786 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc0980ae070 0x7fc098146f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:12.171 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.165+0000 7fc09e346640 1 --2- 172.21.15.164:0/3894333786 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc0980adc90 0x7fc09814a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:12.171 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.165+0000 7fc09e346640 1 -- 172.21.15.164:0/3894333786 >> 172.21.15.164:0/3894333786 conn(0x7fc0980b8ca0 msgr2=0x7fc0980b5cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:12.171 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.165+0000 7fc09e346640 1 -- 172.21.15.164:0/3894333786 shutdown_connections 2024-03-23T14:57:12.171 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.165+0000 7fc09e346640 1 -- 172.21.15.164:0/3894333786 wait complete. 2024-03-23T14:57:12.185 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T14:56:47.186157+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-23T14:56:56.994057+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-23T14:56:57.953862+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-23T14:57:12.185 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-23T14:57:12.296 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.293+0000 7f69597e5640 1 Processor -- start 2024-03-23T14:57:12.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.293+0000 7f69597e5640 1 -- start start 2024-03-23T14:57:12.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.293+0000 7f69597e5640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f695409def0 0x7f69540b45e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:12.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.293+0000 7f69597e5640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f695406d650 con 0x7f69540b4b20 2024-03-23T14:57:12.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.293+0000 7f69597e5640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f695406cd90 con 0x7f69540add50 2024-03-23T14:57:12.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.293+0000 7f69597e5640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f69540af4a0 con 0x7f695409def0 2024-03-23T14:57:12.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.293+0000 7f69537fe640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f69540b4b20 0x7f695406c680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:42568/0 (socket says 172.21.15.164:42568) 2024-03-23T14:57:12.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.293+0000 7f69537fe640 1 -- 172.21.15.164:0/3041867280 learned_addr learned my addr 172.21.15.164:0/3041867280 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-03-23T14:57:12.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.293+0000 7f6952ffd640 1 --2- 172.21.15.164:0/3041867280 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f695409def0 0x7f69540b45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:12.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.293+0000 7f6951ffb640 1 -- 172.21.15.164:0/3041867280 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3193674277 0 0) 0x7f695406d650 con 0x7f69540b4b20 2024-03-23T14:57:12.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.293+0000 7f6951ffb640 1 -- 172.21.15.164:0/3041867280 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f693c003530 con 0x7f69540b4b20 2024-03-23T14:57:12.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.293+0000 7f6951ffb640 1 -- 172.21.15.164:0/3041867280 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2604708687 0 0) 0x7f693c003530 con 0x7f69540b4b20 2024-03-23T14:57:12.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.293+0000 7f6951ffb640 1 -- 172.21.15.164:0/3041867280 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f695409def0 msgr2=0x7f69540b45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:12.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.293+0000 7f6951ffb640 1 --2- 172.21.15.164:0/3041867280 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f695409def0 0x7f69540b45e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:12.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.293+0000 7f6951ffb640 1 -- 172.21.15.164:0/3041867280 >> v1:172.21.15.173:6789/0 conn(0x7f69540add50 legacy=0x7f695406bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:12.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.293+0000 7f6951ffb640 1 -- 172.21.15.164:0/3041867280 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f695409e2d0 con 0x7f69540b4b20 2024-03-23T14:57:12.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.293+0000 7f6951ffb640 1 -- 172.21.15.164:0/3041867280 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2526390548 0 0) 0x7f694c001a00 con 0x7f69540b4b20 2024-03-23T14:57:12.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.293+0000 7f6951ffb640 1 -- 172.21.15.164:0/3041867280 >> v1:172.21.15.164:6789/0 conn(0x7f69540b4b20 legacy=0x7f695406c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:12.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.293+0000 7f6951ffb640 1 --2- 172.21.15.164:0/3041867280 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f693c0039b0 0x7f693c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:12.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.293+0000 7f6951ffb640 1 --2- 172.21.15.164:0/3041867280 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f693c0047d0 0x7f693c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:12.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.293+0000 7f6951ffb640 1 --2- 172.21.15.164:0/3041867280 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f695409def0 0x7f693c005260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:12.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.293+0000 7f6951ffb640 1 -- 172.21.15.164:0/3041867280 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f69540af4a0 con 0x7f693c0039b0 2024-03-23T14:57:12.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.293+0000 7f6951ffb640 1 -- 172.21.15.164:0/3041867280 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f693c00bb50 con 0x7f693c0047d0 2024-03-23T14:57:12.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.293+0000 7f6951ffb640 1 -- 172.21.15.164:0/3041867280 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f693c00bd10 con 0x7f695409def0 2024-03-23T14:57:12.301 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.293+0000 7f6952ffd640 1 --2- 172.21.15.164:0/3041867280 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f693c0039b0 0x7f693c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:12.301 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.293+0000 7f69527fc640 1 --2- 172.21.15.164:0/3041867280 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f693c0047d0 0x7f693c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:12.301 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.293+0000 7f69537fe640 1 --2- 172.21.15.164:0/3041867280 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f695409def0 0x7f693c005260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:12.301 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.293+0000 7f6952ffd640 1 -- 172.21.15.164:0/3041867280 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f695409def0 msgr2=0x7f693c005260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:12.301 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.293+0000 7f6952ffd640 1 --2- 172.21.15.164:0/3041867280 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f695409def0 0x7f693c005260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:12.301 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.293+0000 7f6952ffd640 1 -- 172.21.15.164:0/3041867280 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f693c0047d0 msgr2=0x7f693c004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:12.301 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.293+0000 7f6952ffd640 1 --2- 172.21.15.164:0/3041867280 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f693c0047d0 0x7f693c004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:12.301 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.293+0000 7f6952ffd640 1 -- 172.21.15.164:0/3041867280 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f693c003530 con 0x7f693c0039b0 2024-03-23T14:57:12.302 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.297+0000 7f6952ffd640 1 --2- 172.21.15.164:0/3041867280 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f693c0039b0 0x7f693c003dd0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6944009ac0 tx=0x7f694401ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fdc48cc6ef02022 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:12.302 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.297+0000 7f6951ffb640 1 -- 172.21.15.164:0/3041867280 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69440a5530 con 0x7f693c0039b0 2024-03-23T14:57:12.302 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.297+0000 7f6951ffb640 1 -- 172.21.15.164:0/3041867280 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69440a5690 con 0x7f693c0039b0 2024-03-23T14:57:12.302 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.297+0000 7f6951ffb640 1 -- 172.21.15.164:0/3041867280 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69440a5a20 con 0x7f693c0039b0 2024-03-23T14:57:12.302 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.297+0000 7f69597e5640 1 -- 172.21.15.164:0/3041867280 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f693c0039b0 msgr2=0x7f693c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:12.302 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.297+0000 7f69597e5640 1 --2- 172.21.15.164:0/3041867280 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f693c0039b0 0x7f693c003dd0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6944009ac0 tx=0x7f694401ca50 comp rx=0 tx=0).stop 2024-03-23T14:57:12.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.297+0000 7f69537fe640 1 -- 172.21.15.164:0/3041867280 reap_dead start 2024-03-23T14:57:12.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.297+0000 7f69597e5640 1 -- 172.21.15.164:0/3041867280 shutdown_connections 2024-03-23T14:57:12.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.297+0000 7f69597e5640 1 -- 172.21.15.164:0/3041867280 >> 172.21.15.164:0/3041867280 conn(0x7f69540b8ca0 msgr2=0x7f69540b90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:12.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.297+0000 7f69597e5640 1 -- 172.21.15.164:0/3041867280 shutdown_connections 2024-03-23T14:57:12.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.297+0000 7f69597e5640 1 -- 172.21.15.164:0/3041867280 wait complete. 2024-03-23T14:57:12.305 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.297+0000 7f69597e5640 1 Processor -- start 2024-03-23T14:57:12.305 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.297+0000 7f69597e5640 1 -- start start 2024-03-23T14:57:12.305 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.301+0000 7f69597e5640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f693c0039b0 0x7f6954140e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:12.305 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.301+0000 7f69597e5640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f69540b2770 0x7f6954143930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:12.305 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.301+0000 7f69597e5640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6954143e70 0x7f695413e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:12.305 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.301+0000 7f69597e5640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f69540b1e20 con 0x7f693c0039b0 2024-03-23T14:57:12.306 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.301+0000 7f69597e5640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f69540b1360 con 0x7f69540b2770 2024-03-23T14:57:12.306 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.301+0000 7f69597e5640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f69540b2c10 con 0x7f6954143e70 2024-03-23T14:57:12.306 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.301+0000 7f6952ffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f693c0039b0 0x7f6954140e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:12.306 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.301+0000 7f6952ffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f693c0039b0 0x7f6954140e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:59460/0 (socket says 172.21.15.164:59460) 2024-03-23T14:57:12.306 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.301+0000 7f6952ffd640 1 -- 172.21.15.164:0/1433915009 learned_addr learned my addr 172.21.15.164:0/1433915009 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:12.306 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.301+0000 7f69527fc640 1 --2- 172.21.15.164:0/1433915009 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f69540b2770 0x7f6954143930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:12.306 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.301+0000 7f69537fe640 1 --2- 172.21.15.164:0/1433915009 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6954143e70 0x7f695413e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:12.306 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.301+0000 7f6952ffd640 1 -- 172.21.15.164:0/1433915009 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6954143e70 msgr2=0x7f695413e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:12.306 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.301+0000 7f6952ffd640 1 --2- 172.21.15.164:0/1433915009 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6954143e70 0x7f695413e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:12.306 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.301+0000 7f6952ffd640 1 -- 172.21.15.164:0/1433915009 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f69540b2770 msgr2=0x7f6954143930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:12.306 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.301+0000 7f6952ffd640 1 --2- 172.21.15.164:0/1433915009 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f69540b2770 0x7f6954143930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:12.306 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.301+0000 7f6952ffd640 1 -- 172.21.15.164:0/1433915009 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69540b15d0 con 0x7f693c0039b0 2024-03-23T14:57:12.306 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.301+0000 7f69527fc640 1 --2- 172.21.15.164:0/1433915009 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f69540b2770 0x7f6954143930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T14:57:12.306 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.301+0000 7f6952ffd640 1 --2- 172.21.15.164:0/1433915009 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f693c0039b0 0x7f6954140e40 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f694401c880 tx=0x7f693c005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:12.306 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.301+0000 7f693b7fe640 1 -- 172.21.15.164:0/1433915009 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69440ad440 con 0x7f693c0039b0 2024-03-23T14:57:12.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.301+0000 7f693b7fe640 1 -- 172.21.15.164:0/1433915009 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69440ad5a0 con 0x7f693c0039b0 2024-03-23T14:57:12.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.301+0000 7f69597e5640 1 -- 172.21.15.164:0/1433915009 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69540b1900 con 0x7f693c0039b0 2024-03-23T14:57:12.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.301+0000 7f693b7fe640 1 -- 172.21.15.164:0/1433915009 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69440ad950 con 0x7f693c0039b0 2024-03-23T14:57:12.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.301+0000 7f69597e5640 1 -- 172.21.15.164:0/1433915009 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69540b0bc0 con 0x7f693c0039b0 2024-03-23T14:57:12.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.301+0000 7f69597e5640 1 -- 172.21.15.164:0/1433915009 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69540b1a90 con 0x7f693c0039b0 2024-03-23T14:57:12.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.301+0000 7f693b7fe640 1 -- 172.21.15.164:0/1433915009 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7f69440b4040 con 0x7f693c0039b0 2024-03-23T14:57:12.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.301+0000 7f693b7fe640 1 --2- 172.21.15.164:0/1433915009 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f69400672a0 0x7f6940069760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:12.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.301+0000 7f693b7fe640 1 -- 172.21.15.164:0/1433915009 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f694411eb90 con 0x7f693c0039b0 2024-03-23T14:57:12.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.301+0000 7f69527fc640 1 --2- 172.21.15.164:0/1433915009 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f69400672a0 0x7f6940069760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:12.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.301+0000 7f69527fc640 1 --2- 172.21.15.164:0/1433915009 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f69400672a0 0x7f6940069760 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f695413f430 tx=0x7f6948009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:12.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.305+0000 7f693b7fe640 1 -- 172.21.15.164:0/1433915009 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7f69440eb7d0 con 0x7f693c0039b0 2024-03-23T14:57:12.455 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.449+0000 7f69597e5640 1 -- 172.21.15.164:0/1433915009 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f69540690a0 con 0x7f693c0039b0 2024-03-23T14:57:12.456 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.453+0000 7f693b7fe640 1 -- 172.21.15.164:0/1433915009 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f69440b20a0 con 0x7f693c0039b0 2024-03-23T14:57:12.456 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 1 2024-03-23T14:57:12.459 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.453+0000 7f69597e5640 1 -- 172.21.15.164:0/1433915009 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f69400672a0 msgr2=0x7f6940069760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:12.459 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.453+0000 7f69597e5640 1 --2- 172.21.15.164:0/1433915009 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f69400672a0 0x7f6940069760 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f695413f430 tx=0x7f6948009040 comp rx=0 tx=0).stop 2024-03-23T14:57:12.459 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.453+0000 7f69597e5640 1 -- 172.21.15.164:0/1433915009 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f693c0039b0 msgr2=0x7f6954140e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:12.459 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.453+0000 7f69597e5640 1 --2- 172.21.15.164:0/1433915009 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f693c0039b0 0x7f6954140e40 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f694401c880 tx=0x7f693c005800 comp rx=0 tx=0).stop 2024-03-23T14:57:12.459 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.453+0000 7f69597e5640 1 -- 172.21.15.164:0/1433915009 shutdown_connections 2024-03-23T14:57:12.459 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.453+0000 7f69597e5640 1 --2- 172.21.15.164:0/1433915009 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6954143e70 0x7f695413e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:12.459 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.453+0000 7f69597e5640 1 --2- 172.21.15.164:0/1433915009 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f69400672a0 0x7f6940069760 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:12.459 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.453+0000 7f69597e5640 1 --2- 172.21.15.164:0/1433915009 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f69540b2770 0x7f6954143930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:12.459 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.453+0000 7f69597e5640 1 --2- 172.21.15.164:0/1433915009 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f693c0039b0 0x7f6954140e40 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:12.459 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.453+0000 7f69597e5640 1 -- 172.21.15.164:0/1433915009 >> 172.21.15.164:0/1433915009 conn(0x7f69540b8ca0 msgr2=0x7f695414f170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:12.459 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.453+0000 7f69597e5640 1 -- 172.21.15.164:0/1433915009 shutdown_connections 2024-03-23T14:57:12.459 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.453+0000 7f69597e5640 1 -- 172.21.15.164:0/1433915009 wait complete. 2024-03-23T14:57:12.474 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-23T14:57:12.592 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.589+0000 7f04d8b9e640 1 Processor -- start 2024-03-23T14:57:12.592 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.589+0000 7f04d8b9e640 1 -- start start 2024-03-23T14:57:12.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.589+0000 7f04d8b9e640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f04d40b0230 0x7f04d40b34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:12.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.589+0000 7f04d8b9e640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f04d414db70 0x7f04d414ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:12.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.589+0000 7f04d8b9e640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04d40b51d0 con 0x7f04d4148cf0 2024-03-23T14:57:12.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.589+0000 7f04d8b9e640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 -- 0x7f04d40b0610 con 0x7f04d414db70 2024-03-23T14:57:12.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.589+0000 7f04d8b9e640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f04d40b1500 con 0x7f04d40b0230 2024-03-23T14:57:12.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.589+0000 7f04d259c640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f04d40b0230 0x7f04d40b34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:12.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.589+0000 7f04d259c640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f04d40b0230 0x7f04d40b34d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.164:60498/0 (socket says 172.21.15.164:60498) 2024-03-23T14:57:12.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.589+0000 7f04d259c640 1 -- 172.21.15.164:0/4271208666 learned_addr learned my addr 172.21.15.164:0/4271208666 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:12.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.589+0000 7f04d2d9d640 1 --2- 172.21.15.164:0/4271208666 >> v2:172.21.15.173:3300/0 conn(0x7f04d414db70 0x7f04d414ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:12.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.589+0000 7f04d259c640 1 -- 172.21.15.164:0/4271208666 >> v2:172.21.15.173:3300/0 conn(0x7f04d414db70 msgr2=0x7f04d414ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:12.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.589+0000 7f04d259c640 1 --2- 172.21.15.164:0/4271208666 >> v2:172.21.15.173:3300/0 conn(0x7f04d414db70 0x7f04d414ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:12.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.589+0000 7f04d259c640 1 -- 172.21.15.164:0/4271208666 >> v1:172.21.15.173:6789/0 conn(0x7f04d4148cf0 legacy=0x7f04d4149150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:12.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.589+0000 7f04d259c640 1 -- 172.21.15.164:0/4271208666 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04d406d250 con 0x7f04d40b0230 2024-03-23T14:57:12.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.589+0000 7f04d2d9d640 1 --2- 172.21.15.164:0/4271208666 >> v2:172.21.15.173:3300/0 conn(0x7f04d414db70 0x7f04d414ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T14:57:12.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.593+0000 7f04d259c640 1 --2- 172.21.15.164:0/4271208666 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f04d40b0230 0x7f04d40b34d0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f04c4009870 tx=0x7f04c401c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c6be879cf27be6bc server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:12.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.593+0000 7f04d1d9b640 1 -- 172.21.15.164:0/4271208666 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c40a6040 con 0x7f04d40b0230 2024-03-23T14:57:12.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.593+0000 7f04d1d9b640 1 -- 172.21.15.164:0/4271208666 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04c40aa600 con 0x7f04d40b0230 2024-03-23T14:57:12.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.593+0000 7f04d1d9b640 1 -- 172.21.15.164:0/4271208666 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c40aa990 con 0x7f04d40b0230 2024-03-23T14:57:12.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.593+0000 7f04d8b9e640 1 -- 172.21.15.164:0/4271208666 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f04d40b0230 msgr2=0x7f04d40b34d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:12.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.593+0000 7f04d8b9e640 1 --2- 172.21.15.164:0/4271208666 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f04d40b0230 0x7f04d40b34d0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f04c4009870 tx=0x7f04c401c4a0 comp rx=0 tx=0).stop 2024-03-23T14:57:12.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.593+0000 7f04d8b9e640 1 -- 172.21.15.164:0/4271208666 shutdown_connections 2024-03-23T14:57:12.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.593+0000 7f04d8b9e640 1 --2- 172.21.15.164:0/4271208666 >> v2:172.21.15.173:3300/0 conn(0x7f04d414db70 0x7f04d414ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:12.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.593+0000 7f04d8b9e640 1 --2- 172.21.15.164:0/4271208666 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f04d40b0230 0x7f04d40b34d0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:12.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.593+0000 7f04d8b9e640 1 -- 172.21.15.164:0/4271208666 >> 172.21.15.164:0/4271208666 conn(0x7f04d40b8ca0 msgr2=0x7f04d40b90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:12.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.593+0000 7f04d8b9e640 1 -- 172.21.15.164:0/4271208666 shutdown_connections 2024-03-23T14:57:12.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.593+0000 7f04d8b9e640 1 -- 172.21.15.164:0/4271208666 wait complete. 2024-03-23T14:57:12.599 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.593+0000 7f04d8b9e640 1 Processor -- start 2024-03-23T14:57:12.599 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.593+0000 7f04d8b9e640 1 -- start start 2024-03-23T14:57:12.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.593+0000 7f04d8b9e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f04d40b0230 0x7f04d409d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:12.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.593+0000 7f04d8b9e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f04d4148cf0 0x7f04d40700c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:12.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.593+0000 7f04d8b9e640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f04d414db70 0x7f04d40709f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:12.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.593+0000 7f04d8b9e640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f04d406fc30 con 0x7f04d40b0230 2024-03-23T14:57:12.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.593+0000 7f04d8b9e640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f04d409c5c0 con 0x7f04d4148cf0 2024-03-23T14:57:12.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.593+0000 7f04d8b9e640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f04d4070600 con 0x7f04d414db70 2024-03-23T14:57:12.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.593+0000 7f04d259c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f04d40b0230 0x7f04d409d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:12.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.593+0000 7f04d259c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f04d40b0230 0x7f04d409d3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:60102/0 (socket says 172.21.15.164:60102) 2024-03-23T14:57:12.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.593+0000 7f04d259c640 1 -- 172.21.15.164:0/408677923 learned_addr learned my addr 172.21.15.164:0/408677923 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:12.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.593+0000 7f04d2d9d640 1 --2- 172.21.15.164:0/408677923 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f04d414db70 0x7f04d40709f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:12.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.593+0000 7f04cbfff640 1 --2- 172.21.15.164:0/408677923 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f04d4148cf0 0x7f04d40700c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:12.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.593+0000 7f04d259c640 1 -- 172.21.15.164:0/408677923 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f04d414db70 msgr2=0x7f04d40709f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:12.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.593+0000 7f04d259c640 1 --2- 172.21.15.164:0/408677923 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f04d414db70 0x7f04d40709f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:12.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.593+0000 7f04d259c640 1 -- 172.21.15.164:0/408677923 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f04d4148cf0 msgr2=0x7f04d40700c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:12.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.593+0000 7f04d259c640 1 --2- 172.21.15.164:0/408677923 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f04d4148cf0 0x7f04d40700c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:12.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.593+0000 7f04d259c640 1 -- 172.21.15.164:0/408677923 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04cc009a70 con 0x7f04d40b0230 2024-03-23T14:57:12.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.593+0000 7f04d2d9d640 1 --2- 172.21.15.164:0/408677923 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f04d414db70 0x7f04d40709f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T14:57:12.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.593+0000 7f04cbfff640 1 --2- 172.21.15.164:0/408677923 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f04d4148cf0 0x7f04d40700c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T14:57:12.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.593+0000 7f04d259c640 1 --2- 172.21.15.164:0/408677923 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f04d40b0230 0x7f04d409d3e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f04c401a0d0 tx=0x7f04c401ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:12.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.593+0000 7f04caffd640 1 -- 172.21.15.164:0/408677923 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c40b3600 con 0x7f04d40b0230 2024-03-23T14:57:12.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.593+0000 7f04caffd640 1 -- 172.21.15.164:0/408677923 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04c40b3780 con 0x7f04d40b0230 2024-03-23T14:57:12.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.593+0000 7f04d8b9e640 1 -- 172.21.15.164:0/408677923 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04c40094d0 con 0x7f04d40b0230 2024-03-23T14:57:12.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.593+0000 7f04caffd640 1 -- 172.21.15.164:0/408677923 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c40b3b30 con 0x7f04d40b0230 2024-03-23T14:57:12.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.593+0000 7f04d8b9e640 1 -- 172.21.15.164:0/408677923 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04d406f8c0 con 0x7f04d40b0230 2024-03-23T14:57:12.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.597+0000 7f04d8b9e640 1 -- 172.21.15.164:0/408677923 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04d40690a0 con 0x7f04d40b0230 2024-03-23T14:57:12.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.597+0000 7f04caffd640 1 -- 172.21.15.164:0/408677923 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7f04c40b3c90 con 0x7f04d40b0230 2024-03-23T14:57:12.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.597+0000 7f04caffd640 1 --2- 172.21.15.164:0/408677923 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f04b40672a0 0x7f04b4069760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:12.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.597+0000 7f04caffd640 1 -- 172.21.15.164:0/408677923 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f04c411fb30 con 0x7f04d40b0230 2024-03-23T14:57:12.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.597+0000 7f04cbfff640 1 --2- 172.21.15.164:0/408677923 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f04b40672a0 0x7f04b4069760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:12.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.597+0000 7f04cbfff640 1 --2- 172.21.15.164:0/408677923 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f04b40672a0 0x7f04b4069760 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f04d4149450 tx=0x7f04bc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:12.605 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.601+0000 7f04caffd640 1 -- 172.21.15.164:0/408677923 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7f04c40ec770 con 0x7f04d40b0230 2024-03-23T14:57:12.760 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.753+0000 7f04d8b9e640 1 -- 172.21.15.164:0/408677923 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f04d40b2fa0 con 0x7f04d40b0230 2024-03-23T14:57:12.760 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.757+0000 7f04caffd640 1 -- 172.21.15.164:0/408677923 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f04c40efe30 con 0x7f04d40b0230 2024-03-23T14:57:12.761 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 32 2024-03-23T14:57:12.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.757+0000 7f04d8b9e640 1 -- 172.21.15.164:0/408677923 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f04b40672a0 msgr2=0x7f04b4069760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:12.763 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.757+0000 7f04d8b9e640 1 --2- 172.21.15.164:0/408677923 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f04b40672a0 0x7f04b4069760 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f04d4149450 tx=0x7f04bc009040 comp rx=0 tx=0).stop 2024-03-23T14:57:12.763 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.757+0000 7f04d8b9e640 1 -- 172.21.15.164:0/408677923 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f04d40b0230 msgr2=0x7f04d409d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:12.763 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.757+0000 7f04d8b9e640 1 --2- 172.21.15.164:0/408677923 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f04d40b0230 0x7f04d409d3e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f04c401a0d0 tx=0x7f04c401ace0 comp rx=0 tx=0).stop 2024-03-23T14:57:12.763 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.757+0000 7f04d8b9e640 1 -- 172.21.15.164:0/408677923 shutdown_connections 2024-03-23T14:57:12.763 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.757+0000 7f04d8b9e640 1 --2- 172.21.15.164:0/408677923 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f04d414db70 0x7f04d40709f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:12.763 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.757+0000 7f04d8b9e640 1 --2- 172.21.15.164:0/408677923 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f04b40672a0 0x7f04b4069760 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:12.763 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.757+0000 7f04d8b9e640 1 --2- 172.21.15.164:0/408677923 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f04d4148cf0 0x7f04d40700c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:12.763 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.757+0000 7f04d8b9e640 1 --2- 172.21.15.164:0/408677923 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f04d40b0230 0x7f04d409d3e0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:12.763 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.757+0000 7f04d8b9e640 1 -- 172.21.15.164:0/408677923 >> 172.21.15.164:0/408677923 conn(0x7f04d40b8ca0 msgr2=0x7f04d40b43f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:12.763 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.757+0000 7f04d8b9e640 1 -- 172.21.15.164:0/408677923 shutdown_connections 2024-03-23T14:57:12.763 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.757+0000 7f04d8b9e640 1 -- 172.21.15.164:0/408677923 wait complete. 2024-03-23T14:57:12.778 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-23T14:57:12.895 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.889+0000 7f7fc8b8c640 1 Processor -- start 2024-03-23T14:57:12.895 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.889+0000 7f7fc8b8c640 1 -- start start 2024-03-23T14:57:12.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.889+0000 7f7fc8b8c640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7fc40b56f0 0x7f7fc414a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:12.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.889+0000 7f7fc8b8c640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f7fc40b4a70 0x7f7fc4140f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:12.897 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.889+0000 7f7fc8b8c640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7fc40b4300 con 0x7f7fc4141470 2024-03-23T14:57:12.897 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.889+0000 7f7fc8b8c640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f7fc40b5fa0 con 0x7f7fc40b4a70 2024-03-23T14:57:12.897 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.889+0000 7f7fc8b8c640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f7fc40b45e0 con 0x7f7fc40b56f0 2024-03-23T14:57:12.897 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.893+0000 7f7fc1d9b640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f7fc40b4a70 0x7f7fc4140f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:12.897 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.893+0000 7f7fc1d9b640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f7fc40b4a70 0x7f7fc4140f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:60110/0 (socket says 172.21.15.164:60110) 2024-03-23T14:57:12.897 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.893+0000 7f7fc1d9b640 1 -- 172.21.15.164:0/2864532903 learned_addr learned my addr 172.21.15.164:0/2864532903 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:12.897 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.893+0000 7f7fc259c640 1 --2- 172.21.15.164:0/2864532903 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7fc40b56f0 0x7f7fc414a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:12.897 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.893+0000 7f7fc1d9b640 1 -- 172.21.15.164:0/2864532903 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7fc40b56f0 msgr2=0x7f7fc414a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:12.897 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.893+0000 7f7fc1d9b640 1 --2- 172.21.15.164:0/2864532903 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7fc40b56f0 0x7f7fc414a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:12.897 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.893+0000 7f7fc1d9b640 1 -- 172.21.15.164:0/2864532903 >> v1:172.21.15.173:6789/0 conn(0x7f7fc4141470 legacy=0x7f7fc40b75f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:12.897 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.893+0000 7f7fc1d9b640 1 -- 172.21.15.164:0/2864532903 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fc414ade0 con 0x7f7fc40b4a70 2024-03-23T14:57:12.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.893+0000 7f7fc1d9b640 1 --2- 172.21.15.164:0/2864532903 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7fc40b4a70 0x7f7fc4140f30 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7fb8009870 tx=0x7f7fb801c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=564019963ba0c813 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:12.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.893+0000 7f7fc159a640 1 -- 172.21.15.164:0/2864532903 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fb80ac640 con 0x7f7fc40b4a70 2024-03-23T14:57:12.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.893+0000 7f7fc159a640 1 -- 172.21.15.164:0/2864532903 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fb80ac7a0 con 0x7f7fc40b4a70 2024-03-23T14:57:12.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.893+0000 7f7fc159a640 1 -- 172.21.15.164:0/2864532903 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fb80acb30 con 0x7f7fc40b4a70 2024-03-23T14:57:12.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.893+0000 7f7fc8b8c640 1 -- 172.21.15.164:0/2864532903 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7fc40b4a70 msgr2=0x7f7fc4140f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:12.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.893+0000 7f7fc8b8c640 1 --2- 172.21.15.164:0/2864532903 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7fc40b4a70 0x7f7fc4140f30 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7fb8009870 tx=0x7f7fb801c500 comp rx=0 tx=0).stop 2024-03-23T14:57:12.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.893+0000 7f7fc8b8c640 1 -- 172.21.15.164:0/2864532903 shutdown_connections 2024-03-23T14:57:12.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.893+0000 7f7fc8b8c640 1 --2- 172.21.15.164:0/2864532903 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7fc40b4a70 0x7f7fc4140f30 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:12.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.893+0000 7f7fc8b8c640 1 --2- 172.21.15.164:0/2864532903 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7fc40b56f0 0x7f7fc414a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:12.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.893+0000 7f7fc8b8c640 1 -- 172.21.15.164:0/2864532903 >> 172.21.15.164:0/2864532903 conn(0x7f7fc402c3c0 msgr2=0x7f7fc402c7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:12.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.893+0000 7f7fc8b8c640 1 -- 172.21.15.164:0/2864532903 shutdown_connections 2024-03-23T14:57:12.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.893+0000 7f7fc8b8c640 1 -- 172.21.15.164:0/2864532903 wait complete. 2024-03-23T14:57:12.902 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.897+0000 7f7fc8b8c640 1 Processor -- start 2024-03-23T14:57:12.902 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.897+0000 7f7fc8b8c640 1 -- start start 2024-03-23T14:57:12.902 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.897+0000 7f7fc8b8c640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7fc40b4a70 0x7f7fc41e2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:12.902 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.897+0000 7f7fc8b8c640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7fc40b56f0 0x7f7fc413b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:12.902 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.897+0000 7f7fc8b8c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7fc4141470 0x7f7fc413b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:12.902 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.897+0000 7f7fc8b8c640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f7fc4143a50 con 0x7f7fc4141470 2024-03-23T14:57:12.902 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.897+0000 7f7fc8b8c640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f7fc413deb0 con 0x7f7fc40b4a70 2024-03-23T14:57:12.902 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.897+0000 7f7fc8b8c640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f7fc413e190 con 0x7f7fc40b56f0 2024-03-23T14:57:12.903 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.897+0000 7f7fc2d9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7fc4141470 0x7f7fc413b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:12.903 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.897+0000 7f7fc2d9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7fc4141470 0x7f7fc413b8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:60120/0 (socket says 172.21.15.164:60120) 2024-03-23T14:57:12.903 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.897+0000 7f7fc2d9d640 1 -- 172.21.15.164:0/2611474866 learned_addr learned my addr 172.21.15.164:0/2611474866 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:12.903 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.897+0000 7f7fc259c640 1 --2- 172.21.15.164:0/2611474866 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7fc40b4a70 0x7f7fc41e2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:12.903 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.897+0000 7f7fc1d9b640 1 --2- 172.21.15.164:0/2611474866 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7fc40b56f0 0x7f7fc413b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:12.903 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.897+0000 7f7fc2d9d640 1 -- 172.21.15.164:0/2611474866 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7fc40b56f0 msgr2=0x7f7fc413b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:12.903 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.897+0000 7f7fc2d9d640 1 --2- 172.21.15.164:0/2611474866 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7fc40b56f0 0x7f7fc413b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:12.903 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.897+0000 7f7fc2d9d640 1 -- 172.21.15.164:0/2611474866 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7fc40b4a70 msgr2=0x7f7fc41e2dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:12.903 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.897+0000 7f7fc2d9d640 1 --2- 172.21.15.164:0/2611474866 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7fc40b4a70 0x7f7fc41e2dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:12.903 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.897+0000 7f7fc2d9d640 1 -- 172.21.15.164:0/2611474866 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fb80094d0 con 0x7f7fc4141470 2024-03-23T14:57:12.903 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.897+0000 7f7fc1d9b640 1 --2- 172.21.15.164:0/2611474866 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7fc40b56f0 0x7f7fc413b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T14:57:12.903 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.897+0000 7f7fc259c640 1 --2- 172.21.15.164:0/2611474866 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7fc40b4a70 0x7f7fc41e2dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T14:57:12.903 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.897+0000 7f7fc2d9d640 1 --2- 172.21.15.164:0/2611474866 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7fc4141470 0x7f7fc413b8d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc00b970 tx=0x7f7fbc00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:12.903 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.897+0000 7f7fb2ffd640 1 -- 172.21.15.164:0/2611474866 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fbc00cb40 con 0x7f7fc4141470 2024-03-23T14:57:12.903 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.897+0000 7f7fb2ffd640 1 -- 172.21.15.164:0/2611474866 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fbc00cca0 con 0x7f7fc4141470 2024-03-23T14:57:12.904 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.897+0000 7f7fb2ffd640 1 -- 172.21.15.164:0/2611474866 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fbc014680 con 0x7f7fc4141470 2024-03-23T14:57:12.904 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.897+0000 7f7fc8b8c640 1 -- 172.21.15.164:0/2611474866 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fc4144280 con 0x7f7fc4141470 2024-03-23T14:57:12.904 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.897+0000 7f7fc8b8c640 1 -- 172.21.15.164:0/2611474866 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fc41e38a0 con 0x7f7fc4141470 2024-03-23T14:57:12.904 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.897+0000 7f7fc8b8c640 1 -- 172.21.15.164:0/2611474866 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fc4143b90 con 0x7f7fc4141470 2024-03-23T14:57:12.904 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.897+0000 7f7fb2ffd640 1 -- 172.21.15.164:0/2611474866 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7f7fbc01d3b0 con 0x7f7fc4141470 2024-03-23T14:57:12.904 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.897+0000 7f7fb2ffd640 1 --2- 172.21.15.164:0/2611474866 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f7fa40672a0 0x7f7fa4069760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:12.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.897+0000 7f7fb2ffd640 1 -- 172.21.15.164:0/2611474866 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f7fbc084db0 con 0x7f7fc4141470 2024-03-23T14:57:12.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.897+0000 7f7fc259c640 1 --2- 172.21.15.164:0/2611474866 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f7fa40672a0 0x7f7fa4069760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:12.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.897+0000 7f7fc259c640 1 --2- 172.21.15.164:0/2611474866 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f7fa40672a0 0x7f7fa4069760 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f7fc413acd0 tx=0x7f7fb4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:12.907 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:12.901+0000 7f7fb2ffd640 1 -- 172.21.15.164:0/2611474866 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7f7fc4143b90 con 0x7f7fc4141470 2024-03-23T14:57:13.065 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.061+0000 7f7fc8b8c640 1 -- 172.21.15.164:0/2611474866 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f7fc4143b90 con 0x7f7fc4141470 2024-03-23T14:57:13.065 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.061+0000 7f7fb2ffd640 1 -- 172.21.15.164:0/2611474866 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f7fc4143b90 con 0x7f7fc4141470 2024-03-23T14:57:13.065 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 64 2024-03-23T14:57:13.068 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.061+0000 7f7fc8b8c640 1 -- 172.21.15.164:0/2611474866 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f7fa40672a0 msgr2=0x7f7fa4069760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:13.068 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.061+0000 7f7fc8b8c640 1 --2- 172.21.15.164:0/2611474866 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f7fa40672a0 0x7f7fa4069760 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f7fc413acd0 tx=0x7f7fb4008040 comp rx=0 tx=0).stop 2024-03-23T14:57:13.068 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.061+0000 7f7fc8b8c640 1 -- 172.21.15.164:0/2611474866 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7fc4141470 msgr2=0x7f7fc413b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:13.068 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.061+0000 7f7fc8b8c640 1 --2- 172.21.15.164:0/2611474866 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7fc4141470 0x7f7fc413b8d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc00b970 tx=0x7f7fbc00be30 comp rx=0 tx=0).stop 2024-03-23T14:57:13.068 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.061+0000 7f7fc8b8c640 1 -- 172.21.15.164:0/2611474866 shutdown_connections 2024-03-23T14:57:13.068 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.061+0000 7f7fc8b8c640 1 --2- 172.21.15.164:0/2611474866 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7fc4141470 0x7f7fc413b8d0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:13.068 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.061+0000 7f7fc8b8c640 1 --2- 172.21.15.164:0/2611474866 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7fc40b56f0 0x7f7fc413b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:13.068 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.061+0000 7f7fc8b8c640 1 --2- 172.21.15.164:0/2611474866 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f7fa40672a0 0x7f7fa4069760 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:13.068 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.061+0000 7f7fc8b8c640 1 --2- 172.21.15.164:0/2611474866 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7fc40b4a70 0x7f7fc41e2dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:13.068 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.061+0000 7f7fc8b8c640 1 -- 172.21.15.164:0/2611474866 >> 172.21.15.164:0/2611474866 conn(0x7f7fc402c3c0 msgr2=0x7f7fc40bc350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:13.069 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.061+0000 7f7fc8b8c640 1 -- 172.21.15.164:0/2611474866 shutdown_connections 2024-03-23T14:57:13.069 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.061+0000 7f7fc8b8c640 1 -- 172.21.15.164:0/2611474866 wait complete. 2024-03-23T14:57:13.084 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T14:57:13.200 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.197+0000 7f386a8c9640 1 Processor -- start 2024-03-23T14:57:13.200 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.197+0000 7f386a8c9640 1 -- start start 2024-03-23T14:57:13.203 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.197+0000 7f386a8c9640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f38640add50 0x7f3864147770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:13.203 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.197+0000 7f386a8c9640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f3864147cb0 0x7f3864148140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:13.203 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.197+0000 7f386a8c9640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f386406d650 con 0x7f386409def0 2024-03-23T14:57:13.203 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.197+0000 7f386a8c9640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 -- 0x7f38640af4a0 con 0x7f3864147cb0 2024-03-23T14:57:13.203 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.197+0000 7f386a8c9640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f386409e2d0 con 0x7f38640add50 2024-03-23T14:57:13.203 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.197+0000 7f3863fff640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f386409def0 0x7f38640b45e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:56854/0 (socket says 172.21.15.164:56854) 2024-03-23T14:57:13.203 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.197+0000 7f3863fff640 1 -- 172.21.15.164:0/1977871757 learned_addr learned my addr 172.21.15.164:0/1977871757 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-03-23T14:57:13.203 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.197+0000 7f3868e66640 1 --2- 172.21.15.164:0/1977871757 >> v2:172.21.15.173:3300/0 conn(0x7f3864147cb0 0x7f3864148140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:13.203 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.197+0000 7f38637fe640 1 --2- 172.21.15.164:0/1977871757 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f38640add50 0x7f3864147770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:13.203 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.197+0000 7f3862ffd640 1 -- 172.21.15.164:0/1977871757 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3079301479 0 0) 0x7f386406d650 con 0x7f386409def0 2024-03-23T14:57:13.203 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.197+0000 7f3862ffd640 1 -- 172.21.15.164:0/1977871757 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3850003530 con 0x7f386409def0 2024-03-23T14:57:13.204 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.197+0000 7f3862ffd640 1 -- 172.21.15.164:0/1977871757 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1950247 0 0) 0x7f3850003530 con 0x7f386409def0 2024-03-23T14:57:13.204 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.197+0000 7f3862ffd640 1 -- 172.21.15.164:0/1977871757 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f38640add50 msgr2=0x7f3864147770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:13.204 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.197+0000 7f3862ffd640 1 --2- 172.21.15.164:0/1977871757 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f38640add50 0x7f3864147770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:13.204 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.197+0000 7f3862ffd640 1 -- 172.21.15.164:0/1977871757 >> v2:172.21.15.173:3300/0 conn(0x7f3864147cb0 msgr2=0x7f3864148140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:13.204 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.197+0000 7f3862ffd640 1 --2- 172.21.15.164:0/1977871757 >> v2:172.21.15.173:3300/0 conn(0x7f3864147cb0 0x7f3864148140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:13.204 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.197+0000 7f3862ffd640 1 -- 172.21.15.164:0/1977871757 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38640aeca0 con 0x7f386409def0 2024-03-23T14:57:13.204 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.197+0000 7f3862ffd640 1 -- 172.21.15.164:0/1977871757 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2526390548 0 0) 0x7f3854001680 con 0x7f386409def0 2024-03-23T14:57:13.204 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.197+0000 7f3862ffd640 1 -- 172.21.15.164:0/1977871757 >> v1:172.21.15.164:6789/0 conn(0x7f386409def0 legacy=0x7f38640b45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:13.204 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.197+0000 7f3862ffd640 1 --2- 172.21.15.164:0/1977871757 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f38640add50 0x7f3850003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:13.204 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.197+0000 7f3862ffd640 1 --2- 172.21.15.164:0/1977871757 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3850003f00 0x7f38500047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:13.204 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.197+0000 7f3862ffd640 1 --2- 172.21.15.164:0/1977871757 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3850004d10 0x7f3850005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:13.204 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.197+0000 7f3862ffd640 1 -- 172.21.15.164:0/1977871757 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f38640af4a0 con 0x7f3850003f00 2024-03-23T14:57:13.204 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.197+0000 7f3862ffd640 1 -- 172.21.15.164:0/1977871757 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f386409e2d0 con 0x7f3850004d10 2024-03-23T14:57:13.204 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.197+0000 7f3862ffd640 1 -- 172.21.15.164:0/1977871757 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f385000ba70 con 0x7f38640add50 2024-03-23T14:57:13.204 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.197+0000 7f38637fe640 1 --2- 172.21.15.164:0/1977871757 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3850003f00 0x7f38500047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:13.204 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.197+0000 7f3868e66640 1 --2- 172.21.15.164:0/1977871757 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3850004d10 0x7f3850005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:13.205 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.197+0000 7f38637fe640 1 -- 172.21.15.164:0/1977871757 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f38640add50 msgr2=0x7f3850003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:13.205 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.197+0000 7f38637fe640 1 --2- 172.21.15.164:0/1977871757 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f38640add50 0x7f3850003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:13.205 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.197+0000 7f38637fe640 1 -- 172.21.15.164:0/1977871757 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3850004d10 msgr2=0x7f3850005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:13.205 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.197+0000 7f38637fe640 1 --2- 172.21.15.164:0/1977871757 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3850004d10 0x7f3850005190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:13.205 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.197+0000 7f38637fe640 1 -- 172.21.15.164:0/1977871757 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3850003530 con 0x7f3850003f00 2024-03-23T14:57:13.206 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.201+0000 7f38637fe640 1 --2- 172.21.15.164:0/1977871757 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3850003f00 0x7f38500047d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f384c009ae0 tx=0x7f384c01cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff7b13d154bbc741 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:13.206 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.201+0000 7f3862ffd640 1 -- 172.21.15.164:0/1977871757 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f384c0a5590 con 0x7f3850003f00 2024-03-23T14:57:13.206 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.201+0000 7f3862ffd640 1 -- 172.21.15.164:0/1977871757 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f384c0a56f0 con 0x7f3850003f00 2024-03-23T14:57:13.206 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.201+0000 7f3862ffd640 1 -- 172.21.15.164:0/1977871757 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f384c0a5aa0 con 0x7f3850003f00 2024-03-23T14:57:13.206 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.201+0000 7f386a8c9640 1 -- 172.21.15.164:0/1977871757 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3850003f00 msgr2=0x7f38500047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:13.206 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.201+0000 7f386a8c9640 1 --2- 172.21.15.164:0/1977871757 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3850003f00 0x7f38500047d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f384c009ae0 tx=0x7f384c01cab0 comp rx=0 tx=0).stop 2024-03-23T14:57:13.206 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.201+0000 7f3868e66640 1 -- 172.21.15.164:0/1977871757 reap_dead start 2024-03-23T14:57:13.206 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.201+0000 7f386a8c9640 1 -- 172.21.15.164:0/1977871757 shutdown_connections 2024-03-23T14:57:13.206 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.201+0000 7f386a8c9640 1 -- 172.21.15.164:0/1977871757 >> 172.21.15.164:0/1977871757 conn(0x7f38640b8ca0 msgr2=0x7f38640b90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:13.206 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.201+0000 7f386a8c9640 1 -- 172.21.15.164:0/1977871757 shutdown_connections 2024-03-23T14:57:13.207 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.201+0000 7f386a8c9640 1 -- 172.21.15.164:0/1977871757 wait complete. 2024-03-23T14:57:13.207 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.201+0000 7f386a8c9640 1 Processor -- start 2024-03-23T14:57:13.207 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.201+0000 7f386a8c9640 1 -- start start 2024-03-23T14:57:13.208 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.201+0000 7f386a8c9640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3850003f00 0x7f38641de3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:13.209 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.201+0000 7f386a8c9640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f386409bbf0 0x7f38640b1e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:13.209 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.201+0000 7f386a8c9640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f386413e890 0x7f386413ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:13.209 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.201+0000 7f386a8c9640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f386409c070 con 0x7f386409bbf0 2024-03-23T14:57:13.209 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.201+0000 7f386a8c9640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f3864142af0 con 0x7f386413e890 2024-03-23T14:57:13.209 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.201+0000 7f386a8c9640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f3864142d60 con 0x7f3850003f00 2024-03-23T14:57:13.209 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.201+0000 7f38637fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f386409bbf0 0x7f38640b1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:13.209 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.201+0000 7f38637fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f386409bbf0 0x7f38640b1e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:60140/0 (socket says 172.21.15.164:60140) 2024-03-23T14:57:13.209 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.201+0000 7f38637fe640 1 -- 172.21.15.164:0/1925826670 learned_addr learned my addr 172.21.15.164:0/1925826670 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:13.209 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.201+0000 7f3863fff640 1 --2- 172.21.15.164:0/1925826670 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3850003f00 0x7f38641de3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:13.209 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.201+0000 7f3868e66640 1 --2- 172.21.15.164:0/1925826670 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f386413e890 0x7f386413ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:13.209 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.201+0000 7f38637fe640 1 -- 172.21.15.164:0/1925826670 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3850003f00 msgr2=0x7f38641de3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:13.209 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.201+0000 7f38637fe640 1 --2- 172.21.15.164:0/1925826670 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3850003f00 0x7f38641de3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:13.209 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.201+0000 7f38637fe640 1 -- 172.21.15.164:0/1925826670 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f386413e890 msgr2=0x7f386413ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:13.209 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.201+0000 7f38637fe640 1 --2- 172.21.15.164:0/1925826670 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f386413e890 0x7f386413ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:13.209 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.201+0000 7f38637fe640 1 -- 172.21.15.164:0/1925826670 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38640b4e70 con 0x7f386409bbf0 2024-03-23T14:57:13.209 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.201+0000 7f38637fe640 1 --2- 172.21.15.164:0/1925826670 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f386409bbf0 0x7f38640b1e00 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f384c009ae0 tx=0x7f3850005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:13.210 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.201+0000 7f3860ff9640 1 -- 172.21.15.164:0/1925826670 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f384c0a5590 con 0x7f386409bbf0 2024-03-23T14:57:13.210 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.201+0000 7f3860ff9640 1 -- 172.21.15.164:0/1925826670 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f384c004020 con 0x7f386409bbf0 2024-03-23T14:57:13.210 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.201+0000 7f386a8c9640 1 -- 172.21.15.164:0/1925826670 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38641dd8d0 con 0x7f386409bbf0 2024-03-23T14:57:13.210 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.201+0000 7f3860ff9640 1 -- 172.21.15.164:0/1925826670 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f384c0043b0 con 0x7f386409bbf0 2024-03-23T14:57:13.210 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.201+0000 7f386a8c9640 1 -- 172.21.15.164:0/1925826670 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38641dde90 con 0x7f386409bbf0 2024-03-23T14:57:13.210 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.205+0000 7f386a8c9640 1 -- 172.21.15.164:0/1925826670 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38640690a0 con 0x7f386409bbf0 2024-03-23T14:57:13.210 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.205+0000 7f3860ff9640 1 -- 172.21.15.164:0/1925826670 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7f384c0b4040 con 0x7f386409bbf0 2024-03-23T14:57:13.211 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.205+0000 7f3860ff9640 1 --2- 172.21.15.164:0/1925826670 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f38440672a0 0x7f3844069760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:13.211 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.205+0000 7f3860ff9640 1 -- 172.21.15.164:0/1925826670 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f384c11ee80 con 0x7f386409bbf0 2024-03-23T14:57:13.211 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.205+0000 7f3863fff640 1 --2- 172.21.15.164:0/1925826670 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f38440672a0 0x7f3844069760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:13.211 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.205+0000 7f3863fff640 1 --2- 172.21.15.164:0/1925826670 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f38440672a0 0x7f3844069760 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f3854001810 tx=0x7f38540015c0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:13.214 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.209+0000 7f3860ff9640 1 -- 172.21.15.164:0/1925826670 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7f384c0ebb70 con 0x7f386409bbf0 2024-03-23T14:57:13.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.389+0000 7f386a8c9640 1 -- 172.21.15.164:0/1925826670 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3864141ad0 con 0x7f386409bbf0 2024-03-23T14:57:13.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.389+0000 7f3860ff9640 1 -- 172.21.15.164:0/1925826670 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+3533 (secure 0 0 0) 0x7f384c0b2de0 con 0x7f386409bbf0 2024-03-23T14:57:13.395 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-23T14:57:13.395 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/3352372536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":3352372536},{"type":"v1","addr":"172.21.15.164:6835","nonce":3352372536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-23T14:56:58.953326+0000","modified":"2024-03-23T14:57:00.999623+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4279":{"gid":4279,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.173:6833/3506402771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":3506402771},{"type":"v1","addr":"172.21.15.173:6833","nonce":3506402771}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4279,"qdb_cluster":[4279]},"id":1}]} 2024-03-23T14:57:13.396 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 6 2024-03-23T14:57:13.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.393+0000 7f386a8c9640 1 -- 172.21.15.164:0/1925826670 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f38440672a0 msgr2=0x7f3844069760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:13.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.393+0000 7f386a8c9640 1 --2- 172.21.15.164:0/1925826670 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f38440672a0 0x7f3844069760 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f3854001810 tx=0x7f38540015c0 comp rx=0 tx=0).stop 2024-03-23T14:57:13.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.393+0000 7f386a8c9640 1 -- 172.21.15.164:0/1925826670 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f386409bbf0 msgr2=0x7f38640b1e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:13.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.393+0000 7f386a8c9640 1 --2- 172.21.15.164:0/1925826670 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f386409bbf0 0x7f38640b1e00 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f384c009ae0 tx=0x7f3850005800 comp rx=0 tx=0).stop 2024-03-23T14:57:13.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.393+0000 7f386a8c9640 1 -- 172.21.15.164:0/1925826670 shutdown_connections 2024-03-23T14:57:13.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.393+0000 7f386a8c9640 1 --2- 172.21.15.164:0/1925826670 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f38440672a0 0x7f3844069760 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:13.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.393+0000 7f386a8c9640 1 --2- 172.21.15.164:0/1925826670 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f386413e890 0x7f386413ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:13.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.393+0000 7f386a8c9640 1 --2- 172.21.15.164:0/1925826670 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f386409bbf0 0x7f38640b1e00 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:13.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.393+0000 7f386a8c9640 1 --2- 172.21.15.164:0/1925826670 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3850003f00 0x7f38641de3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:13.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.393+0000 7f386a8c9640 1 -- 172.21.15.164:0/1925826670 >> 172.21.15.164:0/1925826670 conn(0x7f38640b8ca0 msgr2=0x7f38640b5cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:13.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.393+0000 7f386a8c9640 1 -- 172.21.15.164:0/1925826670 shutdown_connections 2024-03-23T14:57:13.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.393+0000 7f386a8c9640 1 -- 172.21.15.164:0/1925826670 wait complete. 2024-03-23T14:57:13.413 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T14:57:13.524 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.521+0000 7f1e8754a640 1 Processor -- start 2024-03-23T14:57:13.524 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.521+0000 7f1e8754a640 1 -- start start 2024-03-23T14:57:13.525 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.521+0000 7f1e8754a640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f1e80148cf0 0x7f1e80149180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:13.525 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.521+0000 7f1e8754a640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1e8014db70 0x7f1e8014ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:13.525 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.521+0000 7f1e8754a640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e800b51d0 con 0x7f1e800b0230 2024-03-23T14:57:13.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.521+0000 7f1e8754a640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f1e800b0610 con 0x7f1e80148cf0 2024-03-23T14:57:13.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.521+0000 7f1e8754a640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f1e800b1500 con 0x7f1e8014db70 2024-03-23T14:57:13.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.521+0000 7f1e84ae5640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f1e80148cf0 0x7f1e80149180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:13.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.521+0000 7f1e84ae5640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f1e80148cf0 0x7f1e80149180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:60156/0 (socket says 172.21.15.164:60156) 2024-03-23T14:57:13.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.521+0000 7f1e84ae5640 1 -- 172.21.15.164:0/42108964 learned_addr learned my addr 172.21.15.164:0/42108964 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:13.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.521+0000 7f1e85ae7640 1 --2- 172.21.15.164:0/42108964 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1e8014db70 0x7f1e8014ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:13.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.521+0000 7f1e84ae5640 1 -- 172.21.15.164:0/42108964 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1e8014db70 msgr2=0x7f1e8014ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:13.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.521+0000 7f1e84ae5640 1 --2- 172.21.15.164:0/42108964 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1e8014db70 0x7f1e8014ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:13.526 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.521+0000 7f1e84ae5640 1 -- 172.21.15.164:0/42108964 >> v1:172.21.15.173:6789/0 conn(0x7f1e800b0230 legacy=0x7f1e800b34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:13.527 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.521+0000 7f1e84ae5640 1 -- 172.21.15.164:0/42108964 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e8006d250 con 0x7f1e80148cf0 2024-03-23T14:57:13.529 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.525+0000 7f1e84ae5640 1 --2- 172.21.15.164:0/42108964 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1e80148cf0 0x7f1e80149180 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1e74009870 tx=0x7f1e7401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1cf170fbead52bd server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:13.530 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.525+0000 7f1e7ffff640 1 -- 172.21.15.164:0/42108964 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e740a6040 con 0x7f1e80148cf0 2024-03-23T14:57:13.530 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.525+0000 7f1e7ffff640 1 -- 172.21.15.164:0/42108964 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e740aa600 con 0x7f1e80148cf0 2024-03-23T14:57:13.530 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.525+0000 7f1e7ffff640 1 -- 172.21.15.164:0/42108964 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e740aa990 con 0x7f1e80148cf0 2024-03-23T14:57:13.530 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.525+0000 7f1e8754a640 1 -- 172.21.15.164:0/42108964 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1e80148cf0 msgr2=0x7f1e80149180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:13.530 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.525+0000 7f1e8754a640 1 --2- 172.21.15.164:0/42108964 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1e80148cf0 0x7f1e80149180 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1e74009870 tx=0x7f1e7401c4a0 comp rx=0 tx=0).stop 2024-03-23T14:57:13.530 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.525+0000 7f1e8754a640 1 -- 172.21.15.164:0/42108964 shutdown_connections 2024-03-23T14:57:13.530 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.525+0000 7f1e8754a640 1 --2- 172.21.15.164:0/42108964 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1e8014db70 0x7f1e8014ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:13.530 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.525+0000 7f1e8754a640 1 --2- 172.21.15.164:0/42108964 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1e80148cf0 0x7f1e80149180 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:13.530 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.525+0000 7f1e8754a640 1 -- 172.21.15.164:0/42108964 >> 172.21.15.164:0/42108964 conn(0x7f1e800b8ca0 msgr2=0x7f1e800b90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:13.530 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.525+0000 7f1e8754a640 1 -- 172.21.15.164:0/42108964 shutdown_connections 2024-03-23T14:57:13.530 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.525+0000 7f1e8754a640 1 -- 172.21.15.164:0/42108964 wait complete. 2024-03-23T14:57:13.530 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.525+0000 7f1e8754a640 1 Processor -- start 2024-03-23T14:57:13.530 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.525+0000 7f1e8754a640 1 -- start start 2024-03-23T14:57:13.533 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.525+0000 7f1e8754a640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1e800b0230 0x7f1e8013c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:13.533 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.525+0000 7f1e8754a640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1e80148cf0 0x7f1e80144d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:13.533 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.525+0000 7f1e8754a640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1e8014db70 0x7f1e8013a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:13.533 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.525+0000 7f1e8754a640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1e8013a950 con 0x7f1e800b0230 2024-03-23T14:57:13.534 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.525+0000 7f1e8754a640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f1e80139ca0 con 0x7f1e8014db70 2024-03-23T14:57:13.534 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.525+0000 7f1e8754a640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f1e80139f80 con 0x7f1e80148cf0 2024-03-23T14:57:13.534 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.525+0000 7f1e852e6640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1e800b0230 0x7f1e8013c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:13.534 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.525+0000 7f1e852e6640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1e800b0230 0x7f1e8013c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:60164/0 (socket says 172.21.15.164:60164) 2024-03-23T14:57:13.534 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.525+0000 7f1e852e6640 1 -- 172.21.15.164:0/3719537346 learned_addr learned my addr 172.21.15.164:0/3719537346 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:13.534 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.525+0000 7f1e84ae5640 1 --2- 172.21.15.164:0/3719537346 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1e80148cf0 0x7f1e80144d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:13.534 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.525+0000 7f1e85ae7640 1 --2- 172.21.15.164:0/3719537346 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1e8014db70 0x7f1e8013a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:13.534 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.525+0000 7f1e852e6640 1 -- 172.21.15.164:0/3719537346 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1e80148cf0 msgr2=0x7f1e80144d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:13.534 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.525+0000 7f1e852e6640 1 --2- 172.21.15.164:0/3719537346 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1e80148cf0 0x7f1e80144d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:13.534 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.525+0000 7f1e852e6640 1 -- 172.21.15.164:0/3719537346 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1e8014db70 msgr2=0x7f1e8013a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:13.534 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.525+0000 7f1e852e6640 1 --2- 172.21.15.164:0/3719537346 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1e8014db70 0x7f1e8013a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:13.534 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.525+0000 7f1e852e6640 1 -- 172.21.15.164:0/3719537346 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e740094d0 con 0x7f1e800b0230 2024-03-23T14:57:13.534 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.525+0000 7f1e85ae7640 1 --2- 172.21.15.164:0/3719537346 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1e8014db70 0x7f1e8013a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T14:57:13.534 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.525+0000 7f1e84ae5640 1 --2- 172.21.15.164:0/3719537346 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1e80148cf0 0x7f1e80144d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T14:57:13.534 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.525+0000 7f1e852e6640 1 --2- 172.21.15.164:0/3719537346 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1e800b0230 0x7f1e8013c9d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1e7000c790 tx=0x7f1e7000cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:13.534 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.525+0000 7f1e7dffb640 1 -- 172.21.15.164:0/3719537346 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e7000cea0 con 0x7f1e800b0230 2024-03-23T14:57:13.535 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.525+0000 7f1e7dffb640 1 -- 172.21.15.164:0/3719537346 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e70006d60 con 0x7f1e800b0230 2024-03-23T14:57:13.535 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.525+0000 7f1e8754a640 1 -- 172.21.15.164:0/3719537346 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e801455e0 con 0x7f1e800b0230 2024-03-23T14:57:13.535 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.525+0000 7f1e7dffb640 1 -- 172.21.15.164:0/3719537346 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e70003bc0 con 0x7f1e800b0230 2024-03-23T14:57:13.535 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.525+0000 7f1e8754a640 1 -- 172.21.15.164:0/3719537346 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e8013c1d0 con 0x7f1e800b0230 2024-03-23T14:57:13.535 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.525+0000 7f1e8754a640 1 -- 172.21.15.164:0/3719537346 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e800690a0 con 0x7f1e800b0230 2024-03-23T14:57:13.535 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.529+0000 7f1e7dffb640 1 -- 172.21.15.164:0/3719537346 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7f1e70003d20 con 0x7f1e800b0230 2024-03-23T14:57:13.535 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.529+0000 7f1e7dffb640 1 --2- 172.21.15.164:0/3719537346 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f1e6c0672a0 0x7f1e6c069760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:13.535 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.529+0000 7f1e7dffb640 1 -- 172.21.15.164:0/3719537346 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f1e70084bf0 con 0x7f1e800b0230 2024-03-23T14:57:13.535 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.529+0000 7f1e84ae5640 1 --2- 172.21.15.164:0/3719537346 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f1e6c0672a0 0x7f1e6c069760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:13.535 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.529+0000 7f1e84ae5640 1 --2- 172.21.15.164:0/3719537346 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f1e6c0672a0 0x7f1e6c069760 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1e740ae000 tx=0x7f1e74008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:13.537 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.533+0000 7f1e7dffb640 1 -- 172.21.15.164:0/3719537346 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7f1e70055360 con 0x7f1e800b0230 2024-03-23T14:57:13.682 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.677+0000 7f1e8754a640 1 -- 172.21.15.164:0/3719537346 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1e800b2fa0 con 0x7f1e800b0230 2024-03-23T14:57:13.684 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.681+0000 7f1e7dffb640 1 -- 172.21.15.164:0/3719537346 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16629 (secure 0 0 0) 0x7f1e70054ef0 con 0x7f1e800b0230 2024-03-23T14:57:13.684 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-23T14:57:13.684 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":20,"fsid":"45c7911b-b69d-4e68-907b-fbfeb780aa3b","created":"2024-03-23T14:56:34.367633+0000","modified":"2024-03-23T14:57:00.611265+0000","last_up_change":"2024-03-23T14:56:46.297403+0000","last_in_change":"2024-03-23T14:56:36.629884+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T14:56:47.186157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-23T14:56:56.994057+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-23T14:56:57.953862+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"80abdc62-bd0d-4e60-a554-7ed6fd67cf54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":984716690},{"type":"v1","addr":"172.21.15.164:6801","nonce":984716690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":984716690},{"type":"v1","addr":"172.21.15.164:6803","nonce":984716690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":984716690},{"type":"v1","addr":"172.21.15.164:6807","nonce":984716690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":984716690},{"type":"v1","addr":"172.21.15.164:6805","nonce":984716690}]},"public_addr":"172.21.15.164:6801/984716690","cluster_addr":"172.21.15.164:6803/984716690","heartbeat_back_addr":"172.21.15.164:6807/984716690","heartbeat_front_addr":"172.21.15.164:6805/984716690","state":["exists","up"]},{"osd":1,"uuid":"f9001b06-be4d-4f42-a0e6-bcef2afa0d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":1349657239},{"type":"v1","addr":"172.21.15.164:6817","nonce":1349657239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":1349657239},{"type":"v1","addr":"172.21.15.164:6819","nonce":1349657239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":1349657239},{"type":"v1","addr":"172.21.15.164:6823","nonce":1349657239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":1349657239},{"type":"v1","addr":"172.21.15.164:6821","nonce":1349657239}]},"public_addr":"172.21.15.164:6817/1349657239","cluster_addr":"172.21.15.164:6819/1349657239","heartbeat_back_addr":"172.21.15.164:6823/1349657239","heartbeat_front_addr":"172.21.15.164:6821/1349657239","state":["exists","up"]},{"osd":2,"uuid":"fa0aeb9f-8b54-4831-a188-3e361c7980d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":887861252},{"type":"v1","addr":"172.21.15.164:6809","nonce":887861252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":887861252},{"type":"v1","addr":"172.21.15.164:6811","nonce":887861252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":887861252},{"type":"v1","addr":"172.21.15.164:6815","nonce":887861252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":887861252},{"type":"v1","addr":"172.21.15.164:6813","nonce":887861252}]},"public_addr":"172.21.15.164:6809/887861252","cluster_addr":"172.21.15.164:6811/887861252","heartbeat_back_addr":"172.21.15.164:6815/887861252","heartbeat_front_addr":"172.21.15.164:6813/887861252","state":["exists","up"]},{"osd":3,"uuid":"7e18c957-3091-46d5-8e17-8f3354c1726c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":812540271},{"type":"v1","addr":"172.21.15.164:6825","nonce":812540271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":812540271},{"type":"v1","addr":"172.21.15.164:6827","nonce":812540271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":812540271},{"type":"v1","addr":"172.21.15.164:6831","nonce":812540271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":812540271},{"type":"v1","addr":"172.21.15.164:6829","nonce":812540271}]},"public_addr":"172.21.15.164:6825/812540271","cluster_addr":"172.21.15.164:6827/812540271","heartbeat_back_addr":"172.21.15.164:6831/812540271","heartbeat_front_addr":"172.21.15.164:6829/812540271","state":["exists","up"]},{"osd":4,"uuid":"8e83f315-42bf-4471-9ce8-d81948da0f9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":2656857421},{"type":"v1","addr":"172.21.15.173:6825","nonce":2656857421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2656857421},{"type":"v1","addr":"172.21.15.173:6827","nonce":2656857421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2656857421},{"type":"v1","addr":"172.21.15.173:6831","nonce":2656857421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":2656857421},{"type":"v1","addr":"172.21.15.173:6829","nonce":2656857421}]},"public_addr":"172.21.15.173:6825/2656857421","cluster_addr":"172.21.15.173:6827/2656857421","heartbeat_back_addr":"172.21.15.173:6831/2656857421","heartbeat_front_addr":"172.21.15.173:6829/2656857421","state":["exists","up"]},{"osd":5,"uuid":"a230c872-71f9-40ab-8999-defd0162d4ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":3676483185},{"type":"v1","addr":"172.21.15.173:6801","nonce":3676483185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":3676483185},{"type":"v1","addr":"172.21.15.173:6803","nonce":3676483185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":3676483185},{"type":"v1","addr":"172.21.15.173:6807","nonce":3676483185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":3676483185},{"type":"v1","addr":"172.21.15.173:6805","nonce":3676483185}]},"public_addr":"172.21.15.173:6801/3676483185","cluster_addr":"172.21.15.173:6803/3676483185","heartbeat_back_addr":"172.21.15.173:6807/3676483185","heartbeat_front_addr":"172.21.15.173:6805/3676483185","state":["exists","up"]},{"osd":6,"uuid":"6193bfd7-8432-4f7f-a117-a3fcee21af99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":3631684087},{"type":"v1","addr":"172.21.15.173:6809","nonce":3631684087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":3631684087},{"type":"v1","addr":"172.21.15.173:6811","nonce":3631684087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":3631684087},{"type":"v1","addr":"172.21.15.173:6815","nonce":3631684087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":3631684087},{"type":"v1","addr":"172.21.15.173:6813","nonce":3631684087}]},"public_addr":"172.21.15.173:6809/3631684087","cluster_addr":"172.21.15.173:6811/3631684087","heartbeat_back_addr":"172.21.15.173:6815/3631684087","heartbeat_front_addr":"172.21.15.173:6813/3631684087","state":["exists","up"]},{"osd":7,"uuid":"b0f1b80a-afc4-4538-83b9-f081d2dcb8cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":3261114948},{"type":"v1","addr":"172.21.15.173:6817","nonce":3261114948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":3261114948},{"type":"v1","addr":"172.21.15.173:6819","nonce":3261114948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":3261114948},{"type":"v1","addr":"172.21.15.173:6823","nonce":3261114948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":3261114948},{"type":"v1","addr":"172.21.15.173:6821","nonce":3261114948}]},"public_addr":"172.21.15.173:6817/3261114948","cluster_addr":"172.21.15.173:6819/3261114948","heartbeat_back_addr":"172.21.15.173:6823/3261114948","heartbeat_front_addr":"172.21.15.173:6821/3261114948","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.188390+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.159932+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.191101+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.181776+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.224365+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.177515+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.153631+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.139972+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T14:57:13.686 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.681+0000 7f1e8754a640 1 -- 172.21.15.164:0/3719537346 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f1e6c0672a0 msgr2=0x7f1e6c069760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:13.686 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.681+0000 7f1e8754a640 1 --2- 172.21.15.164:0/3719537346 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f1e6c0672a0 0x7f1e6c069760 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1e740ae000 tx=0x7f1e74008040 comp rx=0 tx=0).stop 2024-03-23T14:57:13.686 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.681+0000 7f1e8754a640 1 -- 172.21.15.164:0/3719537346 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1e800b0230 msgr2=0x7f1e8013c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:13.687 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.681+0000 7f1e8754a640 1 --2- 172.21.15.164:0/3719537346 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1e800b0230 0x7f1e8013c9d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1e7000c790 tx=0x7f1e7000cc50 comp rx=0 tx=0).stop 2024-03-23T14:57:13.687 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.681+0000 7f1e8754a640 1 -- 172.21.15.164:0/3719537346 shutdown_connections 2024-03-23T14:57:13.687 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.681+0000 7f1e8754a640 1 --2- 172.21.15.164:0/3719537346 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f1e6c0672a0 0x7f1e6c069760 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:13.687 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.681+0000 7f1e8754a640 1 --2- 172.21.15.164:0/3719537346 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1e8014db70 0x7f1e8013a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:13.687 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.681+0000 7f1e8754a640 1 --2- 172.21.15.164:0/3719537346 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1e80148cf0 0x7f1e80144d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:13.687 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.681+0000 7f1e8754a640 1 --2- 172.21.15.164:0/3719537346 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1e800b0230 0x7f1e8013c9d0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:13.687 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.681+0000 7f1e8754a640 1 -- 172.21.15.164:0/3719537346 >> 172.21.15.164:0/3719537346 conn(0x7f1e800b8ca0 msgr2=0x7f1e800b2350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:13.687 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.681+0000 7f1e8754a640 1 -- 172.21.15.164:0/3719537346 shutdown_connections 2024-03-23T14:57:13.687 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.681+0000 7f1e8754a640 1 -- 172.21.15.164:0/3719537346 wait complete. 2024-03-23T14:57:13.702 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T14:57:13.820 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.817+0000 7f5d8ef39640 1 Processor -- start 2024-03-23T14:57:13.820 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.817+0000 7f5d8ef39640 1 -- start start 2024-03-23T14:57:13.822 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.817+0000 7f5d8ef39640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f5d880b0230 0x7f5d880b34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:13.822 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.817+0000 7f5d8ef39640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5d88148cf0 0x7f5d88149180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:13.822 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.817+0000 7f5d8ef39640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f5d880b1c70 0x7f5d880b20a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:13.822 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.817+0000 7f5d8ef39640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f5d880b51d0 con 0x7f5d880b0230 2024-03-23T14:57:13.822 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.817+0000 7f5d8ef39640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 -- 0x7f5d880b0610 con 0x7f5d880b1c70 2024-03-23T14:57:13.822 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.817+0000 7f5d8ef39640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f5d880b1500 con 0x7f5d88148cf0 2024-03-23T14:57:13.822 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.817+0000 7f5d8ccd5640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f5d880b0230 0x7f5d880b34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:13.822 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.817+0000 7f5d8ccd5640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f5d880b0230 0x7f5d880b34d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:60174/0 (socket says 172.21.15.164:60174) 2024-03-23T14:57:13.822 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.817+0000 7f5d8ccd5640 1 -- 172.21.15.164:0/275065812 learned_addr learned my addr 172.21.15.164:0/275065812 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:13.822 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.817+0000 7f5d87fff640 1 --2- 172.21.15.164:0/275065812 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5d88148cf0 0x7f5d88149180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:13.822 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.817+0000 7f5d8d4d6640 1 --2- 172.21.15.164:0/275065812 >> v2:172.21.15.173:3300/0 conn(0x7f5d880b1c70 0x7f5d880b20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:13.823 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.817+0000 7f5d8ccd5640 1 -- 172.21.15.164:0/275065812 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5d88148cf0 msgr2=0x7f5d88149180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:13.823 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.817+0000 7f5d8ccd5640 1 --2- 172.21.15.164:0/275065812 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5d88148cf0 0x7f5d88149180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:13.823 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.817+0000 7f5d8ccd5640 1 -- 172.21.15.164:0/275065812 >> v2:172.21.15.173:3300/0 conn(0x7f5d880b1c70 msgr2=0x7f5d880b20a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:13.823 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.817+0000 7f5d8ccd5640 1 --2- 172.21.15.164:0/275065812 >> v2:172.21.15.173:3300/0 conn(0x7f5d880b1c70 0x7f5d880b20a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:13.823 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.817+0000 7f5d8ccd5640 1 -- 172.21.15.164:0/275065812 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d8806d250 con 0x7f5d880b0230 2024-03-23T14:57:13.823 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.817+0000 7f5d8ccd5640 1 --2- 172.21.15.164:0/275065812 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5d880b0230 0x7f5d880b34d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f5d880b6980 tx=0x7f5d7801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca078bd4c2bf88a5 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:13.824 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.821+0000 7f5d877fe640 1 -- 172.21.15.164:0/275065812 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d880b2df0 con 0x7f5d880b0230 2024-03-23T14:57:13.824 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.821+0000 7f5d877fe640 1 -- 172.21.15.164:0/275065812 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d780ab620 con 0x7f5d880b0230 2024-03-23T14:57:13.824 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.821+0000 7f5d877fe640 1 -- 172.21.15.164:0/275065812 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d780ab9b0 con 0x7f5d880b0230 2024-03-23T14:57:13.825 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.821+0000 7f5d8ef39640 1 -- 172.21.15.164:0/275065812 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5d880b0230 msgr2=0x7f5d880b34d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:13.825 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.821+0000 7f5d8ef39640 1 --2- 172.21.15.164:0/275065812 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5d880b0230 0x7f5d880b34d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f5d880b6980 tx=0x7f5d7801c460 comp rx=0 tx=0).stop 2024-03-23T14:57:13.825 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.821+0000 7f5d8ef39640 1 -- 172.21.15.164:0/275065812 shutdown_connections 2024-03-23T14:57:13.825 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.821+0000 7f5d8ef39640 1 --2- 172.21.15.164:0/275065812 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5d88148cf0 0x7f5d88149180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:13.825 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.821+0000 7f5d8ef39640 1 --2- 172.21.15.164:0/275065812 >> v2:172.21.15.173:3300/0 conn(0x7f5d880b1c70 0x7f5d880b20a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:13.825 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.821+0000 7f5d8ef39640 1 --2- 172.21.15.164:0/275065812 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5d880b0230 0x7f5d880b34d0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:13.825 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.821+0000 7f5d8ef39640 1 -- 172.21.15.164:0/275065812 >> 172.21.15.164:0/275065812 conn(0x7f5d880b8ca0 msgr2=0x7f5d880b90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:13.825 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.821+0000 7f5d8ef39640 1 -- 172.21.15.164:0/275065812 shutdown_connections 2024-03-23T14:57:13.825 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.821+0000 7f5d8ef39640 1 -- 172.21.15.164:0/275065812 wait complete. 2024-03-23T14:57:13.826 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.821+0000 7f5d8ef39640 1 Processor -- start 2024-03-23T14:57:13.826 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.821+0000 7f5d8ef39640 1 -- start start 2024-03-23T14:57:13.828 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.821+0000 7f5d8ef39640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5d880b0230 0x7f5d8813a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:13.828 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.821+0000 7f5d8ef39640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5d880b1c70 0x7f5d8813c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:13.828 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.821+0000 7f5d8ef39640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5d88148cf0 0x7f5d881dd880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:13.828 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.821+0000 7f5d8ef39640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5d8813ab60 con 0x7f5d880b1c70 2024-03-23T14:57:13.828 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.821+0000 7f5d8ef39640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f5d8813a150 con 0x7f5d880b0230 2024-03-23T14:57:13.828 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.821+0000 7f5d8ef39640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f5d88139b90 con 0x7f5d88148cf0 2024-03-23T14:57:13.828 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.821+0000 7f5d87fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5d880b1c70 0x7f5d8813c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:13.828 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.821+0000 7f5d87fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5d880b1c70 0x7f5d8813c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:60178/0 (socket says 172.21.15.164:60178) 2024-03-23T14:57:13.828 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.821+0000 7f5d87fff640 1 -- 172.21.15.164:0/3369770526 learned_addr learned my addr 172.21.15.164:0/3369770526 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:13.828 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.821+0000 7f5d8d4d6640 1 --2- 172.21.15.164:0/3369770526 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5d88148cf0 0x7f5d881dd880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:13.829 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.821+0000 7f5d8ccd5640 1 --2- 172.21.15.164:0/3369770526 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5d880b0230 0x7f5d8813a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:13.829 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.821+0000 7f5d87fff640 1 -- 172.21.15.164:0/3369770526 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5d88148cf0 msgr2=0x7f5d881dd880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:13.829 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.821+0000 7f5d87fff640 1 --2- 172.21.15.164:0/3369770526 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5d88148cf0 0x7f5d881dd880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:13.829 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.821+0000 7f5d87fff640 1 -- 172.21.15.164:0/3369770526 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5d880b0230 msgr2=0x7f5d8813a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:13.829 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.821+0000 7f5d87fff640 1 --2- 172.21.15.164:0/3369770526 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5d880b0230 0x7f5d8813a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:13.829 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.821+0000 7f5d87fff640 1 -- 172.21.15.164:0/3369770526 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d780094d0 con 0x7f5d880b1c70 2024-03-23T14:57:13.829 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.821+0000 7f5d87fff640 1 --2- 172.21.15.164:0/3369770526 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5d880b1c70 0x7f5d8813c320 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c00e870 tx=0x7f5d7c00ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:13.829 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.821+0000 7f5d857fa640 1 -- 172.21.15.164:0/3369770526 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d7c009510 con 0x7f5d880b1c70 2024-03-23T14:57:13.829 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.821+0000 7f5d857fa640 1 -- 172.21.15.164:0/3369770526 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d7c00eec0 con 0x7f5d880b1c70 2024-03-23T14:57:13.829 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.825+0000 7f5d857fa640 1 -- 172.21.15.164:0/3369770526 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d7c00cb40 con 0x7f5d880b1c70 2024-03-23T14:57:13.829 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.825+0000 7f5d8ef39640 1 -- 172.21.15.164:0/3369770526 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d88139e70 con 0x7f5d880b1c70 2024-03-23T14:57:13.829 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.825+0000 7f5d8ef39640 1 -- 172.21.15.164:0/3369770526 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d881dd230 con 0x7f5d880b1c70 2024-03-23T14:57:13.830 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.825+0000 7f5d8ef39640 1 -- 172.21.15.164:0/3369770526 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d880690a0 con 0x7f5d880b1c70 2024-03-23T14:57:13.830 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.825+0000 7f5d857fa640 1 -- 172.21.15.164:0/3369770526 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7f5d7c00cca0 con 0x7f5d880b1c70 2024-03-23T14:57:13.830 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.825+0000 7f5d857fa640 1 --2- 172.21.15.164:0/3369770526 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f5d740672a0 0x7f5d74069760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:13.830 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.825+0000 7f5d857fa640 1 -- 172.21.15.164:0/3369770526 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f5d7c0850d0 con 0x7f5d880b1c70 2024-03-23T14:57:13.830 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.825+0000 7f5d8ccd5640 1 --2- 172.21.15.164:0/3369770526 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f5d740672a0 0x7f5d74069760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:13.830 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.825+0000 7f5d8ccd5640 1 --2- 172.21.15.164:0/3369770526 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f5d740672a0 0x7f5d74069760 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f5d780af000 tx=0x7f5d780a7000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:13.834 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:13.829+0000 7f5d857fa640 1 -- 172.21.15.164:0/3369770526 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7f5d7c051c70 con 0x7f5d880b1c70 2024-03-23T14:57:14.015 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.009+0000 7f5d8ef39640 1 -- 172.21.15.164:0/3369770526 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5d8809e300 con 0x7f5d880b1c70 2024-03-23T14:57:14.016 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.013+0000 7f5d857fa640 1 -- 172.21.15.164:0/3369770526 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+3533 (secure 0 0 0) 0x7f5d7c017de0 con 0x7f5d880b1c70 2024-03-23T14:57:14.016 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-23T14:57:14.016 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/3352372536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":3352372536},{"type":"v1","addr":"172.21.15.164:6835","nonce":3352372536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-23T14:56:58.953326+0000","modified":"2024-03-23T14:57:00.999623+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4279":{"gid":4279,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.173:6833/3506402771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":3506402771},{"type":"v1","addr":"172.21.15.173:6833","nonce":3506402771}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4279,"qdb_cluster":[4279]},"id":1}]} 2024-03-23T14:57:14.016 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 6 2024-03-23T14:57:14.018 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.013+0000 7f5d8ef39640 1 -- 172.21.15.164:0/3369770526 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f5d740672a0 msgr2=0x7f5d74069760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:14.018 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.013+0000 7f5d8ef39640 1 --2- 172.21.15.164:0/3369770526 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f5d740672a0 0x7f5d74069760 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f5d780af000 tx=0x7f5d780a7000 comp rx=0 tx=0).stop 2024-03-23T14:57:14.018 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.013+0000 7f5d8ef39640 1 -- 172.21.15.164:0/3369770526 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5d880b1c70 msgr2=0x7f5d8813c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:14.018 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.013+0000 7f5d8ef39640 1 --2- 172.21.15.164:0/3369770526 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5d880b1c70 0x7f5d8813c320 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c00e870 tx=0x7f5d7c00ec70 comp rx=0 tx=0).stop 2024-03-23T14:57:14.018 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.013+0000 7f5d8ef39640 1 -- 172.21.15.164:0/3369770526 shutdown_connections 2024-03-23T14:57:14.018 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.013+0000 7f5d8ef39640 1 --2- 172.21.15.164:0/3369770526 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5d88148cf0 0x7f5d881dd880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:14.018 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.013+0000 7f5d8ef39640 1 --2- 172.21.15.164:0/3369770526 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5d880b1c70 0x7f5d8813c320 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:14.018 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.013+0000 7f5d8ef39640 1 --2- 172.21.15.164:0/3369770526 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f5d740672a0 0x7f5d74069760 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:14.018 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.013+0000 7f5d8ef39640 1 --2- 172.21.15.164:0/3369770526 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5d880b0230 0x7f5d8813a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:14.018 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.013+0000 7f5d8ef39640 1 -- 172.21.15.164:0/3369770526 >> 172.21.15.164:0/3369770526 conn(0x7f5d880b8ca0 msgr2=0x7f5d880b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:14.018 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.013+0000 7f5d8ef39640 1 -- 172.21.15.164:0/3369770526 shutdown_connections 2024-03-23T14:57:14.018 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.013+0000 7f5d8ef39640 1 -- 172.21.15.164:0/3369770526 wait complete. 2024-03-23T14:57:14.235 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 6, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-23T14:56:58.953326+0000', 'modified': '2024-03-23T14:57:00.999623+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4279}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4279': {'gid': 4279, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.173:6833/3506402771', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6832', 'nonce': 3506402771}, {'type': 'v1', 'addr': '172.21.15.173:6833', 'nonce': 3506402771}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4279, 'qdb_cluster': [4279]} 2024-03-23T14:57:14.235 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-23T14:57:14.235 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-23T14:57:14.357 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.353+0000 7f07e0f9b640 1 Processor -- start 2024-03-23T14:57:14.357 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.353+0000 7f07e0f9b640 1 -- start start 2024-03-23T14:57:14.358 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.353+0000 7f07e0f9b640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f07dc0b0230 0x7f07dc0b34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:14.358 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.353+0000 7f07e0f9b640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f07dc148cf0 0x7f07dc149150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:14.358 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.353+0000 7f07e0f9b640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07dc0b51d0 con 0x7f07dc0b1c70 2024-03-23T14:57:14.358 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.353+0000 7f07e0f9b640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 -- 0x7f07dc0b0610 con 0x7f07dc148cf0 2024-03-23T14:57:14.358 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.353+0000 7f07e0f9b640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f07dc0b1500 con 0x7f07dc0b0230 2024-03-23T14:57:14.359 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.353+0000 7f07da59c640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f07dc0b0230 0x7f07dc0b34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:14.359 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.353+0000 7f07da59c640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f07dc0b0230 0x7f07dc0b34d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.164:60592/0 (socket says 172.21.15.164:60592) 2024-03-23T14:57:14.359 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.353+0000 7f07da59c640 1 -- 172.21.15.164:0/3731618096 learned_addr learned my addr 172.21.15.164:0/3731618096 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:14.359 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.353+0000 7f07d9d9b640 1 --2- 172.21.15.164:0/3731618096 >> v2:172.21.15.173:3300/0 conn(0x7f07dc148cf0 0x7f07dc149150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:14.360 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.353+0000 7f07da59c640 1 -- 172.21.15.164:0/3731618096 >> v2:172.21.15.173:3300/0 conn(0x7f07dc148cf0 msgr2=0x7f07dc149150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:14.360 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.353+0000 7f07da59c640 1 --2- 172.21.15.164:0/3731618096 >> v2:172.21.15.173:3300/0 conn(0x7f07dc148cf0 0x7f07dc149150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:14.360 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.353+0000 7f07da59c640 1 -- 172.21.15.164:0/3731618096 >> v1:172.21.15.173:6789/0 conn(0x7f07dc0b1c70 legacy=0x7f07dc0b2070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:14.360 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.353+0000 7f07da59c640 1 -- 172.21.15.164:0/3731618096 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07dc06d250 con 0x7f07dc0b0230 2024-03-23T14:57:14.361 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.357+0000 7f07da59c640 1 --2- 172.21.15.164:0/3731618096 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f07dc0b0230 0x7f07dc0b34d0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f07d0009870 tx=0x7f07d001c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e4de5b29f0f5f7b server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:14.362 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.357+0000 7f07d959a640 1 -- 172.21.15.164:0/3731618096 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07d00a6040 con 0x7f07dc0b0230 2024-03-23T14:57:14.362 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.357+0000 7f07d959a640 1 -- 172.21.15.164:0/3731618096 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07d00aa600 con 0x7f07dc0b0230 2024-03-23T14:57:14.362 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.357+0000 7f07d959a640 1 -- 172.21.15.164:0/3731618096 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07d00aa990 con 0x7f07dc0b0230 2024-03-23T14:57:14.363 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.357+0000 7f07e0f9b640 1 -- 172.21.15.164:0/3731618096 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f07dc0b0230 msgr2=0x7f07dc0b34d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:14.363 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.357+0000 7f07e0f9b640 1 --2- 172.21.15.164:0/3731618096 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f07dc0b0230 0x7f07dc0b34d0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f07d0009870 tx=0x7f07d001c4a0 comp rx=0 tx=0).stop 2024-03-23T14:57:14.363 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.357+0000 7f07e0f9b640 1 -- 172.21.15.164:0/3731618096 shutdown_connections 2024-03-23T14:57:14.363 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.357+0000 7f07e0f9b640 1 --2- 172.21.15.164:0/3731618096 >> v2:172.21.15.173:3300/0 conn(0x7f07dc148cf0 0x7f07dc149150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:14.363 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.357+0000 7f07e0f9b640 1 --2- 172.21.15.164:0/3731618096 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f07dc0b0230 0x7f07dc0b34d0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:14.363 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.357+0000 7f07e0f9b640 1 -- 172.21.15.164:0/3731618096 >> 172.21.15.164:0/3731618096 conn(0x7f07dc0b8ca0 msgr2=0x7f07dc0b90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:14.363 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.357+0000 7f07e0f9b640 1 -- 172.21.15.164:0/3731618096 shutdown_connections 2024-03-23T14:57:14.363 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.357+0000 7f07e0f9b640 1 -- 172.21.15.164:0/3731618096 wait complete. 2024-03-23T14:57:14.364 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.357+0000 7f07e0f9b640 1 Processor -- start 2024-03-23T14:57:14.364 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.357+0000 7f07e0f9b640 1 -- start start 2024-03-23T14:57:14.365 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.361+0000 7f07e0f9b640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f07dc0b0230 0x7f07dc09c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:14.365 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.361+0000 7f07e0f9b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f07dc0b1c70 0x7f07dc06f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:14.365 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.361+0000 7f07e0f9b640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f07dc148cf0 0x7f07dc144b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:14.366 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.361+0000 7f07e0f9b640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f07dc06fdd0 con 0x7f07dc0b1c70 2024-03-23T14:57:14.366 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.361+0000 7f07e0f9b640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f07dc09d440 con 0x7f07dc148cf0 2024-03-23T14:57:14.366 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.361+0000 7f07e0f9b640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f07dc146330 con 0x7f07dc0b0230 2024-03-23T14:57:14.366 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.361+0000 7f07d9d9b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f07dc0b1c70 0x7f07dc06f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:14.366 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.361+0000 7f07d9d9b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f07dc0b1c70 0x7f07dc06f890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:60192/0 (socket says 172.21.15.164:60192) 2024-03-23T14:57:14.366 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.361+0000 7f07d9d9b640 1 -- 172.21.15.164:0/3108824419 learned_addr learned my addr 172.21.15.164:0/3108824419 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:14.366 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.361+0000 7f07da59c640 1 --2- 172.21.15.164:0/3108824419 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f07dc0b0230 0x7f07dc09c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:14.366 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.361+0000 7f07dad9d640 1 --2- 172.21.15.164:0/3108824419 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f07dc148cf0 0x7f07dc144b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:14.366 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.361+0000 7f07d9d9b640 1 -- 172.21.15.164:0/3108824419 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f07dc0b0230 msgr2=0x7f07dc09c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:14.366 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.361+0000 7f07d9d9b640 1 --2- 172.21.15.164:0/3108824419 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f07dc0b0230 0x7f07dc09c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:14.366 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.361+0000 7f07d9d9b640 1 -- 172.21.15.164:0/3108824419 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f07dc148cf0 msgr2=0x7f07dc144b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:14.366 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.361+0000 7f07d9d9b640 1 --2- 172.21.15.164:0/3108824419 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f07dc148cf0 0x7f07dc144b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:14.366 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.361+0000 7f07d9d9b640 1 -- 172.21.15.164:0/3108824419 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07d00094d0 con 0x7f07dc0b1c70 2024-03-23T14:57:14.366 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.361+0000 7f07d9d9b640 1 --2- 172.21.15.164:0/3108824419 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f07dc0b1c70 0x7f07dc06f890 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f07cc00e870 tx=0x7f07cc00ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:14.366 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.361+0000 7f07caffd640 1 -- 172.21.15.164:0/3108824419 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07cc009510 con 0x7f07dc0b1c70 2024-03-23T14:57:14.367 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.361+0000 7f07caffd640 1 -- 172.21.15.164:0/3108824419 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07cc00eec0 con 0x7f07dc0b1c70 2024-03-23T14:57:14.367 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.361+0000 7f07caffd640 1 -- 172.21.15.164:0/3108824419 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07cc00cb40 con 0x7f07dc0b1c70 2024-03-23T14:57:14.367 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.361+0000 7f07e0f9b640 1 -- 172.21.15.164:0/3108824419 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07dc070050 con 0x7f07dc0b1c70 2024-03-23T14:57:14.367 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.361+0000 7f07e0f9b640 1 -- 172.21.15.164:0/3108824419 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07dc146050 con 0x7f07dc0b1c70 2024-03-23T14:57:14.367 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.361+0000 7f07caffd640 1 -- 172.21.15.164:0/3108824419 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7f07cc00cca0 con 0x7f07dc0b1c70 2024-03-23T14:57:14.367 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.361+0000 7f07caffd640 1 --2- 172.21.15.164:0/3108824419 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f07bc0672a0 0x7f07bc069760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:14.368 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.361+0000 7f07caffd640 1 -- 172.21.15.164:0/3108824419 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f07cc085110 con 0x7f07dc0b1c70 2024-03-23T14:57:14.368 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.361+0000 7f07da59c640 1 --2- 172.21.15.164:0/3108824419 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f07bc0672a0 0x7f07bc069760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:14.369 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.365+0000 7f07da59c640 1 --2- 172.21.15.164:0/3108824419 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f07bc0672a0 0x7f07bc069760 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f07d00ae000 tx=0x7f07d00a7000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:14.369 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.365+0000 7f07e4110000 1 Processor -- start 2024-03-23T14:57:14.369 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.365+0000 7f07e4110000 1 -- start start 2024-03-23T14:57:14.370 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.365+0000 7f07e4110000 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55e4ea974890 0x55e4ea97a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:14.370 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.365+0000 7f07e4110000 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4ea974c70 0x55e4ea97add0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:14.370 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.365+0000 7f07e4110000 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e4ea97b310 0x55e4ea9afdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:14.370 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.365+0000 7f07e4110000 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55e4ea837ed0 con 0x55e4ea974c70 2024-03-23T14:57:14.370 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.365+0000 7f07e4110000 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55e4ea7c99e0 con 0x55e4ea97b310 2024-03-23T14:57:14.370 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.365+0000 7f07e4110000 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x55e4ea7c6f10 con 0x55e4ea974890 2024-03-23T14:57:14.370 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.365+0000 7f07d9d9b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4ea974c70 0x55e4ea97add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:14.370 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.365+0000 7f07d9d9b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4ea974c70 0x55e4ea97add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:60194/0 (socket says 172.21.15.164:60194) 2024-03-23T14:57:14.370 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.365+0000 7f07d9d9b640 1 -- 172.21.15.164:0/24480865 learned_addr learned my addr 172.21.15.164:0/24480865 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:14.370 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.365+0000 7f07da59c640 1 --2- 172.21.15.164:0/24480865 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55e4ea974890 0x55e4ea97a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:14.370 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.365+0000 7f07d9d9b640 1 -- 172.21.15.164:0/24480865 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55e4ea974890 msgr2=0x55e4ea97a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:14.370 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.365+0000 7f07d9d9b640 1 --2- 172.21.15.164:0/24480865 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55e4ea974890 0x55e4ea97a890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:14.370 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.365+0000 7f07dad9d640 1 --2- 172.21.15.164:0/24480865 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e4ea97b310 0x55e4ea9afdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:14.370 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.365+0000 7f07d9d9b640 1 -- 172.21.15.164:0/24480865 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e4ea97b310 msgr2=0x55e4ea9afdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:14.370 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.365+0000 7f07d9d9b640 1 --2- 172.21.15.164:0/24480865 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e4ea97b310 0x55e4ea9afdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:14.371 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.365+0000 7f07d9d9b640 1 -- 172.21.15.164:0/24480865 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4ea4b8890 con 0x55e4ea974c70 2024-03-23T14:57:14.371 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.365+0000 7f07d9d9b640 1 --2- 172.21.15.164:0/24480865 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4ea974c70 0x55e4ea97add0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f07cc05dc90 tx=0x7f07cc0562f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=993039a4f572815f server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:14.371 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.365+0000 7f07bbfff640 1 -- 172.21.15.164:0/24480865 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07cc05e1c0 con 0x55e4ea974c70 2024-03-23T14:57:14.371 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.365+0000 7f07bbfff640 1 -- 172.21.15.164:0/24480865 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07cc053db0 con 0x55e4ea974c70 2024-03-23T14:57:14.371 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.365+0000 7f07bbfff640 1 -- 172.21.15.164:0/24480865 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07cc054f00 con 0x55e4ea974c70 2024-03-23T14:57:14.376 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.365+0000 7f07e4110000 1 -- 172.21.15.164:0/24480865 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4ea974c70 msgr2=0x55e4ea97add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:14.376 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.365+0000 7f07e4110000 1 --2- 172.21.15.164:0/24480865 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4ea974c70 0x55e4ea97add0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f07cc05dc90 tx=0x7f07cc0562f0 comp rx=0 tx=0).stop 2024-03-23T14:57:14.377 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.365+0000 7f07e4110000 1 -- 172.21.15.164:0/24480865 shutdown_connections 2024-03-23T14:57:14.377 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.365+0000 7f07e4110000 1 --2- 172.21.15.164:0/24480865 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e4ea97b310 0x55e4ea9afdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:14.377 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.365+0000 7f07e4110000 1 --2- 172.21.15.164:0/24480865 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4ea974c70 0x55e4ea97add0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:14.377 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.365+0000 7f07e4110000 1 --2- 172.21.15.164:0/24480865 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55e4ea974890 0x55e4ea97a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:14.377 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.365+0000 7f07e4110000 1 -- 172.21.15.164:0/24480865 >> 172.21.15.164:0/24480865 conn(0x55e4ea970d80 msgr2=0x55e4ea973170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:14.377 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.365+0000 7f07e4110000 1 -- 172.21.15.164:0/24480865 shutdown_connections 2024-03-23T14:57:14.377 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.365+0000 7f07e4110000 1 -- 172.21.15.164:0/24480865 wait complete. 2024-03-23T14:57:14.377 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.365+0000 7f07e4110000 1 -- start start 2024-03-23T14:57:14.377 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.365+0000 7f07e4110000 1 Processor -- start 2024-03-23T14:57:14.377 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.365+0000 7f07e4110000 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4ea974890 0x55e4ea9a9780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:14.377 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.365+0000 7f07e4110000 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55e4ea974c70 0x55e4ea9aa400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:14.377 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.365+0000 7f07e4110000 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e4ea97b310 0x55e4ea9aa940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:14.378 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.365+0000 7f07e4110000 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55e4ea7c26f0 con 0x55e4ea974890 2024-03-23T14:57:14.378 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.365+0000 7f07e4110000 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55e4ea75c6c0 con 0x55e4ea97b310 2024-03-23T14:57:14.378 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.365+0000 7f07e4110000 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x55e4ea75c360 con 0x55e4ea974c70 2024-03-23T14:57:14.378 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.365+0000 7f07e4110000 10 client.0 _ll_register_callbacks cb 0x55e4ea77d280 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T14:57:14.378 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.365+0000 7f07da59c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4ea974890 0x55e4ea9a9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:14.378 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.365+0000 7f07da59c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4ea974890 0x55e4ea9a9780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:60210/0 (socket says 172.21.15.164:60210) 2024-03-23T14:57:14.378 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.365+0000 7f07da59c640 1 -- 172.21.15.164:0/1323399583 learned_addr learned my addr 172.21.15.164:0/1323399583 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:14.378 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.365+0000 7f07d9d9b640 1 --2- 172.21.15.164:0/1323399583 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55e4ea974c70 0x55e4ea9aa400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:14.378 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.365+0000 7f07dad9d640 1 --2- 172.21.15.164:0/1323399583 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e4ea97b310 0x55e4ea9aa940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:14.378 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.365+0000 7f07da59c640 1 -- 172.21.15.164:0/1323399583 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55e4ea974c70 msgr2=0x55e4ea9aa400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:14.378 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.365+0000 7f07da59c640 1 --2- 172.21.15.164:0/1323399583 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55e4ea974c70 0x55e4ea9aa400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:14.378 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.365+0000 7f07da59c640 1 -- 172.21.15.164:0/1323399583 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e4ea97b310 msgr2=0x55e4ea9aa940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:14.378 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.365+0000 7f07da59c640 1 --2- 172.21.15.164:0/1323399583 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e4ea97b310 0x55e4ea9aa940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:14.379 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.365+0000 7f07da59c640 1 -- 172.21.15.164:0/1323399583 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07dc146050 con 0x55e4ea974890 2024-03-23T14:57:14.379 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.365+0000 7f07dad9d640 1 --2- 172.21.15.164:0/1323399583 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e4ea97b310 0x55e4ea9aa940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T14:57:14.379 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.369+0000 7f07da59c640 1 --2- 172.21.15.164:0/1323399583 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4ea974890 0x55e4ea9a9780 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f07d001c980 tx=0x7f07d001ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4df8edbeebcc73 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:14.379 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.369+0000 7f07b97fa640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-03-23T14:57:14.379 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.369+0000 7f07b97fa640 1 -- 172.21.15.164:0/1323399583 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07d00c85f0 con 0x55e4ea974890 2024-03-23T14:57:14.379 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.369+0000 7f07b97fa640 1 -- 172.21.15.164:0/1323399583 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f07d00aae10 con 0x55e4ea974890 2024-03-23T14:57:14.379 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.369+0000 7f07e4110000 1 -- 172.21.15.164:0/1323399583 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e4ea7758b0 con 0x55e4ea974890 2024-03-23T14:57:14.379 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.369+0000 7f07b97fa640 1 -- 172.21.15.164:0/1323399583 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=6) ==== 24+0+0 (secure 0 0 0) 0x7f07d00b33f0 con 0x55e4ea974890 2024-03-23T14:57:14.379 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.369+0000 7f07e4110000 10 client.4605 fetch_fsmap learned FSMap version 6 2024-03-23T14:57:14.379 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.369+0000 7f07e4110000 1 -- 172.21.15.164:0/1323399583 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) -- 0x55e4ea781ed0 con 0x55e4ea974890 2024-03-23T14:57:14.379 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.369+0000 7f07b97fa640 1 -- 172.21.15.164:0/1323399583 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55e4ea7758b0 con 0x55e4ea974890 2024-03-23T14:57:14.379 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.369+0000 7f07b97fa640 1 -- 172.21.15.164:0/1323399583 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 6) ==== 2284+0+0 (secure 0 0 0) 0x7f07d00cf020 con 0x55e4ea974890 2024-03-23T14:57:14.379 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.369+0000 7f07e4110000 10 client.4605 fetch_fsmap finished waiting for FSMap version 6 2024-03-23T14:57:14.379 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.369+0000 7f07e4110000 10 client.4605 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-23T14:57:14.379 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.369+0000 7f07e4110000 20 client.4605 populate_metadata read hostname 'smithi164' 2024-03-23T14:57:14.380 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.369+0000 7f07e4110000 1 --2- 172.21.15.164:0/1323399583 >> [v2:172.21.15.173:6832/3506402771,v1:172.21.15.173:6833/3506402771] conn(0x55e4ea9ab350 0x55e4ea9ab730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:14.380 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.369+0000 7f07e4110000 4 client.4605 mds_command: new command op to 4279 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T14:57:14.380 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.369+0000 7f07e4110000 1 -- 172.21.15.164:0/1323399583 --> [v2:172.21.15.173:6832/3506402771,v1:172.21.15.173:6833/3506402771] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e4ea7bf4d0 con 0x55e4ea9ab350 2024-03-23T14:57:14.380 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.369+0000 7f07d9d9b640 1 --2- 172.21.15.164:0/1323399583 >> [v2:172.21.15.173:6832/3506402771,v1:172.21.15.173:6833/3506402771] conn(0x55e4ea9ab350 0x55e4ea9ab730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:14.380 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.369+0000 7f07b97fa640 10 client.4605 ms_handle_connect on v2:172.21.15.173:6832/3506402771 2024-03-23T14:57:14.380 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.369+0000 7f07d9d9b640 1 --2- 172.21.15.164:0/1323399583 >> [v2:172.21.15.173:6832/3506402771,v1:172.21.15.173:6833/3506402771] conn(0x55e4ea9ab350 0x55e4ea9ab730 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dbd192dbc8be7990 server_cookie=c9f78e3c8f5b72a5 in_seq=0 out_seq=0 2024-03-23T14:57:14.380 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.369+0000 7f07b97fa640 1 -- 172.21.15.164:0/1323399583 <== mds.0 v2:172.21.15.173:6832/3506402771 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55e4ea7bf4d0 con 0x55e4ea9ab350 2024-03-23T14:57:14.380 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.369+0000 7f07b97fa640 10 client.4605 handle_command_reply: tid=0 2024-03-23T14:57:14.380 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.369+0000 7f07e4110000 1 client.4605 shutdown 2024-03-23T14:57:14.380 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.369+0000 7f07e4110000 1 -- 172.21.15.164:0/1323399583 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4ea974890 msgr2=0x55e4ea9a9780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:14.380 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.369+0000 7f07e4110000 1 --2- 172.21.15.164:0/1323399583 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4ea974890 0x55e4ea9a9780 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f07d001c980 tx=0x7f07d001ace0 comp rx=0 tx=0).stop 2024-03-23T14:57:14.380 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.369+0000 7f07e4110000 1 -- 172.21.15.164:0/1323399583 shutdown_connections 2024-03-23T14:57:14.380 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.369+0000 7f07e4110000 1 --2- 172.21.15.164:0/1323399583 >> [v2:172.21.15.173:6832/3506402771,v1:172.21.15.173:6833/3506402771] conn(0x55e4ea9ab350 0x55e4ea9ab730 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:14.380 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.369+0000 7f07e4110000 1 --2- 172.21.15.164:0/1323399583 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e4ea97b310 0x55e4ea9aa940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:14.380 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.369+0000 7f07e4110000 1 --2- 172.21.15.164:0/1323399583 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55e4ea974c70 0x55e4ea9aa400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:14.381 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.369+0000 7f07b97fa640 0 client.4605 ms_handle_reset on v2:172.21.15.173:6832/3506402771 2024-03-23T14:57:14.381 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.369+0000 7f07e4110000 1 --2- 172.21.15.164:0/1323399583 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4ea974890 0x55e4ea9a9780 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:14.381 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.369+0000 7f07e4110000 1 -- 172.21.15.164:0/1323399583 >> 172.21.15.164:0/1323399583 conn(0x55e4ea970d80 msgr2=0x55e4ea975050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:14.381 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.369+0000 7f07e4110000 1 -- 172.21.15.164:0/1323399583 shutdown_connections 2024-03-23T14:57:14.381 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.369+0000 7f07e4110000 1 -- 172.21.15.164:0/1323399583 wait complete. 2024-03-23T14:57:14.381 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.369+0000 7f07e4110000 20 client.4605 trim_cache size 0 max 16384 2024-03-23T14:57:14.392 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.389+0000 7f07e4110000 1 Processor -- start 2024-03-23T14:57:14.392 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.389+0000 7f07e4110000 1 -- start start 2024-03-23T14:57:14.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.389+0000 7f07e4110000 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e4ea974890 0x55e4ea9bfe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:14.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.389+0000 7f07e4110000 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4ea974c70 0x55e4ea9babc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:14.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.389+0000 7f07e4110000 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55e4ea97b310 0x55e4ea9bb100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:14.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.389+0000 7f07e4110000 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55e4ea479940 con 0x55e4ea974c70 2024-03-23T14:57:14.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.389+0000 7f07e4110000 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55e4ea7bff00 con 0x55e4ea974890 2024-03-23T14:57:14.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.389+0000 7f07e4110000 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x55e4ea9aaa10 con 0x55e4ea97b310 2024-03-23T14:57:14.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.389+0000 7f07d9d9b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4ea974c70 0x55e4ea9babc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:14.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.389+0000 7f07d9d9b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4ea974c70 0x55e4ea9babc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:60226/0 (socket says 172.21.15.164:60226) 2024-03-23T14:57:14.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.389+0000 7f07d9d9b640 1 -- 172.21.15.164:0/4281715337 learned_addr learned my addr 172.21.15.164:0/4281715337 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:14.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.389+0000 7f07dad9d640 1 --2- 172.21.15.164:0/4281715337 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55e4ea97b310 0x55e4ea9bb100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:14.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.389+0000 7f07da59c640 1 --2- 172.21.15.164:0/4281715337 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e4ea974890 0x55e4ea9bfe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:14.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.389+0000 7f07d9d9b640 1 -- 172.21.15.164:0/4281715337 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55e4ea97b310 msgr2=0x55e4ea9bb100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:14.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.389+0000 7f07d9d9b640 1 --2- 172.21.15.164:0/4281715337 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55e4ea97b310 0x55e4ea9bb100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:14.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.389+0000 7f07d9d9b640 1 -- 172.21.15.164:0/4281715337 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e4ea974890 msgr2=0x55e4ea9bfe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:14.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.389+0000 7f07d9d9b640 1 --2- 172.21.15.164:0/4281715337 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e4ea974890 0x55e4ea9bfe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:14.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.389+0000 7f07d9d9b640 1 -- 172.21.15.164:0/4281715337 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4ea767e70 con 0x55e4ea974c70 2024-03-23T14:57:14.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.389+0000 7f07d9d9b640 1 --2- 172.21.15.164:0/4281715337 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4ea974c70 0x55e4ea9babc0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f07cc0853f0 tx=0x7f07cc057000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3035f7738d3a56a3 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:14.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.389+0000 7f07bb7fe640 1 -- 172.21.15.164:0/4281715337 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07cc055880 con 0x55e4ea974c70 2024-03-23T14:57:14.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.389+0000 7f07bb7fe640 1 -- 172.21.15.164:0/4281715337 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07cc053db0 con 0x55e4ea974c70 2024-03-23T14:57:14.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.389+0000 7f07bb7fe640 1 -- 172.21.15.164:0/4281715337 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07cc083210 con 0x55e4ea974c70 2024-03-23T14:57:14.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.389+0000 7f07e4110000 1 -- 172.21.15.164:0/4281715337 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4ea974c70 msgr2=0x55e4ea9babc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:14.396 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.389+0000 7f07e4110000 1 --2- 172.21.15.164:0/4281715337 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4ea974c70 0x55e4ea9babc0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f07cc0853f0 tx=0x7f07cc057000 comp rx=0 tx=0).stop 2024-03-23T14:57:14.396 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.389+0000 7f07e4110000 1 -- 172.21.15.164:0/4281715337 shutdown_connections 2024-03-23T14:57:14.396 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.389+0000 7f07e4110000 1 --2- 172.21.15.164:0/4281715337 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55e4ea97b310 0x55e4ea9bb100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:14.396 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.389+0000 7f07e4110000 1 --2- 172.21.15.164:0/4281715337 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4ea974c70 0x55e4ea9babc0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:14.396 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.389+0000 7f07e4110000 1 --2- 172.21.15.164:0/4281715337 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e4ea974890 0x55e4ea9bfe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:14.396 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.389+0000 7f07e4110000 1 -- 172.21.15.164:0/4281715337 >> 172.21.15.164:0/4281715337 conn(0x55e4ea970d80 msgr2=0x55e4ea9a94e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:14.396 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.389+0000 7f07e4110000 1 -- 172.21.15.164:0/4281715337 shutdown_connections 2024-03-23T14:57:14.396 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.389+0000 7f07e4110000 1 -- 172.21.15.164:0/4281715337 wait complete. 2024-03-23T14:57:14.396 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.389+0000 7f07e4110000 1 -- start start 2024-03-23T14:57:14.396 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.389+0000 7f07e4110000 1 Processor -- start 2024-03-23T14:57:14.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.389+0000 7f07e4110000 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e4ea974890 0x55e4ea9bd340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:14.399 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.389+0000 7f07e4110000 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55e4ea974c70 0x55e4ea9bd880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:14.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.389+0000 7f07e4110000 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4ea97b310 0x55e4ea9bddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:14.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.389+0000 7f07e4110000 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55e4ea96c540 con 0x55e4ea97b310 2024-03-23T14:57:14.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.389+0000 7f07e4110000 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55e4ea77a160 con 0x55e4ea974890 2024-03-23T14:57:14.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.389+0000 7f07e4110000 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x55e4ea7cd8e0 con 0x55e4ea974c70 2024-03-23T14:57:14.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.389+0000 7f07e4110000 10 client.0 _ll_register_callbacks cb 0x7f07cc009060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T14:57:14.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.389+0000 7f07dad9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4ea97b310 0x55e4ea9bddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:14.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.389+0000 7f07dad9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4ea97b310 0x55e4ea9bddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:60228/0 (socket says 172.21.15.164:60228) 2024-03-23T14:57:14.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.389+0000 7f07dad9d640 1 -- 172.21.15.164:0/3161231272 learned_addr learned my addr 172.21.15.164:0/3161231272 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:14.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.389+0000 7f07da59c640 1 --2- 172.21.15.164:0/3161231272 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e4ea974890 0x55e4ea9bd340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:14.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.389+0000 7f07d9d9b640 1 --2- 172.21.15.164:0/3161231272 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55e4ea974c70 0x55e4ea9bd880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:14.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.389+0000 7f07dad9d640 1 -- 172.21.15.164:0/3161231272 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55e4ea974c70 msgr2=0x55e4ea9bd880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:14.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.389+0000 7f07dad9d640 1 --2- 172.21.15.164:0/3161231272 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55e4ea974c70 0x55e4ea9bd880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:14.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.389+0000 7f07dad9d640 1 -- 172.21.15.164:0/3161231272 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e4ea974890 msgr2=0x55e4ea9bd340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:14.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.389+0000 7f07dad9d640 1 --2- 172.21.15.164:0/3161231272 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e4ea974890 0x55e4ea9bd340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:14.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.389+0000 7f07dad9d640 1 -- 172.21.15.164:0/3161231272 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4ea4b8890 con 0x55e4ea97b310 2024-03-23T14:57:14.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.389+0000 7f07da59c640 1 --2- 172.21.15.164:0/3161231272 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e4ea974890 0x55e4ea9bd340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T14:57:14.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.389+0000 7f07dad9d640 1 --2- 172.21.15.164:0/3161231272 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4ea97b310 0x55e4ea9bddc0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f07d40115f0 tx=0x7f07d4011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64ba35dbe7649383 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:14.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.389+0000 7f07b97fa640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-03-23T14:57:14.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.393+0000 7f07b97fa640 1 -- 172.21.15.164:0/3161231272 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07d4003d90 con 0x55e4ea97b310 2024-03-23T14:57:14.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.393+0000 7f07b97fa640 1 -- 172.21.15.164:0/3161231272 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f07d40050d0 con 0x55e4ea97b310 2024-03-23T14:57:14.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.393+0000 7f07e4110000 1 -- 172.21.15.164:0/3161231272 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e4ea7c9d10 con 0x55e4ea97b310 2024-03-23T14:57:14.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.393+0000 7f07b97fa640 1 -- 172.21.15.164:0/3161231272 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=6) ==== 24+0+0 (secure 0 0 0) 0x7f07d4005290 con 0x55e4ea97b310 2024-03-23T14:57:14.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.393+0000 7f07e4110000 10 client.4611 fetch_fsmap learned FSMap version 6 2024-03-23T14:57:14.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.393+0000 7f07e4110000 1 -- 172.21.15.164:0/3161231272 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) -- 0x55e4ea4b8cc0 con 0x55e4ea97b310 2024-03-23T14:57:14.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.393+0000 7f07b97fa640 1 -- 172.21.15.164:0/3161231272 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55e4ea7c9d10 con 0x55e4ea97b310 2024-03-23T14:57:14.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.393+0000 7f07b97fa640 1 -- 172.21.15.164:0/3161231272 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 6) ==== 2284+0+0 (secure 0 0 0) 0x7f07d4012040 con 0x55e4ea97b310 2024-03-23T14:57:14.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.393+0000 7f07e4110000 10 client.4611 fetch_fsmap finished waiting for FSMap version 6 2024-03-23T14:57:14.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.393+0000 7f07e4110000 10 client.4611 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-23T14:57:14.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.393+0000 7f07e4110000 20 client.4611 populate_metadata read hostname 'smithi164' 2024-03-23T14:57:14.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.393+0000 7f07e4110000 1 --2- 172.21.15.164:0/3161231272 >> [v2:172.21.15.173:6832/3506402771,v1:172.21.15.173:6833/3506402771] conn(0x55e4ea9be300 0x55e4eaa213a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:14.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.393+0000 7f07e4110000 4 client.4611 mds_command: new command op to 4279 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-23T14:57:14.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.393+0000 7f07e4110000 1 -- 172.21.15.164:0/3161231272 --> [v2:172.21.15.173:6832/3506402771,v1:172.21.15.173:6833/3506402771] -- command(tid 0: {"prefix": "status"}) -- 0x55e4ea7ed9e0 con 0x55e4ea9be300 2024-03-23T14:57:14.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.393+0000 7f07da59c640 1 --2- 172.21.15.164:0/3161231272 >> [v2:172.21.15.173:6832/3506402771,v1:172.21.15.173:6833/3506402771] conn(0x55e4ea9be300 0x55e4eaa213a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:14.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.393+0000 7f07da59c640 1 --2- 172.21.15.164:0/3161231272 >> [v2:172.21.15.173:6832/3506402771,v1:172.21.15.173:6833/3506402771] conn(0x55e4ea9be300 0x55e4eaa213a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6877bda778e5b43 server_cookie=5fd992dd0b0d3572 in_seq=0 out_seq=0 2024-03-23T14:57:14.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.393+0000 7f07b97fa640 10 client.4611 ms_handle_connect on v2:172.21.15.173:6832/3506402771 2024-03-23T14:57:14.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.393+0000 7f07b97fa640 1 -- 172.21.15.164:0/3161231272 <== mds.0 v2:172.21.15.173:6832/3506402771 1 ==== command_reply(tid 0: 0 ) ==== 8+0+324 (crc 0 0 0) 0x55e4ea7ed9e0 con 0x55e4ea9be300 2024-03-23T14:57:14.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.393+0000 7f07b97fa640 10 client.4611 handle_command_reply: tid=0 2024-03-23T14:57:14.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.393+0000 7f07e4110000 1 client.4611 shutdown 2024-03-23T14:57:14.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.393+0000 7f07e4110000 1 -- 172.21.15.164:0/3161231272 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4ea97b310 msgr2=0x55e4ea9bddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:14.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.393+0000 7f07e4110000 1 --2- 172.21.15.164:0/3161231272 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4ea97b310 0x55e4ea9bddc0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f07d40115f0 tx=0x7f07d4011ab0 comp rx=0 tx=0).stop 2024-03-23T14:57:14.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.393+0000 7f07e4110000 1 -- 172.21.15.164:0/3161231272 shutdown_connections 2024-03-23T14:57:14.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.393+0000 7f07e4110000 1 --2- 172.21.15.164:0/3161231272 >> [v2:172.21.15.173:6832/3506402771,v1:172.21.15.173:6833/3506402771] conn(0x55e4ea9be300 0x55e4eaa213a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:14.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.393+0000 7f07e4110000 1 --2- 172.21.15.164:0/3161231272 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55e4ea97b310 0x55e4ea9bddc0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:14.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.393+0000 7f07e4110000 1 --2- 172.21.15.164:0/3161231272 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55e4ea974c70 0x55e4ea9bd880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:14.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.393+0000 7f07b97fa640 0 client.4611 ms_handle_reset on v2:172.21.15.173:6832/3506402771 2024-03-23T14:57:14.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.393+0000 7f07e4110000 1 --2- 172.21.15.164:0/3161231272 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e4ea974890 0x55e4ea9bd340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:14.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.393+0000 7f07e4110000 1 -- 172.21.15.164:0/3161231272 >> 172.21.15.164:0/3161231272 conn(0x55e4ea970d80 msgr2=0x55e4ea9a8c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:14.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.393+0000 7f07e4110000 1 -- 172.21.15.164:0/3161231272 shutdown_connections 2024-03-23T14:57:14.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.393+0000 7f07e4110000 1 -- 172.21.15.164:0/3161231272 wait complete. 2024-03-23T14:57:14.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.393+0000 7f07e4110000 20 client.4611 trim_cache size 0 max 16384 2024-03-23T14:57:14.403 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-03-23T14:57:14.403 INFO:teuthology.orchestra.run.smithi164.stdout: "cluster_fsid": "45c7911b-b69d-4e68-907b-fbfeb780aa3b", 2024-03-23T14:57:14.403 INFO:teuthology.orchestra.run.smithi164.stdout: "whoami": 0, 2024-03-23T14:57:14.403 INFO:teuthology.orchestra.run.smithi164.stdout: "id": 4279, 2024-03-23T14:57:14.403 INFO:teuthology.orchestra.run.smithi164.stdout: "want_state": "up:active", 2024-03-23T14:57:14.404 INFO:teuthology.orchestra.run.smithi164.stdout: "state": "up:active", 2024-03-23T14:57:14.404 INFO:teuthology.orchestra.run.smithi164.stdout: "fs_name": "cephfs", 2024-03-23T14:57:14.404 INFO:teuthology.orchestra.run.smithi164.stdout: "rank_uptime": 14.784462266, 2024-03-23T14:57:14.404 INFO:teuthology.orchestra.run.smithi164.stdout: "mdsmap_epoch": 6, 2024-03-23T14:57:14.404 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_epoch": 20, 2024-03-23T14:57:14.404 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_epoch_barrier": 20, 2024-03-23T14:57:14.404 INFO:teuthology.orchestra.run.smithi164.stdout: "uptime": 21.845688504000002 2024-03-23T14:57:14.404 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-03-23T14:57:14.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.393+0000 7f07e0f9b640 1 -- 172.21.15.164:0/3108824419 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f07bc0672a0 msgr2=0x7f07bc069760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:14.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.393+0000 7f07e0f9b640 1 --2- 172.21.15.164:0/3108824419 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f07bc0672a0 0x7f07bc069760 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f07d00ae000 tx=0x7f07d00a7000 comp rx=0 tx=0).stop 2024-03-23T14:57:14.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.393+0000 7f07e0f9b640 1 -- 172.21.15.164:0/3108824419 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f07dc0b1c70 msgr2=0x7f07dc06f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:14.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.393+0000 7f07e0f9b640 1 --2- 172.21.15.164:0/3108824419 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f07dc0b1c70 0x7f07dc06f890 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f07cc00e870 tx=0x7f07cc00ec70 comp rx=0 tx=0).stop 2024-03-23T14:57:14.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.393+0000 7f07e0f9b640 1 -- 172.21.15.164:0/3108824419 shutdown_connections 2024-03-23T14:57:14.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.393+0000 7f07e0f9b640 1 --2- 172.21.15.164:0/3108824419 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f07bc0672a0 0x7f07bc069760 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:14.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.393+0000 7f07e0f9b640 1 --2- 172.21.15.164:0/3108824419 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f07dc148cf0 0x7f07dc144b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:14.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.393+0000 7f07e0f9b640 1 --2- 172.21.15.164:0/3108824419 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f07dc0b1c70 0x7f07dc06f890 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:14.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.393+0000 7f07e0f9b640 1 --2- 172.21.15.164:0/3108824419 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f07dc0b0230 0x7f07dc09c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:14.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.393+0000 7f07e0f9b640 1 -- 172.21.15.164:0/3108824419 >> 172.21.15.164:0/3108824419 conn(0x7f07dc0b8ca0 msgr2=0x7f07dc13f290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:14.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.393+0000 7f07e0f9b640 1 -- 172.21.15.164:0/3108824419 shutdown_connections 2024-03-23T14:57:14.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.393+0000 7f07e0f9b640 1 -- 172.21.15.164:0/3108824419 wait complete. 2024-03-23T14:57:14.413 INFO:teuthology.run_tasks:Running task sequential... 2024-03-23T14:57:14.423 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-03-23T14:57:14.423 INFO:teuthology.task.sequential:In sequential, running task print... 2024-03-23T14:57:14.423 INFO:teuthology.task.print:Enabling mgr modules 2024-03-23T14:57:14.423 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-03-23T14:57:14.444 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-03-23T14:57:14.444 INFO:tasks.ceph_fuse:config is {'client.0': {}} 2024-03-23T14:57:14.445 INFO:tasks.ceph_fuse:client.0 config is {} 2024-03-23T14:57:14.445 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-23T14:57:14.445 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-23T14:57:14.445 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T14:57:14.445 DEBUG:teuthology.orchestra.run.smithi164:> ip netns list 2024-03-23T14:57:14.450 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T14:57:14.450 DEBUG:teuthology.orchestra.run.smithi164:> sudo ip link delete ceph-brx 2024-03-23T14:57:14.502 INFO:teuthology.orchestra.run.smithi164.stderr:Cannot find device "ceph-brx" 2024-03-23T14:57:14.503 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-23T14:57:14.503 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-03-23T14:57:14.503 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-23T14:57:14.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.661+0000 7fb453288640 1 Processor -- start 2024-03-23T14:57:14.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.661+0000 7fb453288640 1 -- start start 2024-03-23T14:57:14.668 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.661+0000 7fb453288640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fb44c0b56f0 0x7fb44c14a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:14.669 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.661+0000 7fb453288640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fb44c141640 0x7fb44c151c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:14.669 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.661+0000 7fb453288640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb44c0b4300 con 0x7fb44c0b4a70 2024-03-23T14:57:14.669 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.661+0000 7fb453288640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fb44c0b5fa0 con 0x7fb44c0b56f0 2024-03-23T14:57:14.669 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.661+0000 7fb453288640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 -- 0x7fb44c0b45e0 con 0x7fb44c141640 2024-03-23T14:57:14.669 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.665+0000 7fb451024640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fb44c0b56f0 0x7fb44c14a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:14.669 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.665+0000 7fb451024640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fb44c0b56f0 0x7fb44c14a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:60236/0 (socket says 172.21.15.164:60236) 2024-03-23T14:57:14.669 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.665+0000 7fb451024640 1 -- 172.21.15.164:0/4033955858 learned_addr learned my addr 172.21.15.164:0/4033955858 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:14.669 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.665+0000 7fb450823640 1 --1- 172.21.15.164:0/4033955858 >> v1:172.21.15.164:6789/0 conn(0x7fb44c0b4a70 0x7fb44c140f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:56866/0 (socket says 172.21.15.164:56866) 2024-03-23T14:57:14.669 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.665+0000 7fb451825640 1 --2- 172.21.15.164:0/4033955858 >> v2:172.21.15.173:3300/0 conn(0x7fb44c141640 0x7fb44c151c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:14.669 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.665+0000 7fb451024640 1 -- 172.21.15.164:0/4033955858 >> v2:172.21.15.173:3300/0 conn(0x7fb44c141640 msgr2=0x7fb44c151c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:14.669 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.665+0000 7fb451024640 1 --2- 172.21.15.164:0/4033955858 >> v2:172.21.15.173:3300/0 conn(0x7fb44c141640 0x7fb44c151c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:14.669 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.665+0000 7fb451024640 1 -- 172.21.15.164:0/4033955858 >> v1:172.21.15.164:6789/0 conn(0x7fb44c0b4a70 legacy=0x7fb44c140f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:14.669 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.665+0000 7fb443fff640 1 -- 172.21.15.164:0/4033955858 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2404400266 0 0) 0x7fb44c0b4300 con 0x7fb44c0b4a70 2024-03-23T14:57:14.669 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.665+0000 7fb451024640 1 -- 172.21.15.164:0/4033955858 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb44c14ade0 con 0x7fb44c0b56f0 2024-03-23T14:57:14.671 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.665+0000 7fb451024640 1 --2- 172.21.15.164:0/4033955858 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb44c0b56f0 0x7fb44c14a8a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb44c073430 tx=0x7fb44801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=250fca80cb30f3f5 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:14.671 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.665+0000 7fb443fff640 1 -- 172.21.15.164:0/4033955858 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4480aa650 con 0x7fb44c0b56f0 2024-03-23T14:57:14.671 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.665+0000 7fb443fff640 1 -- 172.21.15.164:0/4033955858 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4480aa7b0 con 0x7fb44c0b56f0 2024-03-23T14:57:14.671 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.665+0000 7fb443fff640 1 -- 172.21.15.164:0/4033955858 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4480aab40 con 0x7fb44c0b56f0 2024-03-23T14:57:14.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.665+0000 7fb453288640 1 -- 172.21.15.164:0/4033955858 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb44c0b56f0 msgr2=0x7fb44c14a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:14.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.669+0000 7fb453288640 1 --2- 172.21.15.164:0/4033955858 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb44c0b56f0 0x7fb44c14a8a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb44c073430 tx=0x7fb44801c4f0 comp rx=0 tx=0).stop 2024-03-23T14:57:14.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.669+0000 7fb453288640 1 -- 172.21.15.164:0/4033955858 shutdown_connections 2024-03-23T14:57:14.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.669+0000 7fb453288640 1 --2- 172.21.15.164:0/4033955858 >> v2:172.21.15.173:3300/0 conn(0x7fb44c141640 0x7fb44c151c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:14.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.669+0000 7fb453288640 1 --2- 172.21.15.164:0/4033955858 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb44c0b56f0 0x7fb44c14a8a0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:14.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.669+0000 7fb453288640 1 -- 172.21.15.164:0/4033955858 >> 172.21.15.164:0/4033955858 conn(0x7fb44c02c3c0 msgr2=0x7fb44c02c7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:14.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.669+0000 7fb453288640 1 -- 172.21.15.164:0/4033955858 shutdown_connections 2024-03-23T14:57:14.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.669+0000 7fb453288640 1 -- 172.21.15.164:0/4033955858 wait complete. 2024-03-23T14:57:14.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.669+0000 7fb453288640 1 Processor -- start 2024-03-23T14:57:14.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.669+0000 7fb453288640 1 -- start start 2024-03-23T14:57:14.675 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.669+0000 7fb453288640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb44c0b4a70 0x7fb44c141e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:14.675 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.669+0000 7fb453288640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb44c0b56f0 0x7fb44c143890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:14.675 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.669+0000 7fb453288640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb44c141640 0x7fb44c1dfee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:14.675 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.669+0000 7fb453288640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb44c144250 con 0x7fb44c0b4a70 2024-03-23T14:57:14.675 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.669+0000 7fb453288640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fb44c147190 con 0x7fb44c0b56f0 2024-03-23T14:57:14.675 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.669+0000 7fb453288640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fb44c1e3990 con 0x7fb44c141640 2024-03-23T14:57:14.675 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.669+0000 7fb451024640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb44c0b4a70 0x7fb44c141e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:14.675 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.669+0000 7fb451024640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb44c0b4a70 0x7fb44c141e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:60240/0 (socket says 172.21.15.164:60240) 2024-03-23T14:57:14.675 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.669+0000 7fb451024640 1 -- 172.21.15.164:0/1106801167 learned_addr learned my addr 172.21.15.164:0/1106801167 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:14.675 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.669+0000 7fb450823640 1 --2- 172.21.15.164:0/1106801167 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb44c0b56f0 0x7fb44c143890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:14.675 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.669+0000 7fb451825640 1 --2- 172.21.15.164:0/1106801167 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb44c141640 0x7fb44c1dfee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:14.675 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.669+0000 7fb451024640 1 -- 172.21.15.164:0/1106801167 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb44c141640 msgr2=0x7fb44c1dfee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:14.676 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.669+0000 7fb451024640 1 --2- 172.21.15.164:0/1106801167 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb44c141640 0x7fb44c1dfee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:14.676 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.669+0000 7fb451024640 1 -- 172.21.15.164:0/1106801167 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb44c0b56f0 msgr2=0x7fb44c143890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:14.676 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.669+0000 7fb451024640 1 --2- 172.21.15.164:0/1106801167 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb44c0b56f0 0x7fb44c143890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:14.676 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.669+0000 7fb451024640 1 -- 172.21.15.164:0/1106801167 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4480094d0 con 0x7fb44c0b4a70 2024-03-23T14:57:14.676 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.669+0000 7fb450823640 1 --2- 172.21.15.164:0/1106801167 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb44c0b56f0 0x7fb44c143890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T14:57:14.676 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.669+0000 7fb451024640 1 --2- 172.21.15.164:0/1106801167 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb44c0b4a70 0x7fb44c141e00 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb44801a120 tx=0x7fb44801ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:14.676 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.669+0000 7fb441ffb640 1 -- 172.21.15.164:0/1106801167 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4480bd5f0 con 0x7fb44c0b4a70 2024-03-23T14:57:14.676 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.669+0000 7fb441ffb640 1 -- 172.21.15.164:0/1106801167 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4480bd750 con 0x7fb44c0b4a70 2024-03-23T14:57:14.676 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.669+0000 7fb453288640 1 -- 172.21.15.164:0/1106801167 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb44c1e3c70 con 0x7fb44c0b4a70 2024-03-23T14:57:14.676 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.669+0000 7fb441ffb640 1 -- 172.21.15.164:0/1106801167 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4480bdb00 con 0x7fb44c0b4a70 2024-03-23T14:57:14.676 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.669+0000 7fb453288640 1 -- 172.21.15.164:0/1106801167 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb44c143dd0 con 0x7fb44c0b4a70 2024-03-23T14:57:14.676 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.669+0000 7fb453288640 1 -- 172.21.15.164:0/1106801167 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb44c0690a0 con 0x7fb44c0b4a70 2024-03-23T14:57:14.677 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.673+0000 7fb441ffb640 1 -- 172.21.15.164:0/1106801167 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7fb4480b33f0 con 0x7fb44c0b4a70 2024-03-23T14:57:14.677 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.673+0000 7fb441ffb640 1 --2- 172.21.15.164:0/1106801167 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fb4300672a0 0x7fb430069760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:14.677 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.673+0000 7fb441ffb640 1 -- 172.21.15.164:0/1106801167 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fb4481203d0 con 0x7fb44c0b4a70 2024-03-23T14:57:14.677 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.673+0000 7fb450823640 1 --2- 172.21.15.164:0/1106801167 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fb4300672a0 0x7fb430069760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:14.677 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.673+0000 7fb450823640 1 --2- 172.21.15.164:0/1106801167 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fb4300672a0 0x7fb430069760 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fb44c1dda10 tx=0x7fb43c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:14.680 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.677+0000 7fb441ffb640 1 -- 172.21.15.164:0/1106801167 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7fb4480ed0f0 con 0x7fb44c0b4a70 2024-03-23T14:57:14.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.869+0000 7fb453288640 1 -- 172.21.15.164:0/1106801167 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fb44c1e4120 con 0x7fb44c0b4a70 2024-03-23T14:57:14.874 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.869+0000 7fb441ffb640 1 -- 172.21.15.164:0/1106801167 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) ==== 53+0+73 (secure 0 0 0) 0x7fb4480f07b0 con 0x7fb44c0b4a70 2024-03-23T14:57:14.874 INFO:teuthology.orchestra.run.smithi164.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-23T14:57:14.876 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.873+0000 7fb453288640 1 -- 172.21.15.164:0/1106801167 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fb4300672a0 msgr2=0x7fb430069760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:14.876 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.873+0000 7fb453288640 1 --2- 172.21.15.164:0/1106801167 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fb4300672a0 0x7fb430069760 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fb44c1dda10 tx=0x7fb43c009040 comp rx=0 tx=0).stop 2024-03-23T14:57:14.876 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.873+0000 7fb453288640 1 -- 172.21.15.164:0/1106801167 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb44c0b4a70 msgr2=0x7fb44c141e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:14.877 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.873+0000 7fb453288640 1 --2- 172.21.15.164:0/1106801167 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb44c0b4a70 0x7fb44c141e00 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb44801a120 tx=0x7fb44801ad30 comp rx=0 tx=0).stop 2024-03-23T14:57:14.877 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.873+0000 7fb453288640 1 -- 172.21.15.164:0/1106801167 shutdown_connections 2024-03-23T14:57:14.877 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.873+0000 7fb453288640 1 --2- 172.21.15.164:0/1106801167 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb44c141640 0x7fb44c1dfee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:14.877 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.873+0000 7fb453288640 1 --2- 172.21.15.164:0/1106801167 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fb4300672a0 0x7fb430069760 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:14.877 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.873+0000 7fb453288640 1 --2- 172.21.15.164:0/1106801167 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb44c0b56f0 0x7fb44c143890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:14.877 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.873+0000 7fb453288640 1 --2- 172.21.15.164:0/1106801167 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb44c0b4a70 0x7fb44c141e00 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:14.877 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.873+0000 7fb453288640 1 -- 172.21.15.164:0/1106801167 >> 172.21.15.164:0/1106801167 conn(0x7fb44c02c3c0 msgr2=0x7fb44c13fb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:14.877 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.873+0000 7fb453288640 1 -- 172.21.15.164:0/1106801167 shutdown_connections 2024-03-23T14:57:14.877 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14.873+0000 7fb453288640 1 -- 172.21.15.164:0/1106801167 wait complete. 2024-03-23T14:57:14.893 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-23T14:57:14.893 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-23T14:57:14.893 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi164.front.sepia.ceph.com 2024-03-23T14:57:14.893 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-23T14:57:14.893 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-23T14:57:14.893 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-23T14:57:14.893 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-23T14:57:14.893 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-23T14:57:14.893 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-23T14:57:14.894 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T14:57:14.894 DEBUG:teuthology.orchestra.run.smithi164:> ip addr 2024-03-23T14:57:14.900 INFO:teuthology.orchestra.run.smithi164.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-23T14:57:14.900 INFO:teuthology.orchestra.run.smithi164.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-23T14:57:14.900 INFO:teuthology.orchestra.run.smithi164.stdout: inet 127.0.0.1/8 scope host lo 2024-03-23T14:57:14.900 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-03-23T14:57:14.900 INFO:teuthology.orchestra.run.smithi164.stdout: inet6 ::1/128 scope host 2024-03-23T14:57:14.900 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-03-23T14:57:14.900 INFO:teuthology.orchestra.run.smithi164.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-23T14:57:14.900 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether 0c:c4:7a:d9:34:f4 brd ff:ff:ff:ff:ff:ff 2024-03-23T14:57:14.900 INFO:teuthology.orchestra.run.smithi164.stdout: altname enp7s0f0 2024-03-23T14:57:14.900 INFO:teuthology.orchestra.run.smithi164.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-23T14:57:14.900 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether 0c:c4:7a:d9:34:f5 brd ff:ff:ff:ff:ff:ff 2024-03-23T14:57:14.900 INFO:teuthology.orchestra.run.smithi164.stdout: altname enp7s0f1 2024-03-23T14:57:14.901 INFO:teuthology.orchestra.run.smithi164.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-23T14:57:14.901 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether 0c:c4:7a:88:73:f2 brd ff:ff:ff:ff:ff:ff 2024-03-23T14:57:14.901 INFO:teuthology.orchestra.run.smithi164.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-23T14:57:14.901 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether 0c:c4:7a:88:73:f3 brd ff:ff:ff:ff:ff:ff 2024-03-23T14:57:14.901 INFO:teuthology.orchestra.run.smithi164.stdout: inet 172.21.15.164/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-03-23T14:57:14.901 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft 42513sec preferred_lft 42513sec 2024-03-23T14:57:14.901 INFO:teuthology.orchestra.run.smithi164.stdout: inet6 fe80::ec4:7aff:fe88:73f3/64 scope link 2024-03-23T14:57:14.901 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-03-23T14:57:14.901 INFO:teuthology.orchestra.run.smithi164.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-03-23T14:57:14.901 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether 02:42:9e:85:90:15 brd ff:ff:ff:ff:ff:ff 2024-03-23T14:57:14.901 INFO:teuthology.orchestra.run.smithi164.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-03-23T14:57:14.901 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-03-23T14:57:14.901 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-23T14:57:14.902 DEBUG:teuthology.orchestra.run.smithi164:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-23T14:57:14.902 DEBUG:teuthology.orchestra.run.smithi164:> set -e 2024-03-23T14:57:14.902 DEBUG:teuthology.orchestra.run.smithi164:> sudo ip link add name ceph-brx type bridge 2024-03-23T14:57:14.902 DEBUG:teuthology.orchestra.run.smithi164:> sudo ip addr flush dev ceph-brx 2024-03-23T14:57:14.902 DEBUG:teuthology.orchestra.run.smithi164:> sudo ip link set ceph-brx up 2024-03-23T14:57:14.902 DEBUG:teuthology.orchestra.run.smithi164:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-23T14:57:14.902 DEBUG:teuthology.orchestra.run.smithi164:> ') 2024-03-23T14:57:14.986 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-23T14:57:15.022 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-23T14:57:15.029 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T14:57:15.029 DEBUG:teuthology.orchestra.run.smithi164:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-23T14:57:15.080 INFO:teuthology.orchestra.run.smithi164.stdout:1 2024-03-23T14:57:15.081 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T14:57:15.081 DEBUG:teuthology.orchestra.run.smithi164:> route 2024-03-23T14:57:15.159 INFO:teuthology.orchestra.run.smithi164.stdout:Kernel IP routing table 2024-03-23T14:57:15.159 INFO:teuthology.orchestra.run.smithi164.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-23T14:57:15.159 INFO:teuthology.orchestra.run.smithi164.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-03-23T14:57:15.159 INFO:teuthology.orchestra.run.smithi164.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-23T14:57:15.159 INFO:teuthology.orchestra.run.smithi164.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-03-23T14:57:15.159 INFO:teuthology.orchestra.run.smithi164.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-03-23T14:57:15.159 INFO:teuthology.orchestra.run.smithi164.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-23T14:57:15.159 INFO:teuthology.orchestra.run.smithi164.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-23T14:57:15.159 INFO:teuthology.orchestra.run.smithi164.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-23T14:57:15.159 INFO:teuthology.orchestra.run.smithi164.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-23T14:57:15.159 INFO:teuthology.orchestra.run.smithi164.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-23T14:57:15.160 DEBUG:teuthology.orchestra.run.smithi164:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-23T14:57:15.160 DEBUG:teuthology.orchestra.run.smithi164:> set -e 2024-03-23T14:57:15.160 DEBUG:teuthology.orchestra.run.smithi164:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-23T14:57:15.160 DEBUG:teuthology.orchestra.run.smithi164:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-23T14:57:15.160 DEBUG:teuthology.orchestra.run.smithi164:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-23T14:57:15.160 DEBUG:teuthology.orchestra.run.smithi164:> ') 2024-03-23T14:57:15.246 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:15 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-23T14:57:15.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-23T14:57:15.282 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T14:57:15.282 DEBUG:teuthology.orchestra.run.smithi164:> ip netns list 2024-03-23T14:57:15.329 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T14:57:15.329 DEBUG:teuthology.orchestra.run.smithi164:> ip netns list-id 2024-03-23T14:57:15.376 DEBUG:teuthology.orchestra.run.smithi164:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-23T14:57:15.376 DEBUG:teuthology.orchestra.run.smithi164:> set -e 2024-03-23T14:57:15.376 DEBUG:teuthology.orchestra.run.smithi164:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-23T14:57:15.377 DEBUG:teuthology.orchestra.run.smithi164:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-03-23T14:57:15.377 DEBUG:teuthology.orchestra.run.smithi164:> ') 2024-03-23T14:57:15.466 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:15 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-23T14:57:15.487 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-23T14:57:15.495 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-03-23T14:57:15.495 DEBUG:teuthology.orchestra.run.smithi164:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-23T14:57:15.495 DEBUG:teuthology.orchestra.run.smithi164:> set -e 2024-03-23T14:57:15.495 DEBUG:teuthology.orchestra.run.smithi164:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-03-23T14:57:15.495 DEBUG:teuthology.orchestra.run.smithi164:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-23T14:57:15.495 DEBUG:teuthology.orchestra.run.smithi164:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-03-23T14:57:15.495 DEBUG:teuthology.orchestra.run.smithi164:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-03-23T14:57:15.495 DEBUG:teuthology.orchestra.run.smithi164:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-03-23T14:57:15.495 DEBUG:teuthology.orchestra.run.smithi164:> ') 2024-03-23T14:57:15.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:15 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-23T14:57:15.592 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-23T14:57:15.599 DEBUG:teuthology.orchestra.run.smithi164:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-23T14:57:15.599 DEBUG:teuthology.orchestra.run.smithi164:> set -e 2024-03-23T14:57:15.599 DEBUG:teuthology.orchestra.run.smithi164:> sudo ip link set brx.0 up 2024-03-23T14:57:15.599 DEBUG:teuthology.orchestra.run.smithi164:> sudo ip link set dev brx.0 master ceph-brx 2024-03-23T14:57:15.599 DEBUG:teuthology.orchestra.run.smithi164:> ') 2024-03-23T14:57:15.637 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:15 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-23T14:57:15.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-23T14:57:15.659 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-03-23T14:57:15.659 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-23T14:57:15.659 DEBUG:teuthology.orchestra.run.smithi164:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-23T14:57:15.703 INFO:teuthology.orchestra.run.smithi164.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-23T14:57:15.704 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-23T14:57:15.704 DEBUG:teuthology.orchestra.run.smithi164:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-23T14:57:15.752 DEBUG:teuthology.orchestra.run.smithi164:> sudo modprobe fuse 2024-03-23T14:57:15.809 DEBUG:teuthology.orchestra.run.smithi164:> cat /proc/self/mounts | awk '{print $2}' 2024-03-23T14:57:15.857 INFO:teuthology.orchestra.run.smithi164.stdout:/sys 2024-03-23T14:57:15.857 INFO:teuthology.orchestra.run.smithi164.stdout:/proc 2024-03-23T14:57:15.857 INFO:teuthology.orchestra.run.smithi164.stdout:/dev 2024-03-23T14:57:15.858 INFO:teuthology.orchestra.run.smithi164.stdout:/dev/pts 2024-03-23T14:57:15.858 INFO:teuthology.orchestra.run.smithi164.stdout:/run 2024-03-23T14:57:15.858 INFO:teuthology.orchestra.run.smithi164.stdout:/ 2024-03-23T14:57:15.858 INFO:teuthology.orchestra.run.smithi164.stdout:/sys/kernel/security 2024-03-23T14:57:15.858 INFO:teuthology.orchestra.run.smithi164.stdout:/dev/shm 2024-03-23T14:57:15.858 INFO:teuthology.orchestra.run.smithi164.stdout:/run/lock 2024-03-23T14:57:15.858 INFO:teuthology.orchestra.run.smithi164.stdout:/sys/fs/cgroup 2024-03-23T14:57:15.858 INFO:teuthology.orchestra.run.smithi164.stdout:/sys/fs/pstore 2024-03-23T14:57:15.858 INFO:teuthology.orchestra.run.smithi164.stdout:/sys/fs/bpf 2024-03-23T14:57:15.858 INFO:teuthology.orchestra.run.smithi164.stdout:/proc/sys/fs/binfmt_misc 2024-03-23T14:57:15.858 INFO:teuthology.orchestra.run.smithi164.stdout:/dev/hugepages 2024-03-23T14:57:15.858 INFO:teuthology.orchestra.run.smithi164.stdout:/dev/mqueue 2024-03-23T14:57:15.858 INFO:teuthology.orchestra.run.smithi164.stdout:/sys/kernel/debug 2024-03-23T14:57:15.858 INFO:teuthology.orchestra.run.smithi164.stdout:/sys/kernel/tracing 2024-03-23T14:57:15.858 INFO:teuthology.orchestra.run.smithi164.stdout:/proc/fs/nfsd 2024-03-23T14:57:15.859 INFO:teuthology.orchestra.run.smithi164.stdout:/sys/fs/fuse/connections 2024-03-23T14:57:15.859 INFO:teuthology.orchestra.run.smithi164.stdout:/sys/kernel/config 2024-03-23T14:57:15.859 INFO:teuthology.orchestra.run.smithi164.stdout:/run/credentials/systemd-sysusers.service 2024-03-23T14:57:15.859 INFO:teuthology.orchestra.run.smithi164.stdout:/run/rpc_pipefs 2024-03-23T14:57:15.859 INFO:teuthology.orchestra.run.smithi164.stdout:/run/user/1000 2024-03-23T14:57:15.859 INFO:teuthology.orchestra.run.smithi164.stdout:/var/lib/ceph 2024-03-23T14:57:15.859 INFO:teuthology.orchestra.run.smithi164.stdout:/var/lib/ceph/osd/ceph-0 2024-03-23T14:57:15.859 INFO:teuthology.orchestra.run.smithi164.stdout:/var/lib/ceph/osd/ceph-1 2024-03-23T14:57:15.859 INFO:teuthology.orchestra.run.smithi164.stdout:/var/lib/ceph/osd/ceph-2 2024-03-23T14:57:15.859 INFO:teuthology.orchestra.run.smithi164.stdout:/var/lib/ceph/osd/ceph-3 2024-03-23T14:57:15.859 INFO:teuthology.orchestra.run.smithi164.stdout:/run/netns 2024-03-23T14:57:15.859 INFO:teuthology.orchestra.run.smithi164.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-23T14:57:15.859 INFO:teuthology.orchestra.run.smithi164.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-23T14:57:15.860 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T14:57:15.860 DEBUG:teuthology.orchestra.run.smithi164:> ls /sys/fs/fuse/connections 2024-03-23T14:57:15.908 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-23T14:57:15.908 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-03-23T14:57:15.953 DEBUG:teuthology.orchestra.run.smithi164:> sudo modprobe fuse 2024-03-23T14:57:15.966 DEBUG:teuthology.orchestra.run.smithi164:> cat /proc/self/mounts | awk '{print $2}' 2024-03-23T14:57:16.017 INFO:teuthology.orchestra.run.smithi164.stdout:/sys 2024-03-23T14:57:16.017 INFO:teuthology.orchestra.run.smithi164.stdout:/proc 2024-03-23T14:57:16.017 INFO:teuthology.orchestra.run.smithi164.stdout:/dev 2024-03-23T14:57:16.017 INFO:teuthology.orchestra.run.smithi164.stdout:/dev/pts 2024-03-23T14:57:16.017 INFO:teuthology.orchestra.run.smithi164.stdout:/run 2024-03-23T14:57:16.017 INFO:teuthology.orchestra.run.smithi164.stdout:/ 2024-03-23T14:57:16.017 INFO:teuthology.orchestra.run.smithi164.stdout:/sys/kernel/security 2024-03-23T14:57:16.017 INFO:teuthology.orchestra.run.smithi164.stdout:/dev/shm 2024-03-23T14:57:16.018 INFO:teuthology.orchestra.run.smithi164.stdout:/run/lock 2024-03-23T14:57:16.018 INFO:teuthology.orchestra.run.smithi164.stdout:/sys/fs/cgroup 2024-03-23T14:57:16.018 INFO:teuthology.orchestra.run.smithi164.stdout:/sys/fs/pstore 2024-03-23T14:57:16.018 INFO:teuthology.orchestra.run.smithi164.stdout:/sys/fs/bpf 2024-03-23T14:57:16.018 INFO:teuthology.orchestra.run.smithi164.stdout:/proc/sys/fs/binfmt_misc 2024-03-23T14:57:16.018 INFO:teuthology.orchestra.run.smithi164.stdout:/dev/hugepages 2024-03-23T14:57:16.018 INFO:teuthology.orchestra.run.smithi164.stdout:/dev/mqueue 2024-03-23T14:57:16.018 INFO:teuthology.orchestra.run.smithi164.stdout:/sys/kernel/debug 2024-03-23T14:57:16.018 INFO:teuthology.orchestra.run.smithi164.stdout:/sys/kernel/tracing 2024-03-23T14:57:16.018 INFO:teuthology.orchestra.run.smithi164.stdout:/proc/fs/nfsd 2024-03-23T14:57:16.018 INFO:teuthology.orchestra.run.smithi164.stdout:/sys/fs/fuse/connections 2024-03-23T14:57:16.018 INFO:teuthology.orchestra.run.smithi164.stdout:/sys/kernel/config 2024-03-23T14:57:16.018 INFO:teuthology.orchestra.run.smithi164.stdout:/run/credentials/systemd-sysusers.service 2024-03-23T14:57:16.018 INFO:teuthology.orchestra.run.smithi164.stdout:/run/rpc_pipefs 2024-03-23T14:57:16.018 INFO:teuthology.orchestra.run.smithi164.stdout:/run/user/1000 2024-03-23T14:57:16.019 INFO:teuthology.orchestra.run.smithi164.stdout:/var/lib/ceph 2024-03-23T14:57:16.019 INFO:teuthology.orchestra.run.smithi164.stdout:/var/lib/ceph/osd/ceph-0 2024-03-23T14:57:16.019 INFO:teuthology.orchestra.run.smithi164.stdout:/var/lib/ceph/osd/ceph-1 2024-03-23T14:57:16.019 INFO:teuthology.orchestra.run.smithi164.stdout:/var/lib/ceph/osd/ceph-2 2024-03-23T14:57:16.019 INFO:teuthology.orchestra.run.smithi164.stdout:/var/lib/ceph/osd/ceph-3 2024-03-23T14:57:16.019 INFO:teuthology.orchestra.run.smithi164.stdout:/run/netns 2024-03-23T14:57:16.019 INFO:teuthology.orchestra.run.smithi164.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-23T14:57:16.019 INFO:teuthology.orchestra.run.smithi164.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-23T14:57:16.019 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T14:57:16.019 DEBUG:teuthology.orchestra.run.smithi164:> ls /sys/fs/fuse/connections 2024-03-23T14:57:16.039 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi164.stderr:2024-03-23T14:57:16.033+0000 7f769c5df580 -1 init, newargv = 0x56094e7689d0 newargc=15 2024-03-23T14:57:16.039 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi164.stderr:ceph-fuse[34996]: starting ceph client 2024-03-23T14:57:16.046 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi164.stderr:ceph-fuse[34996]: starting fuse 2024-03-23T14:57:17.145 DEBUG:teuthology.orchestra.run.smithi164:> sudo modprobe fuse 2024-03-23T14:57:17.162 DEBUG:teuthology.orchestra.run.smithi164:> cat /proc/self/mounts | awk '{print $2}' 2024-03-23T14:57:17.212 INFO:teuthology.orchestra.run.smithi164.stdout:/sys 2024-03-23T14:57:17.212 INFO:teuthology.orchestra.run.smithi164.stdout:/proc 2024-03-23T14:57:17.212 INFO:teuthology.orchestra.run.smithi164.stdout:/dev 2024-03-23T14:57:17.212 INFO:teuthology.orchestra.run.smithi164.stdout:/dev/pts 2024-03-23T14:57:17.212 INFO:teuthology.orchestra.run.smithi164.stdout:/run 2024-03-23T14:57:17.212 INFO:teuthology.orchestra.run.smithi164.stdout:/ 2024-03-23T14:57:17.212 INFO:teuthology.orchestra.run.smithi164.stdout:/sys/kernel/security 2024-03-23T14:57:17.212 INFO:teuthology.orchestra.run.smithi164.stdout:/dev/shm 2024-03-23T14:57:17.212 INFO:teuthology.orchestra.run.smithi164.stdout:/run/lock 2024-03-23T14:57:17.212 INFO:teuthology.orchestra.run.smithi164.stdout:/sys/fs/cgroup 2024-03-23T14:57:17.213 INFO:teuthology.orchestra.run.smithi164.stdout:/sys/fs/pstore 2024-03-23T14:57:17.213 INFO:teuthology.orchestra.run.smithi164.stdout:/sys/fs/bpf 2024-03-23T14:57:17.213 INFO:teuthology.orchestra.run.smithi164.stdout:/proc/sys/fs/binfmt_misc 2024-03-23T14:57:17.213 INFO:teuthology.orchestra.run.smithi164.stdout:/dev/hugepages 2024-03-23T14:57:17.213 INFO:teuthology.orchestra.run.smithi164.stdout:/dev/mqueue 2024-03-23T14:57:17.213 INFO:teuthology.orchestra.run.smithi164.stdout:/sys/kernel/debug 2024-03-23T14:57:17.213 INFO:teuthology.orchestra.run.smithi164.stdout:/sys/kernel/tracing 2024-03-23T14:57:17.213 INFO:teuthology.orchestra.run.smithi164.stdout:/proc/fs/nfsd 2024-03-23T14:57:17.213 INFO:teuthology.orchestra.run.smithi164.stdout:/sys/fs/fuse/connections 2024-03-23T14:57:17.213 INFO:teuthology.orchestra.run.smithi164.stdout:/sys/kernel/config 2024-03-23T14:57:17.213 INFO:teuthology.orchestra.run.smithi164.stdout:/run/credentials/systemd-sysusers.service 2024-03-23T14:57:17.213 INFO:teuthology.orchestra.run.smithi164.stdout:/run/rpc_pipefs 2024-03-23T14:57:17.213 INFO:teuthology.orchestra.run.smithi164.stdout:/run/user/1000 2024-03-23T14:57:17.213 INFO:teuthology.orchestra.run.smithi164.stdout:/var/lib/ceph 2024-03-23T14:57:17.213 INFO:teuthology.orchestra.run.smithi164.stdout:/var/lib/ceph/osd/ceph-0 2024-03-23T14:57:17.213 INFO:teuthology.orchestra.run.smithi164.stdout:/var/lib/ceph/osd/ceph-1 2024-03-23T14:57:17.214 INFO:teuthology.orchestra.run.smithi164.stdout:/var/lib/ceph/osd/ceph-2 2024-03-23T14:57:17.214 INFO:teuthology.orchestra.run.smithi164.stdout:/var/lib/ceph/osd/ceph-3 2024-03-23T14:57:17.214 INFO:teuthology.orchestra.run.smithi164.stdout:/run/netns 2024-03-23T14:57:17.214 INFO:teuthology.orchestra.run.smithi164.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-23T14:57:17.214 INFO:teuthology.orchestra.run.smithi164.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-23T14:57:17.214 INFO:teuthology.orchestra.run.smithi164.stdout:/home/ubuntu/cephtest/mnt.0 2024-03-23T14:57:17.215 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T14:57:17.215 DEBUG:teuthology.orchestra.run.smithi164:> ls /sys/fs/fuse/connections 2024-03-23T14:57:17.261 INFO:teuthology.orchestra.run.smithi164.stdout:45 2024-03-23T14:57:17.261 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-03-23T14:57:17.261 DEBUG:teuthology.orchestra.run.smithi164:> sudo stdin-killer -- python3 -c ' 2024-03-23T14:57:17.261 DEBUG:teuthology.orchestra.run.smithi164:> import glob 2024-03-23T14:57:17.261 DEBUG:teuthology.orchestra.run.smithi164:> import re 2024-03-23T14:57:17.261 DEBUG:teuthology.orchestra.run.smithi164:> import os 2024-03-23T14:57:17.261 DEBUG:teuthology.orchestra.run.smithi164:> import subprocess 2024-03-23T14:57:17.261 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-23T14:57:17.261 DEBUG:teuthology.orchestra.run.smithi164:> def _find_admin_socket(client_name): 2024-03-23T14:57:17.261 DEBUG:teuthology.orchestra.run.smithi164:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-03-23T14:57:17.262 DEBUG:teuthology.orchestra.run.smithi164:> files = glob.glob(asok_path) 2024-03-23T14:57:17.262 DEBUG:teuthology.orchestra.run.smithi164:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-03-23T14:57:17.262 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-23T14:57:17.262 DEBUG:teuthology.orchestra.run.smithi164:> # Given a non-glob path, it better be there 2024-03-23T14:57:17.262 DEBUG:teuthology.orchestra.run.smithi164:> if "*" not in asok_path: 2024-03-23T14:57:17.262 DEBUG:teuthology.orchestra.run.smithi164:> assert(len(files) == 1) 2024-03-23T14:57:17.262 DEBUG:teuthology.orchestra.run.smithi164:> return files[0] 2024-03-23T14:57:17.262 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-23T14:57:17.262 DEBUG:teuthology.orchestra.run.smithi164:> for f in files: 2024-03-23T14:57:17.262 DEBUG:teuthology.orchestra.run.smithi164:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-23T14:57:17.262 DEBUG:teuthology.orchestra.run.smithi164:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-23T14:57:17.262 DEBUG:teuthology.orchestra.run.smithi164:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-23T14:57:17.262 DEBUG:teuthology.orchestra.run.smithi164:> contents = proc_f.read() 2024-03-23T14:57:17.262 DEBUG:teuthology.orchestra.run.smithi164:> if mountpoint in contents: 2024-03-23T14:57:17.262 DEBUG:teuthology.orchestra.run.smithi164:> return f 2024-03-23T14:57:17.262 DEBUG:teuthology.orchestra.run.smithi164:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-23T14:57:17.263 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-23T14:57:17.263 DEBUG:teuthology.orchestra.run.smithi164:> print(_find_admin_socket("client.0")) 2024-03-23T14:57:17.263 DEBUG:teuthology.orchestra.run.smithi164:> ' 2024-03-23T14:57:17.385 INFO:teuthology.orchestra.run.smithi164.stdout:/var/run/ceph/ceph-client.0.34996.asok 2024-03-23T14:57:17.389 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-23T14:57:17.396 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.34996.asok 2024-03-23T14:57:17.396 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T14:57:17.397 DEBUG:teuthology.orchestra.run.smithi164:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.34996.asok status 2024-03-23T14:57:17.552 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-03-23T14:57:17.552 INFO:teuthology.orchestra.run.smithi164.stdout: "metadata": { 2024-03-23T14:57:17.552 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_sha1": "21b3bd672163092b4e542f772797fcba02d3d4de", 2024-03-23T14:57:17.552 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_version": "ceph version 19.0.0-1855-g21b3bd67 (21b3bd672163092b4e542f772797fcba02d3d4de) squid (dev)", 2024-03-23T14:57:17.552 INFO:teuthology.orchestra.run.smithi164.stdout: "entity_id": "0", 2024-03-23T14:57:17.552 INFO:teuthology.orchestra.run.smithi164.stdout: "hostname": "smithi164", 2024-03-23T14:57:17.552 INFO:teuthology.orchestra.run.smithi164.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-03-23T14:57:17.552 INFO:teuthology.orchestra.run.smithi164.stdout: "pid": "34996", 2024-03-23T14:57:17.552 INFO:teuthology.orchestra.run.smithi164.stdout: "root": "/" 2024-03-23T14:57:17.552 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-03-23T14:57:17.553 INFO:teuthology.orchestra.run.smithi164.stdout: "dentry_count": 0, 2024-03-23T14:57:17.553 INFO:teuthology.orchestra.run.smithi164.stdout: "dentry_pinned_count": 0, 2024-03-23T14:57:17.553 INFO:teuthology.orchestra.run.smithi164.stdout: "id": 4626, 2024-03-23T14:57:17.553 INFO:teuthology.orchestra.run.smithi164.stdout: "inst": { 2024-03-23T14:57:17.553 INFO:teuthology.orchestra.run.smithi164.stdout: "name": { 2024-03-23T14:57:17.553 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "client", 2024-03-23T14:57:17.553 INFO:teuthology.orchestra.run.smithi164.stdout: "num": 4626 2024-03-23T14:57:17.553 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-03-23T14:57:17.553 INFO:teuthology.orchestra.run.smithi164.stdout: "addr": { 2024-03-23T14:57:17.553 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "v1", 2024-03-23T14:57:17.553 INFO:teuthology.orchestra.run.smithi164.stdout: "addr": "192.168.0.1:0", 2024-03-23T14:57:17.553 INFO:teuthology.orchestra.run.smithi164.stdout: "nonce": 537213035 2024-03-23T14:57:17.553 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-03-23T14:57:17.554 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-03-23T14:57:17.554 INFO:teuthology.orchestra.run.smithi164.stdout: "addr": { 2024-03-23T14:57:17.554 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "v1", 2024-03-23T14:57:17.554 INFO:teuthology.orchestra.run.smithi164.stdout: "addr": "192.168.0.1:0", 2024-03-23T14:57:17.554 INFO:teuthology.orchestra.run.smithi164.stdout: "nonce": 537213035 2024-03-23T14:57:17.554 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-03-23T14:57:17.554 INFO:teuthology.orchestra.run.smithi164.stdout: "inst_str": "client.4626 192.168.0.1:0/537213035", 2024-03-23T14:57:17.554 INFO:teuthology.orchestra.run.smithi164.stdout: "addr_str": "192.168.0.1:0/537213035", 2024-03-23T14:57:17.554 INFO:teuthology.orchestra.run.smithi164.stdout: "inode_count": 1, 2024-03-23T14:57:17.554 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_epoch": 6, 2024-03-23T14:57:17.554 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_epoch": 20, 2024-03-23T14:57:17.554 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_epoch_barrier": 0, 2024-03-23T14:57:17.555 INFO:teuthology.orchestra.run.smithi164.stdout: "blocklisted": false, 2024-03-23T14:57:17.555 INFO:teuthology.orchestra.run.smithi164.stdout: "fs_name": "cephfs" 2024-03-23T14:57:17.555 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-03-23T14:57:17.566 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T14:57:17.566 DEBUG:teuthology.orchestra.run.smithi164:> stat --file-system '--printf=%T 2024-03-23T14:57:17.566 DEBUG:teuthology.orchestra.run.smithi164:> ' -- /home/ubuntu/cephtest/mnt.0 2024-03-23T14:57:17.572 INFO:teuthology.orchestra.run.smithi164.stdout:fuseblk 2024-03-23T14:57:17.572 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-03-23T14:57:17.572 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T14:57:17.573 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-23T14:57:17.627 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-03-23T14:57:17.646 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T14:57:17.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:17.797+0000 7fad0beae640 1 Processor -- start 2024-03-23T14:57:17.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:17.797+0000 7fad0beae640 1 -- start start 2024-03-23T14:57:17.801 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:17.797+0000 7fad0beae640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fad040b6360 0x7fad04145a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:17.801 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:17.797+0000 7fad0beae640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fad04145f60 0x7fad0414fb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:17.801 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:17.797+0000 7fad0beae640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad040b5530 con 0x7fad040b6740 2024-03-23T14:57:17.801 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:17.797+0000 7fad0beae640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 -- 0x7fad040b5a60 con 0x7fad04145f60 2024-03-23T14:57:17.801 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:17.797+0000 7fad0beae640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fad04071260 con 0x7fad040b6360 2024-03-23T14:57:17.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:17.797+0000 7fad0a44b640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fad04145f60 0x7fad0414fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:17.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:17.797+0000 7fad09449640 1 --1- >> v1:172.21.15.173:6789/0 conn(0x7fad040b6740 0x7fad040b41d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.173:6789/0 says I am v1:172.21.15.164:50530/0 (socket says 172.21.15.164:50530) 2024-03-23T14:57:17.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:17.797+0000 7fad09449640 1 -- 172.21.15.164:0/3340787447 learned_addr learned my addr 172.21.15.164:0/3340787447 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-03-23T14:57:17.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:17.797+0000 7fad09c4a640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fad040b6360 0x7fad04145a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:17.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:17.797+0000 7fad0a44b640 1 -- 172.21.15.164:0/3340787447 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fad040b6360 msgr2=0x7fad04145a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:17.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:17.797+0000 7fad0a44b640 1 --2- 172.21.15.164:0/3340787447 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fad040b6360 0x7fad04145a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:17.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:17.797+0000 7fad0a44b640 1 -- 172.21.15.164:0/3340787447 >> v1:172.21.15.173:6789/0 conn(0x7fad040b6740 legacy=0x7fad040b41d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:17.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:17.797+0000 7fad0a44b640 1 -- 172.21.15.164:0/3340787447 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad040b2a70 con 0x7fad04145f60 2024-03-23T14:57:17.804 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:17.801+0000 7fad0a44b640 1 --2- 172.21.15.164:0/3340787447 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fad04145f60 0x7fad0414fb90 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7facfc009d90 tx=0x7facfc01c8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f6fd8feb2604d25 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:17.805 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:17.801+0000 7fad08c48640 1 -- 172.21.15.164:0/3340787447 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facfc0a57a0 con 0x7fad04145f60 2024-03-23T14:57:17.805 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:17.801+0000 7fad08c48640 1 -- 172.21.15.164:0/3340787447 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facfc0a5900 con 0x7fad04145f60 2024-03-23T14:57:17.805 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:17.801+0000 7fad08c48640 1 -- 172.21.15.164:0/3340787447 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facfc0a5c90 con 0x7fad04145f60 2024-03-23T14:57:17.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:17.801+0000 7fad0beae640 1 -- 172.21.15.164:0/3340787447 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fad04145f60 msgr2=0x7fad0414fb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:17.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:17.801+0000 7fad0beae640 1 --2- 172.21.15.164:0/3340787447 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fad04145f60 0x7fad0414fb90 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7facfc009d90 tx=0x7facfc01c8e0 comp rx=0 tx=0).stop 2024-03-23T14:57:17.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:17.801+0000 7fad0beae640 1 -- 172.21.15.164:0/3340787447 shutdown_connections 2024-03-23T14:57:17.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:17.801+0000 7fad0beae640 1 --2- 172.21.15.164:0/3340787447 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fad04145f60 0x7fad0414fb90 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:17.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:17.801+0000 7fad0beae640 1 --2- 172.21.15.164:0/3340787447 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fad040b6360 0x7fad04145a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:17.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:17.801+0000 7fad0beae640 1 -- 172.21.15.164:0/3340787447 >> 172.21.15.164:0/3340787447 conn(0x7fad040b7630 msgr2=0x7fad040b7a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:17.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:17.801+0000 7fad0beae640 1 -- 172.21.15.164:0/3340787447 shutdown_connections 2024-03-23T14:57:17.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:17.801+0000 7fad0beae640 1 -- 172.21.15.164:0/3340787447 wait complete. 2024-03-23T14:57:17.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:17.801+0000 7fad0beae640 1 Processor -- start 2024-03-23T14:57:17.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:17.801+0000 7fad0beae640 1 -- start start 2024-03-23T14:57:17.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:17.801+0000 7fad0beae640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fad040b6360 0x7fad0413ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:17.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:17.801+0000 7fad0beae640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fad040b6740 0x7fad0413d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:17.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:17.801+0000 7fad0beae640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fad04145f60 0x7fad041e14b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:17.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:17.801+0000 7fad0beae640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fad040aba70 con 0x7fad04145f60 2024-03-23T14:57:17.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:17.801+0000 7fad0beae640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fad040ae3b0 con 0x7fad040b6740 2024-03-23T14:57:17.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:17.801+0000 7fad0beae640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fad040ac2a0 con 0x7fad040b6360 2024-03-23T14:57:17.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:17.805+0000 7fad0a44b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fad04145f60 0x7fad041e14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:17.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:17.805+0000 7fad0a44b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fad04145f60 0x7fad041e14b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:60248/0 (socket says 172.21.15.164:60248) 2024-03-23T14:57:17.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:17.805+0000 7fad0a44b640 1 -- 172.21.15.164:0/2642940154 learned_addr learned my addr 172.21.15.164:0/2642940154 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:17.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:17.805+0000 7fad09c4a640 1 --2- 172.21.15.164:0/2642940154 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fad040b6360 0x7fad0413ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:17.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:17.805+0000 7fad09449640 1 --2- 172.21.15.164:0/2642940154 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fad040b6740 0x7fad0413d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:17.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:17.805+0000 7fad0a44b640 1 -- 172.21.15.164:0/2642940154 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fad040b6360 msgr2=0x7fad0413ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:17.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:17.805+0000 7fad0a44b640 1 --2- 172.21.15.164:0/2642940154 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fad040b6360 0x7fad0413ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:17.810 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:17.805+0000 7fad0a44b640 1 -- 172.21.15.164:0/2642940154 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fad040b6740 msgr2=0x7fad0413d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:17.810 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:17.805+0000 7fad0a44b640 1 --2- 172.21.15.164:0/2642940154 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fad040b6740 0x7fad0413d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:17.810 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:17.805+0000 7fad0a44b640 1 -- 172.21.15.164:0/2642940154 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facfc009a70 con 0x7fad04145f60 2024-03-23T14:57:17.810 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:17.805+0000 7fad0a44b640 1 --2- 172.21.15.164:0/2642940154 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fad04145f60 0x7fad041e14b0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7facfc009d90 tx=0x7facfc019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:17.810 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:17.805+0000 7facfa7fc640 1 -- 172.21.15.164:0/2642940154 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facfc0ad640 con 0x7fad04145f60 2024-03-23T14:57:17.810 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:17.805+0000 7facfa7fc640 1 -- 172.21.15.164:0/2642940154 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facfc0ad7a0 con 0x7fad04145f60 2024-03-23T14:57:17.810 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:17.805+0000 7facfa7fc640 1 -- 172.21.15.164:0/2642940154 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facfc0adb30 con 0x7fad04145f60 2024-03-23T14:57:17.810 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:17.805+0000 7fad0beae640 1 -- 172.21.15.164:0/2642940154 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad040ae900 con 0x7fad04145f60 2024-03-23T14:57:17.810 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:17.805+0000 7fad0beae640 1 -- 172.21.15.164:0/2642940154 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad040ae620 con 0x7fad04145f60 2024-03-23T14:57:17.811 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:17.805+0000 7fad0beae640 1 -- 172.21.15.164:0/2642940154 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad040690a0 con 0x7fad04145f60 2024-03-23T14:57:17.811 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:17.805+0000 7facfa7fc640 1 -- 172.21.15.164:0/2642940154 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7facfc0b4070 con 0x7fad04145f60 2024-03-23T14:57:17.811 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:17.805+0000 7facfa7fc640 1 --2- 172.21.15.164:0/2642940154 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7face80672a0 0x7face8069760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:17.811 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:17.805+0000 7fad09c4a640 1 --2- 172.21.15.164:0/2642940154 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7face80672a0 0x7face8069760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:17.811 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:17.805+0000 7facfa7fc640 1 -- 172.21.15.164:0/2642940154 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7facfc11fd30 con 0x7fad04145f60 2024-03-23T14:57:17.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:17.805+0000 7fad09c4a640 1 --2- 172.21.15.164:0/2642940154 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7face80672a0 0x7face8069760 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fad00009ac0 tx=0x7fad00009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:17.815 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:17.809+0000 7facfa7fc640 1 -- 172.21.15.164:0/2642940154 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7facfc0eca20 con 0x7fad04145f60 2024-03-23T14:57:17.974 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:17.969+0000 7fad0beae640 1 -- 172.21.15.164:0/2642940154 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fad040724a0 con 0x7fad04145f60 2024-03-23T14:57:17.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:17.973+0000 7facfa7fc640 1 -- 172.21.15.164:0/2642940154 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16629 (secure 0 0 0) 0x7facfc0b2090 con 0x7fad04145f60 2024-03-23T14:57:17.977 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-23T14:57:17.977 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":20,"fsid":"45c7911b-b69d-4e68-907b-fbfeb780aa3b","created":"2024-03-23T14:56:34.367633+0000","modified":"2024-03-23T14:57:00.611265+0000","last_up_change":"2024-03-23T14:56:46.297403+0000","last_in_change":"2024-03-23T14:56:36.629884+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T14:56:47.186157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-23T14:56:56.994057+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-23T14:56:57.953862+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"80abdc62-bd0d-4e60-a554-7ed6fd67cf54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":984716690},{"type":"v1","addr":"172.21.15.164:6801","nonce":984716690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":984716690},{"type":"v1","addr":"172.21.15.164:6803","nonce":984716690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":984716690},{"type":"v1","addr":"172.21.15.164:6807","nonce":984716690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":984716690},{"type":"v1","addr":"172.21.15.164:6805","nonce":984716690}]},"public_addr":"172.21.15.164:6801/984716690","cluster_addr":"172.21.15.164:6803/984716690","heartbeat_back_addr":"172.21.15.164:6807/984716690","heartbeat_front_addr":"172.21.15.164:6805/984716690","state":["exists","up"]},{"osd":1,"uuid":"f9001b06-be4d-4f42-a0e6-bcef2afa0d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":1349657239},{"type":"v1","addr":"172.21.15.164:6817","nonce":1349657239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":1349657239},{"type":"v1","addr":"172.21.15.164:6819","nonce":1349657239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":1349657239},{"type":"v1","addr":"172.21.15.164:6823","nonce":1349657239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":1349657239},{"type":"v1","addr":"172.21.15.164:6821","nonce":1349657239}]},"public_addr":"172.21.15.164:6817/1349657239","cluster_addr":"172.21.15.164:6819/1349657239","heartbeat_back_addr":"172.21.15.164:6823/1349657239","heartbeat_front_addr":"172.21.15.164:6821/1349657239","state":["exists","up"]},{"osd":2,"uuid":"fa0aeb9f-8b54-4831-a188-3e361c7980d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":887861252},{"type":"v1","addr":"172.21.15.164:6809","nonce":887861252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":887861252},{"type":"v1","addr":"172.21.15.164:6811","nonce":887861252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":887861252},{"type":"v1","addr":"172.21.15.164:6815","nonce":887861252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":887861252},{"type":"v1","addr":"172.21.15.164:6813","nonce":887861252}]},"public_addr":"172.21.15.164:6809/887861252","cluster_addr":"172.21.15.164:6811/887861252","heartbeat_back_addr":"172.21.15.164:6815/887861252","heartbeat_front_addr":"172.21.15.164:6813/887861252","state":["exists","up"]},{"osd":3,"uuid":"7e18c957-3091-46d5-8e17-8f3354c1726c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":812540271},{"type":"v1","addr":"172.21.15.164:6825","nonce":812540271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":812540271},{"type":"v1","addr":"172.21.15.164:6827","nonce":812540271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":812540271},{"type":"v1","addr":"172.21.15.164:6831","nonce":812540271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":812540271},{"type":"v1","addr":"172.21.15.164:6829","nonce":812540271}]},"public_addr":"172.21.15.164:6825/812540271","cluster_addr":"172.21.15.164:6827/812540271","heartbeat_back_addr":"172.21.15.164:6831/812540271","heartbeat_front_addr":"172.21.15.164:6829/812540271","state":["exists","up"]},{"osd":4,"uuid":"8e83f315-42bf-4471-9ce8-d81948da0f9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":2656857421},{"type":"v1","addr":"172.21.15.173:6825","nonce":2656857421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2656857421},{"type":"v1","addr":"172.21.15.173:6827","nonce":2656857421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2656857421},{"type":"v1","addr":"172.21.15.173:6831","nonce":2656857421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":2656857421},{"type":"v1","addr":"172.21.15.173:6829","nonce":2656857421}]},"public_addr":"172.21.15.173:6825/2656857421","cluster_addr":"172.21.15.173:6827/2656857421","heartbeat_back_addr":"172.21.15.173:6831/2656857421","heartbeat_front_addr":"172.21.15.173:6829/2656857421","state":["exists","up"]},{"osd":5,"uuid":"a230c872-71f9-40ab-8999-defd0162d4ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":3676483185},{"type":"v1","addr":"172.21.15.173:6801","nonce":3676483185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":3676483185},{"type":"v1","addr":"172.21.15.173:6803","nonce":3676483185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":3676483185},{"type":"v1","addr":"172.21.15.173:6807","nonce":3676483185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":3676483185},{"type":"v1","addr":"172.21.15.173:6805","nonce":3676483185}]},"public_addr":"172.21.15.173:6801/3676483185","cluster_addr":"172.21.15.173:6803/3676483185","heartbeat_back_addr":"172.21.15.173:6807/3676483185","heartbeat_front_addr":"172.21.15.173:6805/3676483185","state":["exists","up"]},{"osd":6,"uuid":"6193bfd7-8432-4f7f-a117-a3fcee21af99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":3631684087},{"type":"v1","addr":"172.21.15.173:6809","nonce":3631684087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":3631684087},{"type":"v1","addr":"172.21.15.173:6811","nonce":3631684087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":3631684087},{"type":"v1","addr":"172.21.15.173:6815","nonce":3631684087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":3631684087},{"type":"v1","addr":"172.21.15.173:6813","nonce":3631684087}]},"public_addr":"172.21.15.173:6809/3631684087","cluster_addr":"172.21.15.173:6811/3631684087","heartbeat_back_addr":"172.21.15.173:6815/3631684087","heartbeat_front_addr":"172.21.15.173:6813/3631684087","state":["exists","up"]},{"osd":7,"uuid":"b0f1b80a-afc4-4538-83b9-f081d2dcb8cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":3261114948},{"type":"v1","addr":"172.21.15.173:6817","nonce":3261114948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":3261114948},{"type":"v1","addr":"172.21.15.173:6819","nonce":3261114948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":3261114948},{"type":"v1","addr":"172.21.15.173:6823","nonce":3261114948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":3261114948},{"type":"v1","addr":"172.21.15.173:6821","nonce":3261114948}]},"public_addr":"172.21.15.173:6817/3261114948","cluster_addr":"172.21.15.173:6819/3261114948","heartbeat_back_addr":"172.21.15.173:6823/3261114948","heartbeat_front_addr":"172.21.15.173:6821/3261114948","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.188390+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.159932+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.191101+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.181776+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.224365+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.177515+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.153631+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.139972+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T14:57:17.980 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:17.973+0000 7fad0beae640 1 -- 172.21.15.164:0/2642940154 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7face80672a0 msgr2=0x7face8069760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:17.980 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:17.973+0000 7fad0beae640 1 --2- 172.21.15.164:0/2642940154 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7face80672a0 0x7face8069760 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fad00009ac0 tx=0x7fad00009190 comp rx=0 tx=0).stop 2024-03-23T14:57:17.980 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:17.973+0000 7fad0beae640 1 -- 172.21.15.164:0/2642940154 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fad04145f60 msgr2=0x7fad041e14b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:17.980 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:17.973+0000 7fad0beae640 1 --2- 172.21.15.164:0/2642940154 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fad04145f60 0x7fad041e14b0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7facfc009d90 tx=0x7facfc019680 comp rx=0 tx=0).stop 2024-03-23T14:57:17.980 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:17.973+0000 7fad0beae640 1 -- 172.21.15.164:0/2642940154 shutdown_connections 2024-03-23T14:57:17.980 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:17.973+0000 7fad0beae640 1 --2- 172.21.15.164:0/2642940154 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fad04145f60 0x7fad041e14b0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:17.980 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:17.973+0000 7fad0beae640 1 --2- 172.21.15.164:0/2642940154 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7face80672a0 0x7face8069760 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:17.980 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:17.973+0000 7fad0beae640 1 --2- 172.21.15.164:0/2642940154 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fad040b6740 0x7fad0413d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:17.980 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:17.973+0000 7fad0beae640 1 --2- 172.21.15.164:0/2642940154 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fad040b6360 0x7fad0413ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:17.980 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:17.973+0000 7fad0beae640 1 -- 172.21.15.164:0/2642940154 >> 172.21.15.164:0/2642940154 conn(0x7fad040b7630 msgr2=0x7fad04071d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:17.980 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:17.977+0000 7fad0beae640 1 -- 172.21.15.164:0/2642940154 shutdown_connections 2024-03-23T14:57:17.981 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:17.977+0000 7fad0beae640 1 -- 172.21.15.164:0/2642940154 wait complete. 2024-03-23T14:57:17.997 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T14:56:47.186157+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-23T14:56:56.994057+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-23T14:56:57.953862+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-23T14:57:17.997 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-23T14:57:18.124 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.121+0000 7fd3db08e640 1 Processor -- start 2024-03-23T14:57:18.124 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.121+0000 7fd3db08e640 1 -- start start 2024-03-23T14:57:18.125 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.121+0000 7fd3db08e640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fd3d409def0 0x7fd3d40b45e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:18.125 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.121+0000 7fd3db08e640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd3d40add50 0x7fd3d406bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:18.125 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.121+0000 7fd3db08e640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3d406d650 con 0x7fd3d40b4b20 2024-03-23T14:57:18.125 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.121+0000 7fd3db08e640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fd3d40af4a0 con 0x7fd3d409def0 2024-03-23T14:57:18.125 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.121+0000 7fd3db08e640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fd3d409e2d0 con 0x7fd3d40add50 2024-03-23T14:57:18.125 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.121+0000 7fd3d8e2a640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fd3d409def0 0x7fd3d40b45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:18.125 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.121+0000 7fd3d8e2a640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fd3d409def0 0x7fd3d40b45e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:60258/0 (socket says 172.21.15.164:60258) 2024-03-23T14:57:18.126 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.121+0000 7fd3d8e2a640 1 -- 172.21.15.164:0/2913175799 learned_addr learned my addr 172.21.15.164:0/2913175799 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:18.126 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.121+0000 7fd3d3fff640 1 --2- 172.21.15.164:0/2913175799 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd3d40add50 0x7fd3d406bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:18.126 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.121+0000 7fd3d8e2a640 1 -- 172.21.15.164:0/2913175799 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd3d40add50 msgr2=0x7fd3d406bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:18.126 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.121+0000 7fd3d8e2a640 1 --2- 172.21.15.164:0/2913175799 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd3d40add50 0x7fd3d406bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:18.126 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.121+0000 7fd3d8e2a640 1 -- 172.21.15.164:0/2913175799 >> v1:172.21.15.173:6789/0 conn(0x7fd3d40b4b20 legacy=0x7fd3d406c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:18.126 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.121+0000 7fd3d8e2a640 1 -- 172.21.15.164:0/2913175799 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3d40aeca0 con 0x7fd3d409def0 2024-03-23T14:57:18.127 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.121+0000 7fd3d8e2a640 1 --2- 172.21.15.164:0/2913175799 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd3d409def0 0x7fd3d40b45e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd3c4009e50 tx=0x7fd3c401c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12ee9f3a147509d6 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:18.127 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.121+0000 7fd3d37fe640 1 -- 172.21.15.164:0/2913175799 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3c40a5730 con 0x7fd3d409def0 2024-03-23T14:57:18.128 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.121+0000 7fd3d37fe640 1 -- 172.21.15.164:0/2913175799 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3c40a5890 con 0x7fd3d409def0 2024-03-23T14:57:18.128 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.121+0000 7fd3d37fe640 1 -- 172.21.15.164:0/2913175799 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3c40a5c20 con 0x7fd3d409def0 2024-03-23T14:57:18.128 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.125+0000 7fd3db08e640 1 -- 172.21.15.164:0/2913175799 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd3d409def0 msgr2=0x7fd3d40b45e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:18.128 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.125+0000 7fd3db08e640 1 --2- 172.21.15.164:0/2913175799 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd3d409def0 0x7fd3d40b45e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd3c4009e50 tx=0x7fd3c401c910 comp rx=0 tx=0).stop 2024-03-23T14:57:18.128 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.125+0000 7fd3db08e640 1 -- 172.21.15.164:0/2913175799 shutdown_connections 2024-03-23T14:57:18.129 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.125+0000 7fd3db08e640 1 --2- 172.21.15.164:0/2913175799 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd3d40add50 0x7fd3d406bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:18.129 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.125+0000 7fd3db08e640 1 --2- 172.21.15.164:0/2913175799 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd3d409def0 0x7fd3d40b45e0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:18.129 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.125+0000 7fd3db08e640 1 -- 172.21.15.164:0/2913175799 >> 172.21.15.164:0/2913175799 conn(0x7fd3d40b8ca0 msgr2=0x7fd3d40b90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:18.129 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.125+0000 7fd3db08e640 1 -- 172.21.15.164:0/2913175799 shutdown_connections 2024-03-23T14:57:18.129 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.125+0000 7fd3db08e640 1 -- 172.21.15.164:0/2913175799 wait complete. 2024-03-23T14:57:18.129 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.125+0000 7fd3db08e640 1 Processor -- start 2024-03-23T14:57:18.129 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.125+0000 7fd3db08e640 1 -- start start 2024-03-23T14:57:18.131 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.125+0000 7fd3db08e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd3d409def0 0x7fd3d41e0f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:18.131 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.125+0000 7fd3db08e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd3d40add50 0x7fd3d413c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:18.131 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.125+0000 7fd3db08e640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd3d40b4b20 0x7fd3d413cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:18.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.125+0000 7fd3db08e640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd3d4070ea0 con 0x7fd3d409def0 2024-03-23T14:57:18.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.125+0000 7fd3db08e640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd3d40709b0 con 0x7fd3d40add50 2024-03-23T14:57:18.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.125+0000 7fd3db08e640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fd3d413ebc0 con 0x7fd3d40b4b20 2024-03-23T14:57:18.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.125+0000 7fd3d8e2a640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd3d409def0 0x7fd3d41e0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:18.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.125+0000 7fd3d8e2a640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd3d409def0 0x7fd3d41e0f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:60270/0 (socket says 172.21.15.164:60270) 2024-03-23T14:57:18.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.125+0000 7fd3d8e2a640 1 -- 172.21.15.164:0/3657073848 learned_addr learned my addr 172.21.15.164:0/3657073848 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:18.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.125+0000 7fd3d962b640 1 --2- 172.21.15.164:0/3657073848 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd3d40b4b20 0x7fd3d413cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:18.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.125+0000 7fd3d3fff640 1 --2- 172.21.15.164:0/3657073848 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd3d40add50 0x7fd3d413c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:18.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.125+0000 7fd3d8e2a640 1 -- 172.21.15.164:0/3657073848 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd3d40b4b20 msgr2=0x7fd3d413cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:18.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.125+0000 7fd3d8e2a640 1 --2- 172.21.15.164:0/3657073848 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd3d40b4b20 0x7fd3d413cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:18.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.125+0000 7fd3d8e2a640 1 -- 172.21.15.164:0/3657073848 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd3d40add50 msgr2=0x7fd3d413c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:18.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.125+0000 7fd3d8e2a640 1 --2- 172.21.15.164:0/3657073848 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd3d40add50 0x7fd3d413c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:18.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.125+0000 7fd3d8e2a640 1 -- 172.21.15.164:0/3657073848 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3c4009b30 con 0x7fd3d409def0 2024-03-23T14:57:18.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.125+0000 7fd3d962b640 1 --2- 172.21.15.164:0/3657073848 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd3d40b4b20 0x7fd3d413cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T14:57:18.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.125+0000 7fd3d8e2a640 1 --2- 172.21.15.164:0/3657073848 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd3d409def0 0x7fd3d41e0f80 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd3c401a540 tx=0x7fd3c40196b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:18.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.125+0000 7fd3d17fa640 1 -- 172.21.15.164:0/3657073848 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3c40b8670 con 0x7fd3d409def0 2024-03-23T14:57:18.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.125+0000 7fd3d17fa640 1 -- 172.21.15.164:0/3657073848 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3c40b87d0 con 0x7fd3d409def0 2024-03-23T14:57:18.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.125+0000 7fd3d17fa640 1 -- 172.21.15.164:0/3657073848 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3c40b8b80 con 0x7fd3d409def0 2024-03-23T14:57:18.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.125+0000 7fd3db08e640 1 -- 172.21.15.164:0/3657073848 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3d413eea0 con 0x7fd3d409def0 2024-03-23T14:57:18.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.125+0000 7fd3db08e640 1 -- 172.21.15.164:0/3657073848 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3d409d6e0 con 0x7fd3d409def0 2024-03-23T14:57:18.133 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.129+0000 7fd3db08e640 1 -- 172.21.15.164:0/3657073848 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3d40690a0 con 0x7fd3d409def0 2024-03-23T14:57:18.134 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.129+0000 7fd3d17fa640 1 -- 172.21.15.164:0/3657073848 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7fd3c40b8ce0 con 0x7fd3d409def0 2024-03-23T14:57:18.134 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.129+0000 7fd3d17fa640 1 --2- 172.21.15.164:0/3657073848 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fd3c00672a0 0x7fd3c0069760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:18.134 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.129+0000 7fd3d3fff640 1 --2- 172.21.15.164:0/3657073848 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fd3c00672a0 0x7fd3c0069760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:18.134 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.129+0000 7fd3d17fa640 1 -- 172.21.15.164:0/3657073848 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fd3c41204c0 con 0x7fd3d409def0 2024-03-23T14:57:18.134 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.129+0000 7fd3d3fff640 1 --2- 172.21.15.164:0/3657073848 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fd3c00672a0 0x7fd3c0069760 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fd3d41e1a60 tx=0x7fd3c8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:18.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.133+0000 7fd3d17fa640 1 -- 172.21.15.164:0/3657073848 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7fd3c40ed060 con 0x7fd3d409def0 2024-03-23T14:57:18.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.293+0000 7fd3db08e640 1 -- 172.21.15.164:0/3657073848 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd3d4146910 con 0x7fd3d409def0 2024-03-23T14:57:18.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.293+0000 7fd3d17fa640 1 -- 172.21.15.164:0/3657073848 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7fd3c40f0720 con 0x7fd3d409def0 2024-03-23T14:57:18.298 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 1 2024-03-23T14:57:18.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.297+0000 7fd3db08e640 1 -- 172.21.15.164:0/3657073848 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fd3c00672a0 msgr2=0x7fd3c0069760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:18.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.297+0000 7fd3db08e640 1 --2- 172.21.15.164:0/3657073848 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fd3c00672a0 0x7fd3c0069760 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fd3d41e1a60 tx=0x7fd3c8008040 comp rx=0 tx=0).stop 2024-03-23T14:57:18.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.297+0000 7fd3db08e640 1 -- 172.21.15.164:0/3657073848 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd3d409def0 msgr2=0x7fd3d41e0f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:18.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.297+0000 7fd3db08e640 1 --2- 172.21.15.164:0/3657073848 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd3d409def0 0x7fd3d41e0f80 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd3c401a540 tx=0x7fd3c40196b0 comp rx=0 tx=0).stop 2024-03-23T14:57:18.301 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.297+0000 7fd3db08e640 1 -- 172.21.15.164:0/3657073848 shutdown_connections 2024-03-23T14:57:18.301 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.297+0000 7fd3db08e640 1 --2- 172.21.15.164:0/3657073848 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd3d40b4b20 0x7fd3d413cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:18.301 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.297+0000 7fd3db08e640 1 --2- 172.21.15.164:0/3657073848 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fd3c00672a0 0x7fd3c0069760 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:18.301 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.297+0000 7fd3db08e640 1 --2- 172.21.15.164:0/3657073848 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd3d40add50 0x7fd3d413c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:18.301 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.297+0000 7fd3db08e640 1 --2- 172.21.15.164:0/3657073848 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd3d409def0 0x7fd3d41e0f80 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:18.301 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.297+0000 7fd3db08e640 1 -- 172.21.15.164:0/3657073848 >> 172.21.15.164:0/3657073848 conn(0x7fd3d40b8ca0 msgr2=0x7fd3d4147110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:18.301 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.297+0000 7fd3db08e640 1 -- 172.21.15.164:0/3657073848 shutdown_connections 2024-03-23T14:57:18.301 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.297+0000 7fd3db08e640 1 -- 172.21.15.164:0/3657073848 wait complete. 2024-03-23T14:57:18.316 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-23T14:57:18.435 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.429+0000 7fc59474e640 1 Processor -- start 2024-03-23T14:57:18.435 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.429+0000 7fc59474e640 1 -- start start 2024-03-23T14:57:18.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.433+0000 7fc59474e640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fc58c0b4a70 0x7fc58c140f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:18.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.433+0000 7fc59474e640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc58c141470 0x7fc58c151c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:18.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.433+0000 7fc59474e640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc58c0b4300 con 0x7fc58c0b56f0 2024-03-23T14:57:18.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.433+0000 7fc59474e640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 -- 0x7fc58c0b5fa0 con 0x7fc58c0b4a70 2024-03-23T14:57:18.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.433+0000 7fc59474e640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fc58c0b45e0 con 0x7fc58c141470 2024-03-23T14:57:18.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.433+0000 7fc5924ea640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fc58c0b56f0 0x7fc58c14a8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:56878/0 (socket says 172.21.15.164:56878) 2024-03-23T14:57:18.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.433+0000 7fc5924ea640 1 -- 172.21.15.164:0/3850865899 learned_addr learned my addr 172.21.15.164:0/3850865899 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-03-23T14:57:18.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.433+0000 7fc592ceb640 1 --2- 172.21.15.164:0/3850865899 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc58c141470 0x7fc58c151c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:18.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.433+0000 7fc591ce9640 1 --2- 172.21.15.164:0/3850865899 >> v2:172.21.15.173:3300/0 conn(0x7fc58c0b4a70 0x7fc58c140f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:18.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.433+0000 7fc5914e8640 1 -- 172.21.15.164:0/3850865899 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3243568291 0 0) 0x7fc58c0b4300 con 0x7fc58c0b56f0 2024-03-23T14:57:18.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.433+0000 7fc5914e8640 1 -- 172.21.15.164:0/3850865899 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc578003530 con 0x7fc58c0b56f0 2024-03-23T14:57:18.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.433+0000 7fc5914e8640 1 -- 172.21.15.164:0/3850865899 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1100539423 0 0) 0x7fc578003530 con 0x7fc58c0b56f0 2024-03-23T14:57:18.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.433+0000 7fc5914e8640 1 -- 172.21.15.164:0/3850865899 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc58c141470 msgr2=0x7fc58c151c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:18.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.433+0000 7fc5914e8640 1 --2- 172.21.15.164:0/3850865899 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc58c141470 0x7fc58c151c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:18.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.433+0000 7fc5914e8640 1 -- 172.21.15.164:0/3850865899 >> v2:172.21.15.173:3300/0 conn(0x7fc58c0b4a70 msgr2=0x7fc58c140f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:18.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.433+0000 7fc5914e8640 1 --2- 172.21.15.164:0/3850865899 >> v2:172.21.15.173:3300/0 conn(0x7fc58c0b4a70 0x7fc58c140f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:18.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.433+0000 7fc5914e8640 1 -- 172.21.15.164:0/3850865899 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc58c0bd7e0 con 0x7fc58c0b56f0 2024-03-23T14:57:18.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.433+0000 7fc5914e8640 1 -- 172.21.15.164:0/3850865899 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2526390548 0 0) 0x7fc588001680 con 0x7fc58c0b56f0 2024-03-23T14:57:18.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.433+0000 7fc5914e8640 1 -- 172.21.15.164:0/3850865899 >> v1:172.21.15.164:6789/0 conn(0x7fc58c0b56f0 legacy=0x7fc58c14a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:18.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.433+0000 7fc5914e8640 1 --2- 172.21.15.164:0/3850865899 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc58c141470 0x7fc578003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:18.438 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.433+0000 7fc5914e8640 1 --2- 172.21.15.164:0/3850865899 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc578003f00 0x7fc5780047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:18.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.433+0000 7fc5914e8640 1 --2- 172.21.15.164:0/3850865899 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc578004d10 0x7fc578005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:18.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.433+0000 7fc5914e8640 1 -- 172.21.15.164:0/3850865899 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc58c0b45e0 con 0x7fc578004d10 2024-03-23T14:57:18.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.433+0000 7fc5914e8640 1 -- 172.21.15.164:0/3850865899 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fc57800b990 con 0x7fc578003f00 2024-03-23T14:57:18.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.433+0000 7fc5914e8640 1 -- 172.21.15.164:0/3850865899 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fc57800bad0 con 0x7fc58c141470 2024-03-23T14:57:18.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.433+0000 7fc592ceb640 1 --2- 172.21.15.164:0/3850865899 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc578004d10 0x7fc578005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:18.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.433+0000 7fc5924ea640 1 --2- 172.21.15.164:0/3850865899 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc58c141470 0x7fc578003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:18.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.433+0000 7fc591ce9640 1 --2- 172.21.15.164:0/3850865899 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc578003f00 0x7fc5780047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:18.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.433+0000 7fc592ceb640 1 -- 172.21.15.164:0/3850865899 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc58c141470 msgr2=0x7fc578003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:18.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.433+0000 7fc592ceb640 1 --2- 172.21.15.164:0/3850865899 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc58c141470 0x7fc578003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:18.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.433+0000 7fc592ceb640 1 -- 172.21.15.164:0/3850865899 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc578003f00 msgr2=0x7fc5780047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:18.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.433+0000 7fc592ceb640 1 --2- 172.21.15.164:0/3850865899 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc578003f00 0x7fc5780047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:18.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.433+0000 7fc592ceb640 1 -- 172.21.15.164:0/3850865899 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc578003530 con 0x7fc578004d10 2024-03-23T14:57:18.439 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.433+0000 7fc591ce9640 1 --2- 172.21.15.164:0/3850865899 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc578003f00 0x7fc5780047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T14:57:18.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.433+0000 7fc592ceb640 1 --2- 172.21.15.164:0/3850865899 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc578004d10 0x7fc578005190 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc584009fc0 tx=0x7fc58401cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a5498520666ac6c server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:18.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.433+0000 7fc5914e8640 1 -- 172.21.15.164:0/3850865899 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5840a5c90 con 0x7fc578004d10 2024-03-23T14:57:18.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.433+0000 7fc5914e8640 1 -- 172.21.15.164:0/3850865899 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc58400f040 con 0x7fc578004d10 2024-03-23T14:57:18.440 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.433+0000 7fc5914e8640 1 -- 172.21.15.164:0/3850865899 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5840ad440 con 0x7fc578004d10 2024-03-23T14:57:18.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.437+0000 7fc59474e640 1 -- 172.21.15.164:0/3850865899 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc578004d10 msgr2=0x7fc578005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:18.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.437+0000 7fc59474e640 1 --2- 172.21.15.164:0/3850865899 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc578004d10 0x7fc578005190 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc584009fc0 tx=0x7fc58401cc20 comp rx=0 tx=0).stop 2024-03-23T14:57:18.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.437+0000 7fc592ceb640 1 -- 172.21.15.164:0/3850865899 reap_dead start 2024-03-23T14:57:18.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.437+0000 7fc59474e640 1 -- 172.21.15.164:0/3850865899 shutdown_connections 2024-03-23T14:57:18.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.437+0000 7fc59474e640 1 -- 172.21.15.164:0/3850865899 >> 172.21.15.164:0/3850865899 conn(0x7fc58c02c3c0 msgr2=0x7fc58c02c7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:18.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.437+0000 7fc59474e640 1 -- 172.21.15.164:0/3850865899 shutdown_connections 2024-03-23T14:57:18.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.437+0000 7fc59474e640 1 -- 172.21.15.164:0/3850865899 wait complete. 2024-03-23T14:57:18.443 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.437+0000 7fc59474e640 1 Processor -- start 2024-03-23T14:57:18.443 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.437+0000 7fc59474e640 1 -- start start 2024-03-23T14:57:18.443 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.437+0000 7fc59474e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc578004d10 0x7fc58c143c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:18.443 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.437+0000 7fc59474e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc58c1df880 0x7fc58c1dfd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:18.443 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.437+0000 7fc59474e640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc58c0b2640 0x7fc58c0b2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:18.443 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.437+0000 7fc59474e640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc58c14b630 con 0x7fc58c1df880 2024-03-23T14:57:18.443 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.437+0000 7fc59474e640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fc58c1433d0 con 0x7fc578004d10 2024-03-23T14:57:18.443 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.437+0000 7fc59474e640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fc58c142d80 con 0x7fc58c0b2640 2024-03-23T14:57:18.443 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.437+0000 7fc591ce9640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc58c1df880 0x7fc58c1dfd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:18.443 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.437+0000 7fc591ce9640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc58c1df880 0x7fc58c1dfd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:60278/0 (socket says 172.21.15.164:60278) 2024-03-23T14:57:18.443 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.437+0000 7fc591ce9640 1 -- 172.21.15.164:0/593773684 learned_addr learned my addr 172.21.15.164:0/593773684 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:18.443 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.437+0000 7fc592ceb640 1 --2- 172.21.15.164:0/593773684 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc58c0b2640 0x7fc58c0b2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:18.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.437+0000 7fc5924ea640 1 --2- 172.21.15.164:0/593773684 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc578004d10 0x7fc58c143c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:18.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.437+0000 7fc591ce9640 1 -- 172.21.15.164:0/593773684 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc58c0b2640 msgr2=0x7fc58c0b2ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:18.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.437+0000 7fc591ce9640 1 --2- 172.21.15.164:0/593773684 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc58c0b2640 0x7fc58c0b2ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:18.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.437+0000 7fc591ce9640 1 -- 172.21.15.164:0/593773684 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc578004d10 msgr2=0x7fc58c143c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:18.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.437+0000 7fc591ce9640 1 --2- 172.21.15.164:0/593773684 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc578004d10 0x7fc58c143c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:18.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.437+0000 7fc591ce9640 1 -- 172.21.15.164:0/593773684 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc58c143060 con 0x7fc58c1df880 2024-03-23T14:57:18.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.437+0000 7fc591ce9640 1 --2- 172.21.15.164:0/593773684 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc58c1df880 0x7fc58c1dfd00 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc57c010980 tx=0x7fc57c010e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:18.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.437+0000 7fc582ffd640 1 -- 172.21.15.164:0/593773684 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc57c00ec20 con 0x7fc58c1df880 2024-03-23T14:57:18.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.437+0000 7fc582ffd640 1 -- 172.21.15.164:0/593773684 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc57c00ed80 con 0x7fc58c1df880 2024-03-23T14:57:18.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.437+0000 7fc59474e640 1 -- 172.21.15.164:0/593773684 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc58c0a9e70 con 0x7fc58c1df880 2024-03-23T14:57:18.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.437+0000 7fc582ffd640 1 -- 172.21.15.164:0/593773684 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc57c004220 con 0x7fc58c1df880 2024-03-23T14:57:18.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.437+0000 7fc59474e640 1 -- 172.21.15.164:0/593773684 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc58c144180 con 0x7fc58c1df880 2024-03-23T14:57:18.445 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.437+0000 7fc59474e640 1 -- 172.21.15.164:0/593773684 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc58c0690a0 con 0x7fc58c1df880 2024-03-23T14:57:18.445 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.441+0000 7fc582ffd640 1 -- 172.21.15.164:0/593773684 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7fc57c004380 con 0x7fc58c1df880 2024-03-23T14:57:18.445 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.441+0000 7fc582ffd640 1 --2- 172.21.15.164:0/593773684 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fc5700672a0 0x7fc570069760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:18.445 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.441+0000 7fc5924ea640 1 --2- 172.21.15.164:0/593773684 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fc5700672a0 0x7fc570069760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:18.445 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.441+0000 7fc582ffd640 1 -- 172.21.15.164:0/593773684 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fc57c084f60 con 0x7fc58c1df880 2024-03-23T14:57:18.445 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.441+0000 7fc5924ea640 1 --2- 172.21.15.164:0/593773684 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fc5700672a0 0x7fc570069760 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc588001ec0 tx=0x7fc588009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:18.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.445+0000 7fc582ffd640 1 -- 172.21.15.164:0/593773684 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7fc57c051ba0 con 0x7fc58c1df880 2024-03-23T14:57:18.606 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.601+0000 7fc59474e640 1 -- 172.21.15.164:0/593773684 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fc58c1431f0 con 0x7fc58c1df880 2024-03-23T14:57:18.607 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.601+0000 7fc582ffd640 1 -- 172.21.15.164:0/593773684 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7fc57c055260 con 0x7fc58c1df880 2024-03-23T14:57:18.608 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 32 2024-03-23T14:57:18.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.605+0000 7fc59474e640 1 -- 172.21.15.164:0/593773684 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fc5700672a0 msgr2=0x7fc570069760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:18.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.605+0000 7fc59474e640 1 --2- 172.21.15.164:0/593773684 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fc5700672a0 0x7fc570069760 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc588001ec0 tx=0x7fc588009040 comp rx=0 tx=0).stop 2024-03-23T14:57:18.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.605+0000 7fc59474e640 1 -- 172.21.15.164:0/593773684 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc58c1df880 msgr2=0x7fc58c1dfd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:18.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.605+0000 7fc59474e640 1 --2- 172.21.15.164:0/593773684 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc58c1df880 0x7fc58c1dfd00 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc57c010980 tx=0x7fc57c010e40 comp rx=0 tx=0).stop 2024-03-23T14:57:18.610 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.605+0000 7fc59474e640 1 -- 172.21.15.164:0/593773684 shutdown_connections 2024-03-23T14:57:18.610 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.605+0000 7fc59474e640 1 --2- 172.21.15.164:0/593773684 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc58c0b2640 0x7fc58c0b2ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:18.610 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.605+0000 7fc59474e640 1 --2- 172.21.15.164:0/593773684 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc58c1df880 0x7fc58c1dfd00 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:18.610 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.605+0000 7fc59474e640 1 --2- 172.21.15.164:0/593773684 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fc5700672a0 0x7fc570069760 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:18.610 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.605+0000 7fc59474e640 1 --2- 172.21.15.164:0/593773684 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc578004d10 0x7fc58c143c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:18.610 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.605+0000 7fc59474e640 1 -- 172.21.15.164:0/593773684 >> 172.21.15.164:0/593773684 conn(0x7fc58c02c3c0 msgr2=0x7fc58c13d2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:18.610 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.605+0000 7fc59474e640 1 -- 172.21.15.164:0/593773684 shutdown_connections 2024-03-23T14:57:18.610 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.605+0000 7fc59474e640 1 -- 172.21.15.164:0/593773684 wait complete. 2024-03-23T14:57:18.626 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-23T14:57:18.747 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.741+0000 7f7c42355640 1 Processor -- start 2024-03-23T14:57:18.747 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.741+0000 7f7c42355640 1 -- start start 2024-03-23T14:57:18.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.745+0000 7f7c42355640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f7c3c09def0 0x7f7c3c0ae620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:18.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.745+0000 7f7c42355640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f7c3c0b4a00 0x7f7c3c0b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:18.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.745+0000 7f7c42355640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c3c09e2d0 con 0x7f7c3c147420 2024-03-23T14:57:18.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.745+0000 7f7c42355640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f7c3c146de0 con 0x7f7c3c0b4a00 2024-03-23T14:57:18.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.745+0000 7f7c42355640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 -- 0x7f7c3c06cb70 con 0x7f7c3c09def0 2024-03-23T14:57:18.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.745+0000 7f7c337fe640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f7c3c0b4a00 0x7f7c3c0b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:18.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.745+0000 7f7c408f2640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f7c3c147420 0x7f7c3c0b34d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:56886/0 (socket says 172.21.15.164:56886) 2024-03-23T14:57:18.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.745+0000 7f7c408f2640 1 -- 172.21.15.164:0/611328897 learned_addr learned my addr 172.21.15.164:0/611328897 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-03-23T14:57:18.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.745+0000 7f7c3bfff640 1 --2- 172.21.15.164:0/611328897 >> v2:172.21.15.173:3300/0 conn(0x7f7c3c09def0 0x7f7c3c0ae620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:18.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.745+0000 7f7c337fe640 1 -- 172.21.15.164:0/611328897 >> v2:172.21.15.173:3300/0 conn(0x7f7c3c09def0 msgr2=0x7f7c3c0ae620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:18.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.745+0000 7f7c337fe640 1 --2- 172.21.15.164:0/611328897 >> v2:172.21.15.173:3300/0 conn(0x7f7c3c09def0 0x7f7c3c0ae620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:18.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.745+0000 7f7c337fe640 1 -- 172.21.15.164:0/611328897 >> v1:172.21.15.164:6789/0 conn(0x7f7c3c147420 legacy=0x7f7c3c0b34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:18.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.745+0000 7f7c3b7fe640 1 -- 172.21.15.164:0/611328897 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1261675459 0 0) 0x7f7c3c09e2d0 con 0x7f7c3c147420 2024-03-23T14:57:18.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.745+0000 7f7c337fe640 1 -- 172.21.15.164:0/611328897 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c3c06ce50 con 0x7f7c3c0b4a00 2024-03-23T14:57:18.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.745+0000 7f7c337fe640 1 --2- 172.21.15.164:0/611328897 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7c3c0b4a00 0x7f7c3c0b4e90 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f7c3c06da30 tx=0x7f7c2401c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad09fee6a25660d0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:18.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.745+0000 7f7c3b7fe640 1 -- 172.21.15.164:0/611328897 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c240ab600 con 0x7f7c3c0b4a00 2024-03-23T14:57:18.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.745+0000 7f7c3b7fe640 1 -- 172.21.15.164:0/611328897 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c240a7050 con 0x7f7c3c0b4a00 2024-03-23T14:57:18.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.745+0000 7f7c3b7fe640 1 -- 172.21.15.164:0/611328897 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c240ab980 con 0x7f7c3c0b4a00 2024-03-23T14:57:18.752 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.749+0000 7f7c42355640 1 -- 172.21.15.164:0/611328897 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7c3c0b4a00 msgr2=0x7f7c3c0b4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:18.752 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.749+0000 7f7c42355640 1 --2- 172.21.15.164:0/611328897 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7c3c0b4a00 0x7f7c3c0b4e90 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f7c3c06da30 tx=0x7f7c2401c3f0 comp rx=0 tx=0).stop 2024-03-23T14:57:18.752 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.749+0000 7f7c42355640 1 -- 172.21.15.164:0/611328897 shutdown_connections 2024-03-23T14:57:18.752 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.749+0000 7f7c42355640 1 --2- 172.21.15.164:0/611328897 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7c3c0b4a00 0x7f7c3c0b4e90 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:18.752 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.749+0000 7f7c42355640 1 --2- 172.21.15.164:0/611328897 >> v2:172.21.15.173:3300/0 conn(0x7f7c3c09def0 0x7f7c3c0ae620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:18.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.749+0000 7f7c42355640 1 -- 172.21.15.164:0/611328897 >> 172.21.15.164:0/611328897 conn(0x7f7c3c0b8ca0 msgr2=0x7f7c3c0b90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:18.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.749+0000 7f7c42355640 1 -- 172.21.15.164:0/611328897 shutdown_connections 2024-03-23T14:57:18.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.749+0000 7f7c42355640 1 -- 172.21.15.164:0/611328897 wait complete. 2024-03-23T14:57:18.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.749+0000 7f7c42355640 1 Processor -- start 2024-03-23T14:57:18.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.749+0000 7f7c42355640 1 -- start start 2024-03-23T14:57:18.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.749+0000 7f7c42355640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7c3c09def0 0x7f7c3c09d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:18.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.749+0000 7f7c42355640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7c3c0b4a00 0x7f7c3c071160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:18.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.749+0000 7f7c42355640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7c3c147420 0x7f7c3c06f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:18.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.749+0000 7f7c42355640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f7c3c150f50 con 0x7f7c3c09def0 2024-03-23T14:57:18.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.749+0000 7f7c42355640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f7c3c070960 con 0x7f7c3c0b4a00 2024-03-23T14:57:18.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.749+0000 7f7c42355640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f7c3c0716a0 con 0x7f7c3c147420 2024-03-23T14:57:18.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.749+0000 7f7c3bfff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7c3c09def0 0x7f7c3c09d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:18.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.749+0000 7f7c3bfff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7c3c09def0 0x7f7c3c09d0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:60288/0 (socket says 172.21.15.164:60288) 2024-03-23T14:57:18.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.749+0000 7f7c3bfff640 1 -- 172.21.15.164:0/2924765563 learned_addr learned my addr 172.21.15.164:0/2924765563 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:18.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.749+0000 7f7c408f2640 1 --2- 172.21.15.164:0/2924765563 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7c3c147420 0x7f7c3c06f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:18.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.749+0000 7f7c3bfff640 1 -- 172.21.15.164:0/2924765563 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7c3c147420 msgr2=0x7f7c3c06f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:18.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.749+0000 7f7c337fe640 1 --2- 172.21.15.164:0/2924765563 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7c3c0b4a00 0x7f7c3c071160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:18.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.749+0000 7f7c3bfff640 1 --2- 172.21.15.164:0/2924765563 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7c3c147420 0x7f7c3c06f2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:18.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.749+0000 7f7c3bfff640 1 -- 172.21.15.164:0/2924765563 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7c3c0b4a00 msgr2=0x7f7c3c071160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:18.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.749+0000 7f7c3bfff640 1 --2- 172.21.15.164:0/2924765563 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7c3c0b4a00 0x7f7c3c071160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:18.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.749+0000 7f7c3bfff640 1 -- 172.21.15.164:0/2924765563 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c24009420 con 0x7f7c3c09def0 2024-03-23T14:57:18.756 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.749+0000 7f7c3bfff640 1 --2- 172.21.15.164:0/2924765563 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7c3c09def0 0x7f7c3c09d0c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f7c2c00d830 tx=0x7f7c2c00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:18.756 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.749+0000 7f7c397fa640 1 -- 172.21.15.164:0/2924765563 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c2c00b6c0 con 0x7f7c3c09def0 2024-03-23T14:57:18.756 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.749+0000 7f7c397fa640 1 -- 172.21.15.164:0/2924765563 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c2c00b820 con 0x7f7c3c09def0 2024-03-23T14:57:18.756 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.749+0000 7f7c42355640 1 -- 172.21.15.164:0/2924765563 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c3c0addc0 con 0x7f7c3c09def0 2024-03-23T14:57:18.756 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.749+0000 7f7c397fa640 1 -- 172.21.15.164:0/2924765563 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c2c00bb90 con 0x7f7c3c09def0 2024-03-23T14:57:18.756 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.749+0000 7f7c42355640 1 -- 172.21.15.164:0/2924765563 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c3c0705a0 con 0x7f7c3c09def0 2024-03-23T14:57:18.757 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.749+0000 7f7c42355640 1 -- 172.21.15.164:0/2924765563 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c3c0690a0 con 0x7f7c3c09def0 2024-03-23T14:57:18.757 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.753+0000 7f7c397fa640 1 -- 172.21.15.164:0/2924765563 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7f7c2c01e460 con 0x7f7c3c09def0 2024-03-23T14:57:18.757 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.753+0000 7f7c397fa640 1 --2- 172.21.15.164:0/2924765563 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f7c1c0672a0 0x7f7c1c069760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:18.757 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.753+0000 7f7c337fe640 1 --2- 172.21.15.164:0/2924765563 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f7c1c0672a0 0x7f7c1c069760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:18.757 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.753+0000 7f7c397fa640 1 -- 172.21.15.164:0/2924765563 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f7c2c0857f0 con 0x7f7c3c09def0 2024-03-23T14:57:18.757 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.753+0000 7f7c337fe640 1 --2- 172.21.15.164:0/2924765563 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f7c1c0672a0 0x7f7c1c069760 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f7c240ad000 tx=0x7f7c24008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:18.760 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.757+0000 7f7c397fa640 1 -- 172.21.15.164:0/2924765563 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7f7c2c052410 con 0x7f7c3c09def0 2024-03-23T14:57:18.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.901+0000 7f7c42355640 1 -- 172.21.15.164:0/2924765563 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f7c3c145340 con 0x7f7c3c09def0 2024-03-23T14:57:18.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.901+0000 7f7c397fa640 1 -- 172.21.15.164:0/2924765563 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f7c2c055ad0 con 0x7f7c3c09def0 2024-03-23T14:57:18.906 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 64 2024-03-23T14:57:18.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.905+0000 7f7c42355640 1 -- 172.21.15.164:0/2924765563 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f7c1c0672a0 msgr2=0x7f7c1c069760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:18.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.905+0000 7f7c42355640 1 --2- 172.21.15.164:0/2924765563 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f7c1c0672a0 0x7f7c1c069760 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f7c240ad000 tx=0x7f7c24008040 comp rx=0 tx=0).stop 2024-03-23T14:57:18.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.905+0000 7f7c42355640 1 -- 172.21.15.164:0/2924765563 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7c3c09def0 msgr2=0x7f7c3c09d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:18.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.905+0000 7f7c42355640 1 --2- 172.21.15.164:0/2924765563 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7c3c09def0 0x7f7c3c09d0c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f7c2c00d830 tx=0x7f7c2c00dcf0 comp rx=0 tx=0).stop 2024-03-23T14:57:18.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.905+0000 7f7c42355640 1 -- 172.21.15.164:0/2924765563 shutdown_connections 2024-03-23T14:57:18.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.905+0000 7f7c42355640 1 --2- 172.21.15.164:0/2924765563 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7c3c147420 0x7f7c3c06f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:18.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.905+0000 7f7c42355640 1 --2- 172.21.15.164:0/2924765563 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f7c1c0672a0 0x7f7c1c069760 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:18.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.905+0000 7f7c42355640 1 --2- 172.21.15.164:0/2924765563 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7c3c0b4a00 0x7f7c3c071160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:18.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.905+0000 7f7c42355640 1 --2- 172.21.15.164:0/2924765563 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7c3c09def0 0x7f7c3c09d0c0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:18.910 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.905+0000 7f7c42355640 1 -- 172.21.15.164:0/2924765563 >> 172.21.15.164:0/2924765563 conn(0x7f7c3c0b8ca0 msgr2=0x7f7c3c0b9080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:18.910 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.905+0000 7f7c42355640 1 -- 172.21.15.164:0/2924765563 shutdown_connections 2024-03-23T14:57:18.910 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:18.905+0000 7f7c42355640 1 -- 172.21.15.164:0/2924765563 wait complete. 2024-03-23T14:57:18.925 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T14:57:19.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.041+0000 7f7dbfb8c640 1 Processor -- start 2024-03-23T14:57:19.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.041+0000 7f7dbfb8c640 1 -- start start 2024-03-23T14:57:19.045 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.041+0000 7f7dbfb8c640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f7db80b56f0 0x7f7db814a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:19.045 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.041+0000 7f7dbfb8c640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f7db80b4a70 0x7f7db8140f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:19.045 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.041+0000 7f7dbfb8c640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7db8141470 0x7f7db80b75f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:19.045 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.041+0000 7f7dbfb8c640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f7db80b5fa0 con 0x7f7db80b4a70 2024-03-23T14:57:19.045 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.041+0000 7f7dbfb8c640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 -- 0x7f7db80b4300 con 0x7f7db80b56f0 2024-03-23T14:57:19.046 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.041+0000 7f7dbfb8c640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f7db80b45e0 con 0x7f7db8141470 2024-03-23T14:57:19.046 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.041+0000 7f7dbd127640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f7db80b4a70 0x7f7db8140f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:19.046 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.041+0000 7f7dbd127640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f7db80b4a70 0x7f7db8140f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:60298/0 (socket says 172.21.15.164:60298) 2024-03-23T14:57:19.046 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.041+0000 7f7dbd127640 1 -- 172.21.15.164:0/3834064293 learned_addr learned my addr 172.21.15.164:0/3834064293 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:19.046 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.041+0000 7f7dbe129640 1 --2- 172.21.15.164:0/3834064293 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7db8141470 0x7f7db80b75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:19.046 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.041+0000 7f7dbd928640 1 --2- 172.21.15.164:0/3834064293 >> v2:172.21.15.173:3300/0 conn(0x7f7db80b56f0 0x7f7db814a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:19.046 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.041+0000 7f7dbd127640 1 -- 172.21.15.164:0/3834064293 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7db8141470 msgr2=0x7f7db80b75f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:19.046 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.041+0000 7f7dbd127640 1 --2- 172.21.15.164:0/3834064293 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7db8141470 0x7f7db80b75f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:19.046 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.041+0000 7f7dbd127640 1 -- 172.21.15.164:0/3834064293 >> v2:172.21.15.173:3300/0 conn(0x7f7db80b56f0 msgr2=0x7f7db814a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:19.046 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.041+0000 7f7dbd127640 1 --2- 172.21.15.164:0/3834064293 >> v2:172.21.15.173:3300/0 conn(0x7f7db80b56f0 0x7f7db814a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:19.046 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.041+0000 7f7dbd127640 1 -- 172.21.15.164:0/3834064293 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7db814ade0 con 0x7f7db80b4a70 2024-03-23T14:57:19.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.041+0000 7f7dbd127640 1 --2- 172.21.15.164:0/3834064293 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7db80b4a70 0x7f7db8140f30 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f7db80ba730 tx=0x7f7da801c230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ccf6014d21dda75 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:19.048 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.041+0000 7f7dbc926640 1 -- 172.21.15.164:0/3834064293 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7da80ab600 con 0x7f7db80b4a70 2024-03-23T14:57:19.048 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.045+0000 7f7dbc926640 1 -- 172.21.15.164:0/3834064293 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7da80a7050 con 0x7f7db80b4a70 2024-03-23T14:57:19.048 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.045+0000 7f7dbc926640 1 -- 172.21.15.164:0/3834064293 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7da80ab980 con 0x7f7db80b4a70 2024-03-23T14:57:19.048 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.045+0000 7f7dbfb8c640 1 -- 172.21.15.164:0/3834064293 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7db80b4a70 msgr2=0x7f7db8140f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:19.048 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.045+0000 7f7dbfb8c640 1 --2- 172.21.15.164:0/3834064293 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7db80b4a70 0x7f7db8140f30 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f7db80ba730 tx=0x7f7da801c230 comp rx=0 tx=0).stop 2024-03-23T14:57:19.049 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.045+0000 7f7dbfb8c640 1 -- 172.21.15.164:0/3834064293 shutdown_connections 2024-03-23T14:57:19.049 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.045+0000 7f7dbfb8c640 1 --2- 172.21.15.164:0/3834064293 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7db8141470 0x7f7db80b75f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:19.049 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.045+0000 7f7dbfb8c640 1 --2- 172.21.15.164:0/3834064293 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7db80b4a70 0x7f7db8140f30 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:19.049 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.045+0000 7f7dbfb8c640 1 --2- 172.21.15.164:0/3834064293 >> v2:172.21.15.173:3300/0 conn(0x7f7db80b56f0 0x7f7db814a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:19.049 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.045+0000 7f7dbfb8c640 1 -- 172.21.15.164:0/3834064293 >> 172.21.15.164:0/3834064293 conn(0x7f7db802c3c0 msgr2=0x7f7db802c7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:19.049 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.045+0000 7f7dbfb8c640 1 -- 172.21.15.164:0/3834064293 shutdown_connections 2024-03-23T14:57:19.049 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.045+0000 7f7dbfb8c640 1 -- 172.21.15.164:0/3834064293 wait complete. 2024-03-23T14:57:19.049 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.045+0000 7f7dbfb8c640 1 Processor -- start 2024-03-23T14:57:19.049 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.045+0000 7f7dbfb8c640 1 -- start start 2024-03-23T14:57:19.051 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.045+0000 7f7dbfb8c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7db80b4a70 0x7f7db81420d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:19.051 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.045+0000 7f7dbfb8c640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7db80b56f0 0x7f7db8135ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:19.051 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.045+0000 7f7dbfb8c640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7db8141470 0x7f7db813b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:19.051 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.045+0000 7f7dbfb8c640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f7db8142610 con 0x7f7db80b4a70 2024-03-23T14:57:19.051 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.045+0000 7f7dbfb8c640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f7db8143c20 con 0x7f7db8141470 2024-03-23T14:57:19.052 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.045+0000 7f7dbfb8c640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f7db81447b0 con 0x7f7db80b56f0 2024-03-23T14:57:19.052 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.045+0000 7f7dbd928640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7db80b4a70 0x7f7db81420d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:19.052 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.045+0000 7f7dbd928640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7db80b4a70 0x7f7db81420d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:60302/0 (socket says 172.21.15.164:60302) 2024-03-23T14:57:19.052 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.045+0000 7f7dbd928640 1 -- 172.21.15.164:0/299574031 learned_addr learned my addr 172.21.15.164:0/299574031 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:19.052 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.045+0000 7f7dbe129640 1 --2- 172.21.15.164:0/299574031 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7db8141470 0x7f7db813b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:19.052 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.045+0000 7f7dbd127640 1 --2- 172.21.15.164:0/299574031 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7db80b56f0 0x7f7db8135ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:19.052 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.045+0000 7f7dbd928640 1 -- 172.21.15.164:0/299574031 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7db80b56f0 msgr2=0x7f7db8135ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:19.052 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.045+0000 7f7dbd928640 1 --2- 172.21.15.164:0/299574031 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7db80b56f0 0x7f7db8135ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:19.052 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.045+0000 7f7dbd928640 1 -- 172.21.15.164:0/299574031 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7db8141470 msgr2=0x7f7db813b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:19.052 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.045+0000 7f7dbd928640 1 --2- 172.21.15.164:0/299574031 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7db8141470 0x7f7db813b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:19.052 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.045+0000 7f7dbd928640 1 -- 172.21.15.164:0/299574031 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7da80094d0 con 0x7f7db80b4a70 2024-03-23T14:57:19.052 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.045+0000 7f7dbd928640 1 --2- 172.21.15.164:0/299574031 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7db80b4a70 0x7f7db81420d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f7db400d830 tx=0x7f7db400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:19.052 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.045+0000 7f7dae7fc640 1 -- 172.21.15.164:0/299574031 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7db40096a0 con 0x7f7db80b4a70 2024-03-23T14:57:19.052 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.045+0000 7f7dae7fc640 1 -- 172.21.15.164:0/299574031 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7db4009800 con 0x7f7db80b4a70 2024-03-23T14:57:19.053 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.045+0000 7f7dbfb8c640 1 -- 172.21.15.164:0/299574031 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7db8142d20 con 0x7f7db80b4a70 2024-03-23T14:57:19.053 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.045+0000 7f7dae7fc640 1 -- 172.21.15.164:0/299574031 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7db4009b70 con 0x7f7db80b4a70 2024-03-23T14:57:19.053 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.045+0000 7f7dbfb8c640 1 -- 172.21.15.164:0/299574031 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7db8143940 con 0x7f7db80b4a70 2024-03-23T14:57:19.053 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.045+0000 7f7dbfb8c640 1 -- 172.21.15.164:0/299574031 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7db80690a0 con 0x7f7db80b4a70 2024-03-23T14:57:19.053 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.049+0000 7f7dae7fc640 1 -- 172.21.15.164:0/299574031 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7f7db4014410 con 0x7f7db80b4a70 2024-03-23T14:57:19.053 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.049+0000 7f7dae7fc640 1 --2- 172.21.15.164:0/299574031 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f7d9c0672a0 0x7f7d9c069760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:19.054 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.049+0000 7f7dae7fc640 1 -- 172.21.15.164:0/299574031 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f7db4085e40 con 0x7f7db80b4a70 2024-03-23T14:57:19.054 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.049+0000 7f7dbd127640 1 --2- 172.21.15.164:0/299574031 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f7d9c0672a0 0x7f7d9c069760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:19.054 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.049+0000 7f7dbd127640 1 --2- 172.21.15.164:0/299574031 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f7d9c0672a0 0x7f7d9c069760 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f7da80ad000 tx=0x7f7da8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:19.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.053+0000 7f7dae7fc640 1 -- 172.21.15.164:0/299574031 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7f7db4052b60 con 0x7f7db80b4a70 2024-03-23T14:57:19.219 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.213+0000 7f7dbfb8c640 1 -- 172.21.15.164:0/299574031 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7db8144a20 con 0x7f7db80b4a70 2024-03-23T14:57:19.223 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.217+0000 7f7dae7fc640 1 -- 172.21.15.164:0/299574031 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16629 (secure 0 0 0) 0x7f7db4056220 con 0x7f7db80b4a70 2024-03-23T14:57:19.223 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-23T14:57:19.223 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":20,"fsid":"45c7911b-b69d-4e68-907b-fbfeb780aa3b","created":"2024-03-23T14:56:34.367633+0000","modified":"2024-03-23T14:57:00.611265+0000","last_up_change":"2024-03-23T14:56:46.297403+0000","last_in_change":"2024-03-23T14:56:36.629884+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T14:56:47.186157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-23T14:56:56.994057+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-23T14:56:57.953862+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"80abdc62-bd0d-4e60-a554-7ed6fd67cf54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":984716690},{"type":"v1","addr":"172.21.15.164:6801","nonce":984716690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":984716690},{"type":"v1","addr":"172.21.15.164:6803","nonce":984716690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":984716690},{"type":"v1","addr":"172.21.15.164:6807","nonce":984716690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":984716690},{"type":"v1","addr":"172.21.15.164:6805","nonce":984716690}]},"public_addr":"172.21.15.164:6801/984716690","cluster_addr":"172.21.15.164:6803/984716690","heartbeat_back_addr":"172.21.15.164:6807/984716690","heartbeat_front_addr":"172.21.15.164:6805/984716690","state":["exists","up"]},{"osd":1,"uuid":"f9001b06-be4d-4f42-a0e6-bcef2afa0d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":1349657239},{"type":"v1","addr":"172.21.15.164:6817","nonce":1349657239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":1349657239},{"type":"v1","addr":"172.21.15.164:6819","nonce":1349657239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":1349657239},{"type":"v1","addr":"172.21.15.164:6823","nonce":1349657239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":1349657239},{"type":"v1","addr":"172.21.15.164:6821","nonce":1349657239}]},"public_addr":"172.21.15.164:6817/1349657239","cluster_addr":"172.21.15.164:6819/1349657239","heartbeat_back_addr":"172.21.15.164:6823/1349657239","heartbeat_front_addr":"172.21.15.164:6821/1349657239","state":["exists","up"]},{"osd":2,"uuid":"fa0aeb9f-8b54-4831-a188-3e361c7980d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":887861252},{"type":"v1","addr":"172.21.15.164:6809","nonce":887861252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":887861252},{"type":"v1","addr":"172.21.15.164:6811","nonce":887861252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":887861252},{"type":"v1","addr":"172.21.15.164:6815","nonce":887861252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":887861252},{"type":"v1","addr":"172.21.15.164:6813","nonce":887861252}]},"public_addr":"172.21.15.164:6809/887861252","cluster_addr":"172.21.15.164:6811/887861252","heartbeat_back_addr":"172.21.15.164:6815/887861252","heartbeat_front_addr":"172.21.15.164:6813/887861252","state":["exists","up"]},{"osd":3,"uuid":"7e18c957-3091-46d5-8e17-8f3354c1726c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":812540271},{"type":"v1","addr":"172.21.15.164:6825","nonce":812540271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":812540271},{"type":"v1","addr":"172.21.15.164:6827","nonce":812540271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":812540271},{"type":"v1","addr":"172.21.15.164:6831","nonce":812540271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":812540271},{"type":"v1","addr":"172.21.15.164:6829","nonce":812540271}]},"public_addr":"172.21.15.164:6825/812540271","cluster_addr":"172.21.15.164:6827/812540271","heartbeat_back_addr":"172.21.15.164:6831/812540271","heartbeat_front_addr":"172.21.15.164:6829/812540271","state":["exists","up"]},{"osd":4,"uuid":"8e83f315-42bf-4471-9ce8-d81948da0f9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":2656857421},{"type":"v1","addr":"172.21.15.173:6825","nonce":2656857421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2656857421},{"type":"v1","addr":"172.21.15.173:6827","nonce":2656857421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2656857421},{"type":"v1","addr":"172.21.15.173:6831","nonce":2656857421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":2656857421},{"type":"v1","addr":"172.21.15.173:6829","nonce":2656857421}]},"public_addr":"172.21.15.173:6825/2656857421","cluster_addr":"172.21.15.173:6827/2656857421","heartbeat_back_addr":"172.21.15.173:6831/2656857421","heartbeat_front_addr":"172.21.15.173:6829/2656857421","state":["exists","up"]},{"osd":5,"uuid":"a230c872-71f9-40ab-8999-defd0162d4ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":3676483185},{"type":"v1","addr":"172.21.15.173:6801","nonce":3676483185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":3676483185},{"type":"v1","addr":"172.21.15.173:6803","nonce":3676483185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":3676483185},{"type":"v1","addr":"172.21.15.173:6807","nonce":3676483185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":3676483185},{"type":"v1","addr":"172.21.15.173:6805","nonce":3676483185}]},"public_addr":"172.21.15.173:6801/3676483185","cluster_addr":"172.21.15.173:6803/3676483185","heartbeat_back_addr":"172.21.15.173:6807/3676483185","heartbeat_front_addr":"172.21.15.173:6805/3676483185","state":["exists","up"]},{"osd":6,"uuid":"6193bfd7-8432-4f7f-a117-a3fcee21af99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":3631684087},{"type":"v1","addr":"172.21.15.173:6809","nonce":3631684087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":3631684087},{"type":"v1","addr":"172.21.15.173:6811","nonce":3631684087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":3631684087},{"type":"v1","addr":"172.21.15.173:6815","nonce":3631684087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":3631684087},{"type":"v1","addr":"172.21.15.173:6813","nonce":3631684087}]},"public_addr":"172.21.15.173:6809/3631684087","cluster_addr":"172.21.15.173:6811/3631684087","heartbeat_back_addr":"172.21.15.173:6815/3631684087","heartbeat_front_addr":"172.21.15.173:6813/3631684087","state":["exists","up"]},{"osd":7,"uuid":"b0f1b80a-afc4-4538-83b9-f081d2dcb8cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":3261114948},{"type":"v1","addr":"172.21.15.173:6817","nonce":3261114948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":3261114948},{"type":"v1","addr":"172.21.15.173:6819","nonce":3261114948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":3261114948},{"type":"v1","addr":"172.21.15.173:6823","nonce":3261114948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":3261114948},{"type":"v1","addr":"172.21.15.173:6821","nonce":3261114948}]},"public_addr":"172.21.15.173:6817/3261114948","cluster_addr":"172.21.15.173:6819/3261114948","heartbeat_back_addr":"172.21.15.173:6823/3261114948","heartbeat_front_addr":"172.21.15.173:6821/3261114948","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.188390+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.159932+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.191101+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.181776+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.224365+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.177515+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.153631+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.139972+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T14:57:19.225 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.221+0000 7f7dbfb8c640 1 -- 172.21.15.164:0/299574031 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f7d9c0672a0 msgr2=0x7f7d9c069760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:19.226 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.221+0000 7f7dbfb8c640 1 --2- 172.21.15.164:0/299574031 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f7d9c0672a0 0x7f7d9c069760 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f7da80ad000 tx=0x7f7da8008040 comp rx=0 tx=0).stop 2024-03-23T14:57:19.226 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.221+0000 7f7dbfb8c640 1 -- 172.21.15.164:0/299574031 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7db80b4a70 msgr2=0x7f7db81420d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:19.226 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.221+0000 7f7dbfb8c640 1 --2- 172.21.15.164:0/299574031 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7db80b4a70 0x7f7db81420d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f7db400d830 tx=0x7f7db400dcf0 comp rx=0 tx=0).stop 2024-03-23T14:57:19.226 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.221+0000 7f7dbfb8c640 1 -- 172.21.15.164:0/299574031 shutdown_connections 2024-03-23T14:57:19.226 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.221+0000 7f7dbfb8c640 1 --2- 172.21.15.164:0/299574031 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f7d9c0672a0 0x7f7d9c069760 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:19.226 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.221+0000 7f7dbfb8c640 1 --2- 172.21.15.164:0/299574031 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7db8141470 0x7f7db813b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:19.226 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.221+0000 7f7dbfb8c640 1 --2- 172.21.15.164:0/299574031 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7db80b56f0 0x7f7db8135ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:19.226 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.221+0000 7f7dbfb8c640 1 --2- 172.21.15.164:0/299574031 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7db80b4a70 0x7f7db81420d0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:19.226 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.221+0000 7f7dbfb8c640 1 -- 172.21.15.164:0/299574031 >> 172.21.15.164:0/299574031 conn(0x7f7db802c3c0 msgr2=0x7f7db80bc020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:19.226 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.221+0000 7f7dbfb8c640 1 -- 172.21.15.164:0/299574031 shutdown_connections 2024-03-23T14:57:19.226 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.221+0000 7f7dbfb8c640 1 -- 172.21.15.164:0/299574031 wait complete. 2024-03-23T14:57:19.242 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T14:56:47.186157+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-23T14:56:56.994057+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-23T14:56:57.953862+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-23T14:57:19.242 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-23T14:57:19.356 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.353+0000 7f186c9f6640 1 Processor -- start 2024-03-23T14:57:19.356 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.353+0000 7f186c9f6640 1 -- start start 2024-03-23T14:57:19.357 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.353+0000 7f186c9f6640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f186809def0 0x7f18680b45e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:19.357 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.353+0000 7f186c9f6640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f18680b4b20 0x7f186806c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:19.357 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.353+0000 7f186c9f6640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f186806d650 con 0x7f18680add50 2024-03-23T14:57:19.357 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.353+0000 7f186c9f6640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f18680af4a0 con 0x7f186809def0 2024-03-23T14:57:19.357 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.353+0000 7f186c9f6640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 -- 0x7f186809e2d0 con 0x7f18680b4b20 2024-03-23T14:57:19.358 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.353+0000 7f186659c640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f186809def0 0x7f18680b45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:19.358 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.353+0000 7f186659c640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f186809def0 0x7f18680b45e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:60308/0 (socket says 172.21.15.164:60308) 2024-03-23T14:57:19.358 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.353+0000 7f186659c640 1 -- 172.21.15.164:0/2426657622 learned_addr learned my addr 172.21.15.164:0/2426657622 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:19.358 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.353+0000 7f1866d9d640 1 --2- 172.21.15.164:0/2426657622 >> v2:172.21.15.173:3300/0 conn(0x7f18680b4b20 0x7f186806c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:19.358 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.353+0000 7f186659c640 1 -- 172.21.15.164:0/2426657622 >> v2:172.21.15.173:3300/0 conn(0x7f18680b4b20 msgr2=0x7f186806c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:19.358 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.353+0000 7f186659c640 1 --2- 172.21.15.164:0/2426657622 >> v2:172.21.15.173:3300/0 conn(0x7f18680b4b20 0x7f186806c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:19.358 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.353+0000 7f186659c640 1 -- 172.21.15.164:0/2426657622 >> v1:172.21.15.173:6789/0 conn(0x7f18680add50 legacy=0x7f186806bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:19.358 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.353+0000 7f186659c640 1 -- 172.21.15.164:0/2426657622 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18680aeca0 con 0x7f186809def0 2024-03-23T14:57:19.359 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.353+0000 7f186659c640 1 --2- 172.21.15.164:0/2426657622 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f186809def0 0x7f18680b45e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f18680b54c0 tx=0x7f185801c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e98a49e7c5ce951b server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:19.360 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.353+0000 7f186559a640 1 -- 172.21.15.164:0/2426657622 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18580a56b0 con 0x7f186809def0 2024-03-23T14:57:19.360 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.357+0000 7f186559a640 1 -- 172.21.15.164:0/2426657622 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18580a5810 con 0x7f186809def0 2024-03-23T14:57:19.360 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.357+0000 7f186559a640 1 -- 172.21.15.164:0/2426657622 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18580a5ba0 con 0x7f186809def0 2024-03-23T14:57:19.360 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.357+0000 7f186c9f6640 1 -- 172.21.15.164:0/2426657622 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f186809def0 msgr2=0x7f18680b45e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:19.360 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.357+0000 7f186c9f6640 1 --2- 172.21.15.164:0/2426657622 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f186809def0 0x7f18680b45e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f18680b54c0 tx=0x7f185801c890 comp rx=0 tx=0).stop 2024-03-23T14:57:19.360 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.357+0000 7f186c9f6640 1 -- 172.21.15.164:0/2426657622 shutdown_connections 2024-03-23T14:57:19.361 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.357+0000 7f186c9f6640 1 --2- 172.21.15.164:0/2426657622 >> v2:172.21.15.173:3300/0 conn(0x7f18680b4b20 0x7f186806c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:19.361 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.357+0000 7f186c9f6640 1 --2- 172.21.15.164:0/2426657622 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f186809def0 0x7f18680b45e0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:19.361 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.357+0000 7f186c9f6640 1 -- 172.21.15.164:0/2426657622 >> 172.21.15.164:0/2426657622 conn(0x7f18680b8ca0 msgr2=0x7f18680b90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:19.361 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.357+0000 7f186c9f6640 1 -- 172.21.15.164:0/2426657622 shutdown_connections 2024-03-23T14:57:19.361 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.357+0000 7f186c9f6640 1 -- 172.21.15.164:0/2426657622 wait complete. 2024-03-23T14:57:19.361 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.357+0000 7f186c9f6640 1 Processor -- start 2024-03-23T14:57:19.361 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.357+0000 7f186c9f6640 1 -- start start 2024-03-23T14:57:19.363 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.357+0000 7f186c9f6640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f186809def0 0x7f186809d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:19.363 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.357+0000 7f186c9f6640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f18680add50 0x7f186809b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:19.363 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.357+0000 7f186c9f6640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f18680b4b20 0x7f186813ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:19.363 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.357+0000 7f186c9f6640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f186809af70 con 0x7f18680add50 2024-03-23T14:57:19.363 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.357+0000 7f186c9f6640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f186809ad20 con 0x7f186809def0 2024-03-23T14:57:19.363 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.357+0000 7f186c9f6640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f1868071c20 con 0x7f18680b4b20 2024-03-23T14:57:19.363 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.357+0000 7f1865d9b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f18680add50 0x7f186809b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:19.364 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.357+0000 7f1865d9b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f18680add50 0x7f186809b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:60310/0 (socket says 172.21.15.164:60310) 2024-03-23T14:57:19.364 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.357+0000 7f1865d9b640 1 -- 172.21.15.164:0/1479086529 learned_addr learned my addr 172.21.15.164:0/1479086529 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:19.364 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.357+0000 7f186659c640 1 --2- 172.21.15.164:0/1479086529 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f186809def0 0x7f186809d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:19.364 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.357+0000 7f1866d9d640 1 --2- 172.21.15.164:0/1479086529 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f18680b4b20 0x7f186813ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:19.364 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.357+0000 7f1865d9b640 1 -- 172.21.15.164:0/1479086529 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f18680b4b20 msgr2=0x7f186813ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:19.364 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.357+0000 7f1865d9b640 1 --2- 172.21.15.164:0/1479086529 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f18680b4b20 0x7f186813ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:19.364 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.357+0000 7f1865d9b640 1 -- 172.21.15.164:0/1479086529 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f186809def0 msgr2=0x7f186809d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:19.364 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.357+0000 7f1865d9b640 1 --2- 172.21.15.164:0/1479086529 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f186809def0 0x7f186809d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:19.364 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.357+0000 7f1865d9b640 1 -- 172.21.15.164:0/1479086529 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1858009b30 con 0x7f18680add50 2024-03-23T14:57:19.364 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.357+0000 7f1865d9b640 1 --2- 172.21.15.164:0/1479086529 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f18680add50 0x7f186809b300 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f185000c480 tx=0x7f185000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:19.364 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.357+0000 7f185effd640 1 -- 172.21.15.164:0/1479086529 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1850012830 con 0x7f18680add50 2024-03-23T14:57:19.364 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.357+0000 7f185effd640 1 -- 172.21.15.164:0/1479086529 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18500129b0 con 0x7f18680add50 2024-03-23T14:57:19.364 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.357+0000 7f186c9f6640 1 -- 172.21.15.164:0/1479086529 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f186809cac0 con 0x7f18680add50 2024-03-23T14:57:19.364 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.357+0000 7f185effd640 1 -- 172.21.15.164:0/1479086529 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1850012d60 con 0x7f18680add50 2024-03-23T14:57:19.364 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.357+0000 7f186c9f6640 1 -- 172.21.15.164:0/1479086529 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18680720b0 con 0x7f18680add50 2024-03-23T14:57:19.365 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.361+0000 7f186c9f6640 1 -- 172.21.15.164:0/1479086529 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1868071750 con 0x7f18680add50 2024-03-23T14:57:19.365 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.361+0000 7f185effd640 1 -- 172.21.15.164:0/1479086529 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7f185001c460 con 0x7f18680add50 2024-03-23T14:57:19.365 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.361+0000 7f185effd640 1 --2- 172.21.15.164:0/1479086529 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f18480672d0 0x7f1848069790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:19.365 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.361+0000 7f186659c640 1 --2- 172.21.15.164:0/1479086529 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f18480672d0 0x7f1848069790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:19.365 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.361+0000 7f185effd640 1 -- 172.21.15.164:0/1479086529 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f1850084ec0 con 0x7f18680add50 2024-03-23T14:57:19.365 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.361+0000 7f186659c640 1 --2- 172.21.15.164:0/1479086529 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f18480672d0 0x7f1848069790 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f18580ae000 tx=0x7f18580a7070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:19.369 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.365+0000 7f185effd640 1 -- 172.21.15.164:0/1479086529 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7f1868071750 con 0x7f18680add50 2024-03-23T14:57:19.517 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.513+0000 7f186c9f6640 1 -- 172.21.15.164:0/1479086529 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1868071750 con 0x7f18680add50 2024-03-23T14:57:19.518 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.513+0000 7f185effd640 1 -- 172.21.15.164:0/1479086529 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f1868071750 con 0x7f18680add50 2024-03-23T14:57:19.518 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 1 2024-03-23T14:57:19.520 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.517+0000 7f186c9f6640 1 -- 172.21.15.164:0/1479086529 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f18480672d0 msgr2=0x7f1848069790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:19.521 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.517+0000 7f186c9f6640 1 --2- 172.21.15.164:0/1479086529 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f18480672d0 0x7f1848069790 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f18580ae000 tx=0x7f18580a7070 comp rx=0 tx=0).stop 2024-03-23T14:57:19.521 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.517+0000 7f186c9f6640 1 -- 172.21.15.164:0/1479086529 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f18680add50 msgr2=0x7f186809b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:19.521 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.517+0000 7f186c9f6640 1 --2- 172.21.15.164:0/1479086529 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f18680add50 0x7f186809b300 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f185000c480 tx=0x7f185000c940 comp rx=0 tx=0).stop 2024-03-23T14:57:19.521 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.517+0000 7f186c9f6640 1 -- 172.21.15.164:0/1479086529 shutdown_connections 2024-03-23T14:57:19.521 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.517+0000 7f186c9f6640 1 --2- 172.21.15.164:0/1479086529 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f18680b4b20 0x7f186813ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:19.521 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.517+0000 7f186c9f6640 1 --2- 172.21.15.164:0/1479086529 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f18680add50 0x7f186809b300 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:19.521 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.517+0000 7f186c9f6640 1 --2- 172.21.15.164:0/1479086529 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f18480672d0 0x7f1848069790 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:19.521 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.517+0000 7f186c9f6640 1 --2- 172.21.15.164:0/1479086529 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f186809def0 0x7f186809d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:19.521 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.517+0000 7f186c9f6640 1 -- 172.21.15.164:0/1479086529 >> 172.21.15.164:0/1479086529 conn(0x7f18680b8ca0 msgr2=0x7f1868145b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:19.521 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.517+0000 7f186c9f6640 1 -- 172.21.15.164:0/1479086529 shutdown_connections 2024-03-23T14:57:19.521 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.517+0000 7f186c9f6640 1 -- 172.21.15.164:0/1479086529 wait complete. 2024-03-23T14:57:19.536 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-23T14:57:19.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.653+0000 7ff1fe609640 1 Processor -- start 2024-03-23T14:57:19.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.653+0000 7ff1fe609640 1 -- start start 2024-03-23T14:57:19.661 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.657+0000 7ff1fe609640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7ff1f80b0230 0x7ff1f80b34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:19.661 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.657+0000 7ff1fe609640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7ff1f8148cf0 0x7ff1f8149180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:19.661 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.657+0000 7ff1fe609640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff1f80b1c70 0x7ff1f80b20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:19.661 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.657+0000 7ff1fe609640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7ff1f80b51d0 con 0x7ff1f8148cf0 2024-03-23T14:57:19.661 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.657+0000 7ff1fe609640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 -- 0x7ff1f80b0610 con 0x7ff1f80b0230 2024-03-23T14:57:19.661 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.657+0000 7ff1fe609640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7ff1f80b1500 con 0x7ff1f80b1c70 2024-03-23T14:57:19.661 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.657+0000 7ff1f77fe640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7ff1f8148cf0 0x7ff1f8149180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:19.661 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.657+0000 7ff1f77fe640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7ff1f8148cf0 0x7ff1f8149180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:60320/0 (socket says 172.21.15.164:60320) 2024-03-23T14:57:19.661 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.657+0000 7ff1f77fe640 1 -- 172.21.15.164:0/2812897096 learned_addr learned my addr 172.21.15.164:0/2812897096 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:19.661 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.657+0000 7ff1f7fff640 1 --2- 172.21.15.164:0/2812897096 >> v2:172.21.15.173:3300/0 conn(0x7ff1f80b0230 0x7ff1f80b34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:19.661 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.657+0000 7ff1fcba6640 1 --2- 172.21.15.164:0/2812897096 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff1f80b1c70 0x7ff1f80b20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:19.661 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.657+0000 7ff1f77fe640 1 -- 172.21.15.164:0/2812897096 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff1f80b1c70 msgr2=0x7ff1f80b20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:19.661 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.657+0000 7ff1f77fe640 1 --2- 172.21.15.164:0/2812897096 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff1f80b1c70 0x7ff1f80b20e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:19.661 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.657+0000 7ff1f77fe640 1 -- 172.21.15.164:0/2812897096 >> v2:172.21.15.173:3300/0 conn(0x7ff1f80b0230 msgr2=0x7ff1f80b34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:19.662 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.657+0000 7ff1f77fe640 1 --2- 172.21.15.164:0/2812897096 >> v2:172.21.15.173:3300/0 conn(0x7ff1f80b0230 0x7ff1f80b34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:19.662 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.657+0000 7ff1f77fe640 1 -- 172.21.15.164:0/2812897096 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1f806d250 con 0x7ff1f8148cf0 2024-03-23T14:57:19.662 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.657+0000 7ff1f7fff640 1 --2- 172.21.15.164:0/2812897096 >> v2:172.21.15.173:3300/0 conn(0x7ff1f80b0230 0x7ff1f80b34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T14:57:19.663 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.657+0000 7ff1f77fe640 1 --2- 172.21.15.164:0/2812897096 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff1f8148cf0 0x7ff1f8149180 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ff1f806e8c0 tx=0x7ff1e801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3b613b43eba2745 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:19.663 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.657+0000 7ff1f6ffd640 1 -- 172.21.15.164:0/2812897096 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e80ab620 con 0x7ff1f8148cf0 2024-03-23T14:57:19.663 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.657+0000 7ff1f6ffd640 1 -- 172.21.15.164:0/2812897096 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1e80ab780 con 0x7ff1f8148cf0 2024-03-23T14:57:19.664 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.657+0000 7ff1f6ffd640 1 -- 172.21.15.164:0/2812897096 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e80abb10 con 0x7ff1f8148cf0 2024-03-23T14:57:19.664 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.657+0000 7ff1fe609640 1 -- 172.21.15.164:0/2812897096 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff1f8148cf0 msgr2=0x7ff1f8149180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:19.664 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.657+0000 7ff1fe609640 1 --2- 172.21.15.164:0/2812897096 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff1f8148cf0 0x7ff1f8149180 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ff1f806e8c0 tx=0x7ff1e801c460 comp rx=0 tx=0).stop 2024-03-23T14:57:19.664 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.657+0000 7ff1fe609640 1 -- 172.21.15.164:0/2812897096 shutdown_connections 2024-03-23T14:57:19.664 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.657+0000 7ff1fe609640 1 --2- 172.21.15.164:0/2812897096 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff1f80b1c70 0x7ff1f80b20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:19.664 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.657+0000 7ff1fe609640 1 --2- 172.21.15.164:0/2812897096 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff1f8148cf0 0x7ff1f8149180 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:19.664 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.657+0000 7ff1fe609640 1 --2- 172.21.15.164:0/2812897096 >> v2:172.21.15.173:3300/0 conn(0x7ff1f80b0230 0x7ff1f80b34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:19.664 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.657+0000 7ff1fe609640 1 -- 172.21.15.164:0/2812897096 >> 172.21.15.164:0/2812897096 conn(0x7ff1f80b8ca0 msgr2=0x7ff1f80b90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:19.664 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.657+0000 7ff1fe609640 1 -- 172.21.15.164:0/2812897096 shutdown_connections 2024-03-23T14:57:19.664 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.657+0000 7ff1fe609640 1 -- 172.21.15.164:0/2812897096 wait complete. 2024-03-23T14:57:19.665 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.661+0000 7ff1fe609640 1 Processor -- start 2024-03-23T14:57:19.665 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.661+0000 7ff1fe609640 1 -- start start 2024-03-23T14:57:19.666 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.661+0000 7ff1fe609640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff1f80b0230 0x7ff1f813e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:19.666 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.661+0000 7ff1fe609640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff1f80b1c70 0x7ff1f8070640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:19.666 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.661+0000 7ff1fe609640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff1f8148cf0 0x7ff1f813b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:19.666 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.661+0000 7ff1fe609640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f813f050 con 0x7ff1f8148cf0 2024-03-23T14:57:19.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.661+0000 7ff1fe609640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f8071070 con 0x7ff1f80b1c70 2024-03-23T14:57:19.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.661+0000 7ff1fe609640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7ff1f8071350 con 0x7ff1f80b0230 2024-03-23T14:57:19.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.661+0000 7ff1fcba6640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff1f8148cf0 0x7ff1f813b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:19.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.661+0000 7ff1fcba6640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff1f8148cf0 0x7ff1f813b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:60322/0 (socket says 172.21.15.164:60322) 2024-03-23T14:57:19.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.661+0000 7ff1fcba6640 1 -- 172.21.15.164:0/553486742 learned_addr learned my addr 172.21.15.164:0/553486742 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:19.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.661+0000 7ff1f77fe640 1 --2- 172.21.15.164:0/553486742 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff1f80b1c70 0x7ff1f8070640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:19.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.661+0000 7ff1f7fff640 1 --2- 172.21.15.164:0/553486742 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff1f80b0230 0x7ff1f813e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:19.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.661+0000 7ff1fcba6640 1 -- 172.21.15.164:0/553486742 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff1f80b0230 msgr2=0x7ff1f813e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:19.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.661+0000 7ff1fcba6640 1 --2- 172.21.15.164:0/553486742 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff1f80b0230 0x7ff1f813e280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:19.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.661+0000 7ff1fcba6640 1 -- 172.21.15.164:0/553486742 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff1f80b1c70 msgr2=0x7ff1f8070640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:19.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.661+0000 7ff1fcba6640 1 --2- 172.21.15.164:0/553486742 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff1f80b1c70 0x7ff1f8070640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:19.668 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.661+0000 7ff1fcba6640 1 -- 172.21.15.164:0/553486742 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1e80094d0 con 0x7ff1f8148cf0 2024-03-23T14:57:19.668 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.661+0000 7ff1f77fe640 1 --2- 172.21.15.164:0/553486742 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff1f80b1c70 0x7ff1f8070640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T14:57:19.668 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.661+0000 7ff1fcba6640 1 --2- 172.21.15.164:0/553486742 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff1f8148cf0 0x7ff1f813b550 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ff1f0007e40 tx=0x7ff1f000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:19.668 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.661+0000 7ff1f4ff9640 1 -- 172.21.15.164:0/553486742 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1f0014620 con 0x7ff1f8148cf0 2024-03-23T14:57:19.668 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.661+0000 7ff1f4ff9640 1 -- 172.21.15.164:0/553486742 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1f000ee90 con 0x7ff1f8148cf0 2024-03-23T14:57:19.668 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.661+0000 7ff1fe609640 1 -- 172.21.15.164:0/553486742 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1f8071f00 con 0x7ff1f8148cf0 2024-03-23T14:57:19.668 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.661+0000 7ff1f4ff9640 1 -- 172.21.15.164:0/553486742 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1f00149e0 con 0x7ff1f8148cf0 2024-03-23T14:57:19.668 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.661+0000 7ff1fe609640 1 -- 172.21.15.164:0/553486742 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1f8070d50 con 0x7ff1f8148cf0 2024-03-23T14:57:19.669 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.661+0000 7ff1fe609640 1 -- 172.21.15.164:0/553486742 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1f8072140 con 0x7ff1f8148cf0 2024-03-23T14:57:19.669 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.661+0000 7ff1f4ff9640 1 -- 172.21.15.164:0/553486742 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7ff1f0014b40 con 0x7ff1f8148cf0 2024-03-23T14:57:19.669 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.665+0000 7ff1f4ff9640 1 --2- 172.21.15.164:0/553486742 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7ff1e40672a0 0x7ff1e4069760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:19.669 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.665+0000 7ff1f4ff9640 1 -- 172.21.15.164:0/553486742 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ff1f0085980 con 0x7ff1f8148cf0 2024-03-23T14:57:19.669 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.665+0000 7ff1f7fff640 1 --2- 172.21.15.164:0/553486742 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7ff1e40672a0 0x7ff1e4069760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:19.669 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.665+0000 7ff1f7fff640 1 --2- 172.21.15.164:0/553486742 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7ff1e40672a0 0x7ff1e4069760 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec00d8b0 tx=0x7ff1ec008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:19.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.669+0000 7ff1f4ff9640 1 -- 172.21.15.164:0/553486742 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7ff1f8072140 con 0x7ff1f8148cf0 2024-03-23T14:57:19.831 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.825+0000 7ff1fe609640 1 -- 172.21.15.164:0/553486742 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ff1f8072140 con 0x7ff1f8148cf0 2024-03-23T14:57:19.831 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.825+0000 7ff1f4ff9640 1 -- 172.21.15.164:0/553486742 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7ff1f8072140 con 0x7ff1f8148cf0 2024-03-23T14:57:19.831 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 32 2024-03-23T14:57:19.834 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.829+0000 7ff1fe609640 1 -- 172.21.15.164:0/553486742 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7ff1e40672a0 msgr2=0x7ff1e4069760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:19.834 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.829+0000 7ff1fe609640 1 --2- 172.21.15.164:0/553486742 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7ff1e40672a0 0x7ff1e4069760 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec00d8b0 tx=0x7ff1ec008040 comp rx=0 tx=0).stop 2024-03-23T14:57:19.834 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.829+0000 7ff1fe609640 1 -- 172.21.15.164:0/553486742 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff1f8148cf0 msgr2=0x7ff1f813b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:19.834 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.829+0000 7ff1fe609640 1 --2- 172.21.15.164:0/553486742 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff1f8148cf0 0x7ff1f813b550 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ff1f0007e40 tx=0x7ff1f000e4b0 comp rx=0 tx=0).stop 2024-03-23T14:57:19.834 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.829+0000 7ff1fe609640 1 -- 172.21.15.164:0/553486742 shutdown_connections 2024-03-23T14:57:19.834 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.829+0000 7ff1fe609640 1 --2- 172.21.15.164:0/553486742 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff1f8148cf0 0x7ff1f813b550 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:19.834 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.829+0000 7ff1fe609640 1 --2- 172.21.15.164:0/553486742 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7ff1e40672a0 0x7ff1e4069760 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:19.834 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.829+0000 7ff1fe609640 1 --2- 172.21.15.164:0/553486742 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff1f80b1c70 0x7ff1f8070640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:19.834 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.829+0000 7ff1fe609640 1 --2- 172.21.15.164:0/553486742 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff1f80b0230 0x7ff1f813e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:19.834 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.829+0000 7ff1fe609640 1 -- 172.21.15.164:0/553486742 >> 172.21.15.164:0/553486742 conn(0x7ff1f80b8ca0 msgr2=0x7ff1f80b5cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:19.834 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.829+0000 7ff1fe609640 1 -- 172.21.15.164:0/553486742 shutdown_connections 2024-03-23T14:57:19.834 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.829+0000 7ff1fe609640 1 -- 172.21.15.164:0/553486742 wait complete. 2024-03-23T14:57:19.850 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-23T14:57:19.974 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.969+0000 7f3534937640 1 Processor -- start 2024-03-23T14:57:19.974 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.969+0000 7f3534937640 1 -- start start 2024-03-23T14:57:19.976 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.969+0000 7f3534937640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f353009def0 0x7f35300b45e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:19.976 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.969+0000 7f3534937640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f35300add50 0x7f353006bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:19.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.969+0000 7f3534937640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f353006d650 con 0x7f35300b4b20 2024-03-23T14:57:19.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.969+0000 7f3534937640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 -- 0x7f35300af4a0 con 0x7f35300add50 2024-03-23T14:57:19.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.969+0000 7f3534937640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f353009e2d0 con 0x7f353009def0 2024-03-23T14:57:19.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.969+0000 7f352ed9d640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f35300b4b20 0x7f353006c4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:56890/0 (socket says 172.21.15.164:56890) 2024-03-23T14:57:19.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.969+0000 7f352ed9d640 1 -- 172.21.15.164:0/3505678588 learned_addr learned my addr 172.21.15.164:0/3505678588 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-03-23T14:57:19.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.969+0000 7f352e59c640 1 --2- 172.21.15.164:0/3505678588 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f353009def0 0x7f35300b45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:19.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.969+0000 7f352dd9b640 1 --2- 172.21.15.164:0/3505678588 >> v2:172.21.15.173:3300/0 conn(0x7f35300add50 0x7f353006bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:19.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.969+0000 7f352d59a640 1 -- 172.21.15.164:0/3505678588 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1270222000 0 0) 0x7f353006d650 con 0x7f35300b4b20 2024-03-23T14:57:19.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.969+0000 7f352d59a640 1 -- 172.21.15.164:0/3505678588 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3518005740 con 0x7f35300b4b20 2024-03-23T14:57:19.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.969+0000 7f352d59a640 1 -- 172.21.15.164:0/3505678588 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3951215539 0 0) 0x7f3518005740 con 0x7f35300b4b20 2024-03-23T14:57:19.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.969+0000 7f352d59a640 1 -- 172.21.15.164:0/3505678588 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f353009def0 msgr2=0x7f35300b45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:19.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.969+0000 7f352d59a640 1 --2- 172.21.15.164:0/3505678588 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f353009def0 0x7f35300b45e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:19.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.969+0000 7f352d59a640 1 -- 172.21.15.164:0/3505678588 >> v2:172.21.15.173:3300/0 conn(0x7f35300add50 msgr2=0x7f353006bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:19.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.969+0000 7f352d59a640 1 --2- 172.21.15.164:0/3505678588 >> v2:172.21.15.173:3300/0 conn(0x7f35300add50 0x7f353006bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:19.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.969+0000 7f352d59a640 1 -- 172.21.15.164:0/3505678588 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35300aeca0 con 0x7f35300b4b20 2024-03-23T14:57:19.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.969+0000 7f352d59a640 1 -- 172.21.15.164:0/3505678588 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2526390548 0 0) 0x7f3528001a00 con 0x7f35300b4b20 2024-03-23T14:57:19.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.969+0000 7f352d59a640 1 -- 172.21.15.164:0/3505678588 >> v1:172.21.15.164:6789/0 conn(0x7f35300b4b20 legacy=0x7f353006c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:19.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.969+0000 7f352d59a640 1 --2- 172.21.15.164:0/3505678588 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f353009def0 0x7f3518005b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:19.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.969+0000 7f352d59a640 1 --2- 172.21.15.164:0/3505678588 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3518006110 0x7f35180069e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:19.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.973+0000 7f352d59a640 1 --2- 172.21.15.164:0/3505678588 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3518006f20 0x7f35180073a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:19.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.973+0000 7f352d59a640 1 -- 172.21.15.164:0/3505678588 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f35300af4a0 con 0x7f3518006f20 2024-03-23T14:57:19.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.973+0000 7f352d59a640 1 -- 172.21.15.164:0/3505678588 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f353009e2d0 con 0x7f3518006110 2024-03-23T14:57:19.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.973+0000 7f352d59a640 1 -- 172.21.15.164:0/3505678588 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f351800dc80 con 0x7f353009def0 2024-03-23T14:57:19.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.973+0000 7f352ed9d640 1 --2- 172.21.15.164:0/3505678588 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3518006f20 0x7f35180073a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:19.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.973+0000 7f352e59c640 1 --2- 172.21.15.164:0/3505678588 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f353009def0 0x7f3518005b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:19.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.973+0000 7f352dd9b640 1 --2- 172.21.15.164:0/3505678588 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3518006110 0x7f35180069e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:19.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.973+0000 7f352ed9d640 1 -- 172.21.15.164:0/3505678588 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f353009def0 msgr2=0x7f3518005b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:19.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.973+0000 7f352ed9d640 1 --2- 172.21.15.164:0/3505678588 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f353009def0 0x7f3518005b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:19.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.973+0000 7f352ed9d640 1 -- 172.21.15.164:0/3505678588 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3518006110 msgr2=0x7f35180069e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:19.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.973+0000 7f352ed9d640 1 --2- 172.21.15.164:0/3505678588 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3518006110 0x7f35180069e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:19.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.973+0000 7f352ed9d640 1 -- 172.21.15.164:0/3505678588 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3518005740 con 0x7f3518006f20 2024-03-23T14:57:19.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.973+0000 7f352dd9b640 1 --2- 172.21.15.164:0/3505678588 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3518006110 0x7f35180069e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T14:57:19.979 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.973+0000 7f352ed9d640 1 --2- 172.21.15.164:0/3505678588 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3518006f20 0x7f35180073a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f352800a860 tx=0x7f352801d380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f57db46f49af04ae server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:19.979 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.973+0000 7f352d59a640 1 -- 172.21.15.164:0/3505678588 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35280a5ea0 con 0x7f3518006f20 2024-03-23T14:57:19.980 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.973+0000 7f352d59a640 1 -- 172.21.15.164:0/3505678588 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35280a8070 con 0x7f3518006f20 2024-03-23T14:57:19.980 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.973+0000 7f352d59a640 1 -- 172.21.15.164:0/3505678588 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35280ad8d0 con 0x7f3518006f20 2024-03-23T14:57:19.980 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.973+0000 7f3534937640 1 -- 172.21.15.164:0/3505678588 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3518006f20 msgr2=0x7f35180073a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:19.980 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.973+0000 7f3534937640 1 --2- 172.21.15.164:0/3505678588 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3518006f20 0x7f35180073a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f352800a860 tx=0x7f352801d380 comp rx=0 tx=0).stop 2024-03-23T14:57:19.980 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.973+0000 7f352ed9d640 1 -- 172.21.15.164:0/3505678588 reap_dead start 2024-03-23T14:57:19.980 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.973+0000 7f3534937640 1 -- 172.21.15.164:0/3505678588 shutdown_connections 2024-03-23T14:57:19.980 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.973+0000 7f3534937640 1 -- 172.21.15.164:0/3505678588 >> 172.21.15.164:0/3505678588 conn(0x7f35300b8ca0 msgr2=0x7f35300b90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:19.980 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.973+0000 7f3534937640 1 -- 172.21.15.164:0/3505678588 shutdown_connections 2024-03-23T14:57:19.980 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.973+0000 7f3534937640 1 -- 172.21.15.164:0/3505678588 wait complete. 2024-03-23T14:57:19.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.977+0000 7f3534937640 1 Processor -- start 2024-03-23T14:57:19.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.977+0000 7f3534937640 1 -- start start 2024-03-23T14:57:19.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.977+0000 7f3534937640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f35300b1350 0x7f35301db8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:19.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.977+0000 7f3534937640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f35301dbe30 0x7f353013fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:19.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.977+0000 7f3534937640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3530140230 0x7f35301de830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:19.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.977+0000 7f3534937640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f35301dc2b0 con 0x7f3530140230 2024-03-23T14:57:19.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.977+0000 7f3534937640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f35300b1ec0 con 0x7f35300b1350 2024-03-23T14:57:19.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.977+0000 7f3534937640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f35300b1ba0 con 0x7f35301dbe30 2024-03-23T14:57:19.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.977+0000 7f352ed9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3530140230 0x7f35301de830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:19.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.977+0000 7f352ed9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3530140230 0x7f35301de830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:60346/0 (socket says 172.21.15.164:60346) 2024-03-23T14:57:19.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.977+0000 7f352ed9d640 1 -- 172.21.15.164:0/3717623148 learned_addr learned my addr 172.21.15.164:0/3717623148 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:19.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.977+0000 7f352e59c640 1 --2- 172.21.15.164:0/3717623148 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f35300b1350 0x7f35301db8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:19.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.977+0000 7f352dd9b640 1 --2- 172.21.15.164:0/3717623148 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f35301dbe30 0x7f353013fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:19.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.977+0000 7f352ed9d640 1 -- 172.21.15.164:0/3717623148 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f35301dbe30 msgr2=0x7f353013fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:19.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.977+0000 7f352ed9d640 1 --2- 172.21.15.164:0/3717623148 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f35301dbe30 0x7f353013fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:19.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.977+0000 7f352ed9d640 1 -- 172.21.15.164:0/3717623148 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f35300b1350 msgr2=0x7f35301db8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:19.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.977+0000 7f352ed9d640 1 --2- 172.21.15.164:0/3717623148 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f35300b1350 0x7f35301db8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:19.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.977+0000 7f352ed9d640 1 -- 172.21.15.164:0/3717623148 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35300700e0 con 0x7f3530140230 2024-03-23T14:57:19.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.977+0000 7f352ed9d640 1 --2- 172.21.15.164:0/3717623148 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3530140230 0x7f35301de830 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f352801dc20 tx=0x7f3518007a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:19.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.977+0000 7f351effd640 1 -- 172.21.15.164:0/3717623148 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35280a5ea0 con 0x7f3530140230 2024-03-23T14:57:19.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.977+0000 7f351effd640 1 -- 172.21.15.164:0/3717623148 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3528001fe0 con 0x7f3530140230 2024-03-23T14:57:19.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.977+0000 7f3534937640 1 -- 172.21.15.164:0/3717623148 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35301db3d0 con 0x7f3530140230 2024-03-23T14:57:19.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.977+0000 7f351effd640 1 -- 172.21.15.164:0/3717623148 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35280019c0 con 0x7f3530140230 2024-03-23T14:57:19.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.977+0000 7f3534937640 1 -- 172.21.15.164:0/3717623148 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35301406b0 con 0x7f3530140230 2024-03-23T14:57:19.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.977+0000 7f3534937640 1 -- 172.21.15.164:0/3717623148 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35300690a0 con 0x7f3530140230 2024-03-23T14:57:19.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.977+0000 7f351effd640 1 -- 172.21.15.164:0/3717623148 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7f35280aa030 con 0x7f3530140230 2024-03-23T14:57:19.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.977+0000 7f351effd640 1 --2- 172.21.15.164:0/3717623148 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f35100672a0 0x7f3510069760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:19.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.977+0000 7f352e59c640 1 --2- 172.21.15.164:0/3717623148 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f35100672a0 0x7f3510069760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:19.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.977+0000 7f351effd640 1 -- 172.21.15.164:0/3717623148 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f352811e0e0 con 0x7f3530140230 2024-03-23T14:57:19.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.981+0000 7f352e59c640 1 --2- 172.21.15.164:0/3717623148 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f35100672a0 0x7f3510069760 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f3520009a30 tx=0x7f3520008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:19.987 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:19.981+0000 7f351effd640 1 -- 172.21.15.164:0/3717623148 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7f35280ead50 con 0x7f3530140230 2024-03-23T14:57:20.135 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.129+0000 7f3534937640 1 -- 172.21.15.164:0/3717623148 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f3530145270 con 0x7f3530140230 2024-03-23T14:57:20.135 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.129+0000 7f351effd640 1 -- 172.21.15.164:0/3717623148 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f35280ee410 con 0x7f3530140230 2024-03-23T14:57:20.135 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 64 2024-03-23T14:57:20.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.133+0000 7f3534937640 1 -- 172.21.15.164:0/3717623148 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f35100672a0 msgr2=0x7f3510069760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:20.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.133+0000 7f3534937640 1 --2- 172.21.15.164:0/3717623148 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f35100672a0 0x7f3510069760 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f3520009a30 tx=0x7f3520008040 comp rx=0 tx=0).stop 2024-03-23T14:57:20.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.133+0000 7f3534937640 1 -- 172.21.15.164:0/3717623148 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3530140230 msgr2=0x7f35301de830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:20.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.133+0000 7f3534937640 1 --2- 172.21.15.164:0/3717623148 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3530140230 0x7f35301de830 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f352801dc20 tx=0x7f3518007a10 comp rx=0 tx=0).stop 2024-03-23T14:57:20.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.133+0000 7f3534937640 1 -- 172.21.15.164:0/3717623148 shutdown_connections 2024-03-23T14:57:20.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.133+0000 7f3534937640 1 --2- 172.21.15.164:0/3717623148 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3530140230 0x7f35301de830 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:20.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.133+0000 7f3534937640 1 --2- 172.21.15.164:0/3717623148 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f35301dbe30 0x7f353013fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:20.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.133+0000 7f3534937640 1 --2- 172.21.15.164:0/3717623148 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f35100672a0 0x7f3510069760 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:20.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.133+0000 7f3534937640 1 --2- 172.21.15.164:0/3717623148 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f35300b1350 0x7f35301db8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:20.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.133+0000 7f3534937640 1 -- 172.21.15.164:0/3717623148 >> 172.21.15.164:0/3717623148 conn(0x7f35300b8ca0 msgr2=0x7f3530144a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:20.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.133+0000 7f3534937640 1 -- 172.21.15.164:0/3717623148 shutdown_connections 2024-03-23T14:57:20.138 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.133+0000 7f3534937640 1 -- 172.21.15.164:0/3717623148 wait complete. 2024-03-23T14:57:20.153 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T14:57:20.263 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.257+0000 7f7dec86a640 1 Processor -- start 2024-03-23T14:57:20.263 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.257+0000 7f7dec86a640 1 -- start start 2024-03-23T14:57:20.264 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.257+0000 7f7dec86a640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f7de80b0230 0x7f7de80b34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:20.264 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.257+0000 7f7dec86a640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f7de814db70 0x7f7de814ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:20.264 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.257+0000 7f7dec86a640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7de80b51d0 con 0x7f7de8148cf0 2024-03-23T14:57:20.264 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.257+0000 7f7dec86a640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f7de80b0610 con 0x7f7de80b0230 2024-03-23T14:57:20.264 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.257+0000 7f7dec86a640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 -- 0x7f7de80b1500 con 0x7f7de814db70 2024-03-23T14:57:20.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.261+0000 7f7de5d9b640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f7de8148cf0 0x7f7de8149180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:56894/0 (socket says 172.21.15.164:56894) 2024-03-23T14:57:20.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.261+0000 7f7de5d9b640 1 -- 172.21.15.164:0/191131072 learned_addr learned my addr 172.21.15.164:0/191131072 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-03-23T14:57:20.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.261+0000 7f7de659c640 1 --2- 172.21.15.164:0/191131072 >> v2:172.21.15.164:3300/0 conn(0x7f7de80b0230 0x7f7de80b34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:20.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.261+0000 7f7de6d9d640 1 --2- 172.21.15.164:0/191131072 >> v2:172.21.15.173:3300/0 conn(0x7f7de814db70 0x7f7de814ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:20.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.261+0000 7f7de559a640 1 -- 172.21.15.164:0/191131072 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3139461926 0 0) 0x7f7de80b51d0 con 0x7f7de8148cf0 2024-03-23T14:57:20.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.261+0000 7f7de559a640 1 -- 172.21.15.164:0/191131072 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7dd0003530 con 0x7f7de8148cf0 2024-03-23T14:57:20.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.261+0000 7f7de659c640 1 -- 172.21.15.164:0/191131072 >> v2:172.21.15.173:3300/0 conn(0x7f7de814db70 msgr2=0x7f7de814ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:20.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.261+0000 7f7de659c640 1 --2- 172.21.15.164:0/191131072 >> v2:172.21.15.173:3300/0 conn(0x7f7de814db70 0x7f7de814ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:20.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.261+0000 7f7de659c640 1 -- 172.21.15.164:0/191131072 >> v1:172.21.15.164:6789/0 conn(0x7f7de8148cf0 legacy=0x7f7de8149180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:20.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.261+0000 7f7de659c640 1 -- 172.21.15.164:0/191131072 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7de806d250 con 0x7f7de80b0230 2024-03-23T14:57:20.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.261+0000 7f7de659c640 1 --2- 172.21.15.164:0/191131072 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7de80b0230 0x7f7de80b34d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f7de81498c0 tx=0x7f7ddc01c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fec43ae88ee219d1 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:20.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.261+0000 7f7de559a640 1 -- 172.21.15.164:0/191131072 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ddc0ab600 con 0x7f7de80b0230 2024-03-23T14:57:20.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.261+0000 7f7de559a640 1 -- 172.21.15.164:0/191131072 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ddc0a7050 con 0x7f7de80b0230 2024-03-23T14:57:20.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.261+0000 7f7de559a640 1 -- 172.21.15.164:0/191131072 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ddc0ab980 con 0x7f7de80b0230 2024-03-23T14:57:20.267 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.261+0000 7f7dec86a640 1 -- 172.21.15.164:0/191131072 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7de80b0230 msgr2=0x7f7de80b34d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:20.267 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.261+0000 7f7dec86a640 1 --2- 172.21.15.164:0/191131072 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7de80b0230 0x7f7de80b34d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f7de81498c0 tx=0x7f7ddc01c330 comp rx=0 tx=0).stop 2024-03-23T14:57:20.267 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.261+0000 7f7dec86a640 1 -- 172.21.15.164:0/191131072 shutdown_connections 2024-03-23T14:57:20.267 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.261+0000 7f7dec86a640 1 --2- 172.21.15.164:0/191131072 >> v2:172.21.15.173:3300/0 conn(0x7f7de814db70 0x7f7de814ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:20.267 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.261+0000 7f7dec86a640 1 --2- 172.21.15.164:0/191131072 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7de80b0230 0x7f7de80b34d0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:20.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.261+0000 7f7dec86a640 1 -- 172.21.15.164:0/191131072 >> 172.21.15.164:0/191131072 conn(0x7f7de80b8ca0 msgr2=0x7f7de80b90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:20.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.261+0000 7f7dec86a640 1 -- 172.21.15.164:0/191131072 shutdown_connections 2024-03-23T14:57:20.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.261+0000 7f7dec86a640 1 -- 172.21.15.164:0/191131072 wait complete. 2024-03-23T14:57:20.269 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.265+0000 7f7dec86a640 1 Processor -- start 2024-03-23T14:57:20.269 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.265+0000 7f7dec86a640 1 -- start start 2024-03-23T14:57:20.269 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.265+0000 7f7dec86a640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7de80b0230 0x7f7de80706a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:20.269 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.265+0000 7f7dec86a640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7de8148cf0 0x7f7de80718c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:20.269 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.265+0000 7f7dec86a640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7de814db70 0x7f7de809aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:20.269 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.265+0000 7f7dec86a640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f7de809d5e0 con 0x7f7de814db70 2024-03-23T14:57:20.269 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.265+0000 7f7dec86a640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f7de809cff0 con 0x7f7de80b0230 2024-03-23T14:57:20.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.265+0000 7f7dec86a640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f7de8070be0 con 0x7f7de8148cf0 2024-03-23T14:57:20.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.265+0000 7f7de6d9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7de814db70 0x7f7de809aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:20.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.265+0000 7f7de6d9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7de814db70 0x7f7de809aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:60366/0 (socket says 172.21.15.164:60366) 2024-03-23T14:57:20.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.265+0000 7f7de6d9d640 1 -- 172.21.15.164:0/1468854924 learned_addr learned my addr 172.21.15.164:0/1468854924 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:20.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.265+0000 7f7de5d9b640 1 --2- 172.21.15.164:0/1468854924 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7de8148cf0 0x7f7de80718c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:20.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.265+0000 7f7de659c640 1 --2- 172.21.15.164:0/1468854924 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7de80b0230 0x7f7de80706a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:20.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.265+0000 7f7de6d9d640 1 -- 172.21.15.164:0/1468854924 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7de8148cf0 msgr2=0x7f7de80718c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:20.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.265+0000 7f7de6d9d640 1 --2- 172.21.15.164:0/1468854924 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7de8148cf0 0x7f7de80718c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:20.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.265+0000 7f7de6d9d640 1 -- 172.21.15.164:0/1468854924 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7de80b0230 msgr2=0x7f7de80706a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:20.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.265+0000 7f7de6d9d640 1 --2- 172.21.15.164:0/1468854924 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7de80b0230 0x7f7de80706a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:20.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.265+0000 7f7de6d9d640 1 -- 172.21.15.164:0/1468854924 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ddc009420 con 0x7f7de814db70 2024-03-23T14:57:20.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.265+0000 7f7de6d9d640 1 --2- 172.21.15.164:0/1468854924 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7de814db70 0x7f7de809aa50 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f7de0007e90 tx=0x7f7de000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:20.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.265+0000 7f7dd6ffd640 1 -- 172.21.15.164:0/1468854924 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de0014420 con 0x7f7de814db70 2024-03-23T14:57:20.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.265+0000 7f7dd6ffd640 1 -- 172.21.15.164:0/1468854924 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7de0014580 con 0x7f7de814db70 2024-03-23T14:57:20.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.265+0000 7f7dd6ffd640 1 -- 172.21.15.164:0/1468854924 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de0014930 con 0x7f7de814db70 2024-03-23T14:57:20.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.265+0000 7f7dec86a640 1 -- 172.21.15.164:0/1468854924 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7de8070e60 con 0x7f7de814db70 2024-03-23T14:57:20.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.265+0000 7f7dec86a640 1 -- 172.21.15.164:0/1468854924 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7de809d270 con 0x7f7de814db70 2024-03-23T14:57:20.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.265+0000 7f7dec86a640 1 -- 172.21.15.164:0/1468854924 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7de809d790 con 0x7f7de814db70 2024-03-23T14:57:20.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.265+0000 7f7dd6ffd640 1 -- 172.21.15.164:0/1468854924 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7f7de0014a90 con 0x7f7de814db70 2024-03-23T14:57:20.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.265+0000 7f7dd6ffd640 1 --2- 172.21.15.164:0/1468854924 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f7dc80672a0 0x7f7dc8069760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:20.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.265+0000 7f7dd6ffd640 1 -- 172.21.15.164:0/1468854924 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f7de0085860 con 0x7f7de814db70 2024-03-23T14:57:20.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.265+0000 7f7de659c640 1 --2- 172.21.15.164:0/1468854924 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f7dc80672a0 0x7f7dc8069760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:20.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.269+0000 7f7de659c640 1 --2- 172.21.15.164:0/1468854924 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f7dc80672a0 0x7f7dc8069760 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc0ad000 tx=0x7f7ddc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:20.275 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.269+0000 7f7dd6ffd640 1 -- 172.21.15.164:0/1468854924 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7f7de809d790 con 0x7f7de814db70 2024-03-23T14:57:20.431 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.425+0000 7f7dec86a640 1 -- 172.21.15.164:0/1468854924 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7de80690a0 con 0x7f7de814db70 2024-03-23T14:57:20.434 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.429+0000 7f7dd6ffd640 1 -- 172.21.15.164:0/1468854924 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16629 (secure 0 0 0) 0x7f7de0055fd0 con 0x7f7de814db70 2024-03-23T14:57:20.434 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-23T14:57:20.434 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":20,"fsid":"45c7911b-b69d-4e68-907b-fbfeb780aa3b","created":"2024-03-23T14:56:34.367633+0000","modified":"2024-03-23T14:57:00.611265+0000","last_up_change":"2024-03-23T14:56:46.297403+0000","last_in_change":"2024-03-23T14:56:36.629884+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T14:56:47.186157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-23T14:56:56.994057+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-23T14:56:57.953862+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"80abdc62-bd0d-4e60-a554-7ed6fd67cf54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":984716690},{"type":"v1","addr":"172.21.15.164:6801","nonce":984716690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":984716690},{"type":"v1","addr":"172.21.15.164:6803","nonce":984716690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":984716690},{"type":"v1","addr":"172.21.15.164:6807","nonce":984716690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":984716690},{"type":"v1","addr":"172.21.15.164:6805","nonce":984716690}]},"public_addr":"172.21.15.164:6801/984716690","cluster_addr":"172.21.15.164:6803/984716690","heartbeat_back_addr":"172.21.15.164:6807/984716690","heartbeat_front_addr":"172.21.15.164:6805/984716690","state":["exists","up"]},{"osd":1,"uuid":"f9001b06-be4d-4f42-a0e6-bcef2afa0d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":1349657239},{"type":"v1","addr":"172.21.15.164:6817","nonce":1349657239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":1349657239},{"type":"v1","addr":"172.21.15.164:6819","nonce":1349657239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":1349657239},{"type":"v1","addr":"172.21.15.164:6823","nonce":1349657239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":1349657239},{"type":"v1","addr":"172.21.15.164:6821","nonce":1349657239}]},"public_addr":"172.21.15.164:6817/1349657239","cluster_addr":"172.21.15.164:6819/1349657239","heartbeat_back_addr":"172.21.15.164:6823/1349657239","heartbeat_front_addr":"172.21.15.164:6821/1349657239","state":["exists","up"]},{"osd":2,"uuid":"fa0aeb9f-8b54-4831-a188-3e361c7980d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":887861252},{"type":"v1","addr":"172.21.15.164:6809","nonce":887861252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":887861252},{"type":"v1","addr":"172.21.15.164:6811","nonce":887861252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":887861252},{"type":"v1","addr":"172.21.15.164:6815","nonce":887861252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":887861252},{"type":"v1","addr":"172.21.15.164:6813","nonce":887861252}]},"public_addr":"172.21.15.164:6809/887861252","cluster_addr":"172.21.15.164:6811/887861252","heartbeat_back_addr":"172.21.15.164:6815/887861252","heartbeat_front_addr":"172.21.15.164:6813/887861252","state":["exists","up"]},{"osd":3,"uuid":"7e18c957-3091-46d5-8e17-8f3354c1726c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":812540271},{"type":"v1","addr":"172.21.15.164:6825","nonce":812540271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":812540271},{"type":"v1","addr":"172.21.15.164:6827","nonce":812540271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":812540271},{"type":"v1","addr":"172.21.15.164:6831","nonce":812540271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":812540271},{"type":"v1","addr":"172.21.15.164:6829","nonce":812540271}]},"public_addr":"172.21.15.164:6825/812540271","cluster_addr":"172.21.15.164:6827/812540271","heartbeat_back_addr":"172.21.15.164:6831/812540271","heartbeat_front_addr":"172.21.15.164:6829/812540271","state":["exists","up"]},{"osd":4,"uuid":"8e83f315-42bf-4471-9ce8-d81948da0f9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":2656857421},{"type":"v1","addr":"172.21.15.173:6825","nonce":2656857421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2656857421},{"type":"v1","addr":"172.21.15.173:6827","nonce":2656857421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2656857421},{"type":"v1","addr":"172.21.15.173:6831","nonce":2656857421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":2656857421},{"type":"v1","addr":"172.21.15.173:6829","nonce":2656857421}]},"public_addr":"172.21.15.173:6825/2656857421","cluster_addr":"172.21.15.173:6827/2656857421","heartbeat_back_addr":"172.21.15.173:6831/2656857421","heartbeat_front_addr":"172.21.15.173:6829/2656857421","state":["exists","up"]},{"osd":5,"uuid":"a230c872-71f9-40ab-8999-defd0162d4ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":3676483185},{"type":"v1","addr":"172.21.15.173:6801","nonce":3676483185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":3676483185},{"type":"v1","addr":"172.21.15.173:6803","nonce":3676483185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":3676483185},{"type":"v1","addr":"172.21.15.173:6807","nonce":3676483185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":3676483185},{"type":"v1","addr":"172.21.15.173:6805","nonce":3676483185}]},"public_addr":"172.21.15.173:6801/3676483185","cluster_addr":"172.21.15.173:6803/3676483185","heartbeat_back_addr":"172.21.15.173:6807/3676483185","heartbeat_front_addr":"172.21.15.173:6805/3676483185","state":["exists","up"]},{"osd":6,"uuid":"6193bfd7-8432-4f7f-a117-a3fcee21af99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":3631684087},{"type":"v1","addr":"172.21.15.173:6809","nonce":3631684087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":3631684087},{"type":"v1","addr":"172.21.15.173:6811","nonce":3631684087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":3631684087},{"type":"v1","addr":"172.21.15.173:6815","nonce":3631684087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":3631684087},{"type":"v1","addr":"172.21.15.173:6813","nonce":3631684087}]},"public_addr":"172.21.15.173:6809/3631684087","cluster_addr":"172.21.15.173:6811/3631684087","heartbeat_back_addr":"172.21.15.173:6815/3631684087","heartbeat_front_addr":"172.21.15.173:6813/3631684087","state":["exists","up"]},{"osd":7,"uuid":"b0f1b80a-afc4-4538-83b9-f081d2dcb8cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":3261114948},{"type":"v1","addr":"172.21.15.173:6817","nonce":3261114948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":3261114948},{"type":"v1","addr":"172.21.15.173:6819","nonce":3261114948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":3261114948},{"type":"v1","addr":"172.21.15.173:6823","nonce":3261114948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":3261114948},{"type":"v1","addr":"172.21.15.173:6821","nonce":3261114948}]},"public_addr":"172.21.15.173:6817/3261114948","cluster_addr":"172.21.15.173:6819/3261114948","heartbeat_back_addr":"172.21.15.173:6823/3261114948","heartbeat_front_addr":"172.21.15.173:6821/3261114948","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.188390+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.159932+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.191101+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.181776+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.224365+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.177515+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.153631+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.139972+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T14:57:20.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.429+0000 7f7dec86a640 1 -- 172.21.15.164:0/1468854924 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f7dc80672a0 msgr2=0x7f7dc8069760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:20.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.429+0000 7f7dec86a640 1 --2- 172.21.15.164:0/1468854924 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f7dc80672a0 0x7f7dc8069760 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc0ad000 tx=0x7f7ddc008040 comp rx=0 tx=0).stop 2024-03-23T14:57:20.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.433+0000 7f7dec86a640 1 -- 172.21.15.164:0/1468854924 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7de814db70 msgr2=0x7f7de809aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:20.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.433+0000 7f7dec86a640 1 --2- 172.21.15.164:0/1468854924 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7de814db70 0x7f7de809aa50 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f7de0007e90 tx=0x7f7de000e4b0 comp rx=0 tx=0).stop 2024-03-23T14:57:20.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.433+0000 7f7dec86a640 1 -- 172.21.15.164:0/1468854924 shutdown_connections 2024-03-23T14:57:20.436 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.433+0000 7f7dec86a640 1 --2- 172.21.15.164:0/1468854924 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7de814db70 0x7f7de809aa50 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:20.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.433+0000 7f7dec86a640 1 --2- 172.21.15.164:0/1468854924 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7de8148cf0 0x7f7de80718c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:20.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.433+0000 7f7dec86a640 1 --2- 172.21.15.164:0/1468854924 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f7dc80672a0 0x7f7dc8069760 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:20.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.433+0000 7f7dec86a640 1 --2- 172.21.15.164:0/1468854924 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7de80b0230 0x7f7de80706a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:20.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.433+0000 7f7dec86a640 1 -- 172.21.15.164:0/1468854924 >> 172.21.15.164:0/1468854924 conn(0x7f7de80b8ca0 msgr2=0x7f7de80b2350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:20.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.433+0000 7f7dec86a640 1 -- 172.21.15.164:0/1468854924 shutdown_connections 2024-03-23T14:57:20.437 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.433+0000 7f7dec86a640 1 -- 172.21.15.164:0/1468854924 wait complete. 2024-03-23T14:57:20.452 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T14:56:47.186157+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-23T14:56:56.994057+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-23T14:56:57.953862+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-23T14:57:20.452 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-23T14:57:20.562 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.557+0000 7f736e5b0640 1 Processor -- start 2024-03-23T14:57:20.563 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.557+0000 7f736e5b0640 1 -- start start 2024-03-23T14:57:20.564 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.557+0000 7f736e5b0640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f736809def0 0x7f73680ae620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:20.564 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.557+0000 7f736e5b0640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f73680b4e90 0x7f736814e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:20.564 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.557+0000 7f736e5b0640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f736809e2d0 con 0x7f73680b4a00 2024-03-23T14:57:20.564 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.557+0000 7f736e5b0640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 -- 0x7f7368146de0 con 0x7f736809def0 2024-03-23T14:57:20.564 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.557+0000 7f736e5b0640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f73680b52e0 con 0x7f73680b4e90 2024-03-23T14:57:20.564 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.561+0000 7f7367fff640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f736809def0 0x7f73680ae620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:20.564 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.561+0000 7f736cb4d640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f73680b4e90 0x7f736814e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:20.564 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.561+0000 7f7367fff640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f736809def0 0x7f73680ae620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.164:33496/0 (socket says 172.21.15.164:33496) 2024-03-23T14:57:20.564 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.561+0000 7f7367fff640 1 -- 172.21.15.164:0/202697098 learned_addr learned my addr 172.21.15.164:0/202697098 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:20.565 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.561+0000 7f7367fff640 1 -- 172.21.15.164:0/202697098 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f73680b4e90 msgr2=0x7f736814e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:20.565 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.561+0000 7f7367fff640 1 --2- 172.21.15.164:0/202697098 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f73680b4e90 0x7f736814e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:20.565 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.561+0000 7f7367fff640 1 -- 172.21.15.164:0/202697098 >> v1:172.21.15.173:6789/0 conn(0x7f73680b4a00 legacy=0x7f7368147420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:20.565 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.561+0000 7f7367fff640 1 -- 172.21.15.164:0/202697098 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f736806cb70 con 0x7f736809def0 2024-03-23T14:57:20.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.561+0000 7f7367fff640 1 --2- 172.21.15.164:0/202697098 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f736809def0 0x7f73680ae620 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f736000d150 tx=0x7f736001eba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9865bccf9d092a58 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:20.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.561+0000 7f7366ffd640 1 -- 172.21.15.164:0/202697098 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73600a7b60 con 0x7f736809def0 2024-03-23T14:57:20.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.561+0000 7f7366ffd640 1 -- 172.21.15.164:0/202697098 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73600ab070 con 0x7f736809def0 2024-03-23T14:57:20.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.561+0000 7f7366ffd640 1 -- 172.21.15.164:0/202697098 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73600b1400 con 0x7f736809def0 2024-03-23T14:57:20.569 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.561+0000 7f736e5b0640 1 -- 172.21.15.164:0/202697098 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f736809def0 msgr2=0x7f73680ae620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:20.569 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.561+0000 7f736e5b0640 1 --2- 172.21.15.164:0/202697098 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f736809def0 0x7f73680ae620 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f736000d150 tx=0x7f736001eba0 comp rx=0 tx=0).stop 2024-03-23T14:57:20.569 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.565+0000 7f736e5b0640 1 -- 172.21.15.164:0/202697098 shutdown_connections 2024-03-23T14:57:20.569 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.565+0000 7f736e5b0640 1 --2- 172.21.15.164:0/202697098 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f73680b4e90 0x7f736814e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:20.569 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.565+0000 7f736e5b0640 1 --2- 172.21.15.164:0/202697098 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f736809def0 0x7f73680ae620 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:20.569 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.565+0000 7f736e5b0640 1 -- 172.21.15.164:0/202697098 >> 172.21.15.164:0/202697098 conn(0x7f73680b8ca0 msgr2=0x7f73680b90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:20.569 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.565+0000 7f736e5b0640 1 -- 172.21.15.164:0/202697098 shutdown_connections 2024-03-23T14:57:20.569 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.565+0000 7f736e5b0640 1 -- 172.21.15.164:0/202697098 wait complete. 2024-03-23T14:57:20.569 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.565+0000 7f736e5b0640 1 Processor -- start 2024-03-23T14:57:20.569 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.565+0000 7f736e5b0640 1 -- start start 2024-03-23T14:57:20.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.565+0000 7f736e5b0640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f736809def0 0x7f736809ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:20.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.565+0000 7f736e5b0640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f73680b4a00 0x7f7368143e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:20.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.565+0000 7f736e5b0640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f73680b4e90 0x7f73680b17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:20.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.565+0000 7f736e5b0640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f73680b21b0 con 0x7f736809def0 2024-03-23T14:57:20.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.565+0000 7f736e5b0640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f73680b1010 con 0x7f73680b4a00 2024-03-23T14:57:20.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.565+0000 7f736e5b0640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f736809b1b0 con 0x7f73680b4e90 2024-03-23T14:57:20.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.565+0000 7f7367fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f736809def0 0x7f736809ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:20.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.565+0000 7f7367fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f736809def0 0x7f736809ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:60368/0 (socket says 172.21.15.164:60368) 2024-03-23T14:57:20.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.565+0000 7f7367fff640 1 -- 172.21.15.164:0/2162407906 learned_addr learned my addr 172.21.15.164:0/2162407906 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:20.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.565+0000 7f736cb4d640 1 --2- 172.21.15.164:0/2162407906 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f73680b4e90 0x7f73680b17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:20.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.565+0000 7f73677fe640 1 --2- 172.21.15.164:0/2162407906 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f73680b4a00 0x7f7368143e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:20.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.565+0000 7f7367fff640 1 -- 172.21.15.164:0/2162407906 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f73680b4e90 msgr2=0x7f73680b17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:20.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.565+0000 7f7367fff640 1 --2- 172.21.15.164:0/2162407906 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f73680b4e90 0x7f73680b17f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:20.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.565+0000 7f7367fff640 1 -- 172.21.15.164:0/2162407906 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f73680b4a00 msgr2=0x7f7368143e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:20.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.565+0000 7f7367fff640 1 --2- 172.21.15.164:0/2162407906 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f73680b4a00 0x7f7368143e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:20.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.565+0000 7f7367fff640 1 -- 172.21.15.164:0/2162407906 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f736000bd40 con 0x7f736809def0 2024-03-23T14:57:20.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.565+0000 7f73677fe640 1 --2- 172.21.15.164:0/2162407906 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f73680b4a00 0x7f7368143e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T14:57:20.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.565+0000 7f7367fff640 1 --2- 172.21.15.164:0/2162407906 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f736809def0 0x7f736809ab00 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f736001c7d0 tx=0x7f736001b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:20.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.565+0000 7f7364ff9640 1 -- 172.21.15.164:0/2162407906 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73600bc850 con 0x7f736809def0 2024-03-23T14:57:20.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.565+0000 7f7364ff9640 1 -- 172.21.15.164:0/2162407906 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73600bc9b0 con 0x7f736809def0 2024-03-23T14:57:20.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.565+0000 7f736e5b0640 1 -- 172.21.15.164:0/2162407906 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73680addc0 con 0x7f736809def0 2024-03-23T14:57:20.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.565+0000 7f7364ff9640 1 -- 172.21.15.164:0/2162407906 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73600bcd40 con 0x7f736809def0 2024-03-23T14:57:20.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.565+0000 7f736e5b0640 1 -- 172.21.15.164:0/2162407906 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73680b0350 con 0x7f736809def0 2024-03-23T14:57:20.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.565+0000 7f736e5b0640 1 -- 172.21.15.164:0/2162407906 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73680690a0 con 0x7f736809def0 2024-03-23T14:57:20.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.569+0000 7f7364ff9640 1 -- 172.21.15.164:0/2162407906 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7f73600ba410 con 0x7f736809def0 2024-03-23T14:57:20.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.569+0000 7f7364ff9640 1 --2- 172.21.15.164:0/2162407906 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f73540672a0 0x7f7354069760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:20.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.569+0000 7f7364ff9640 1 -- 172.21.15.164:0/2162407906 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f7360122240 con 0x7f736809def0 2024-03-23T14:57:20.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.569+0000 7f73677fe640 1 --2- 172.21.15.164:0/2162407906 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f73540672a0 0x7f7354069760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:20.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.569+0000 7f73677fe640 1 --2- 172.21.15.164:0/2162407906 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f73540672a0 0x7f7354069760 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7368144760 tx=0x7f735c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:20.575 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.573+0000 7f7364ff9640 1 -- 172.21.15.164:0/2162407906 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7f73600eef60 con 0x7f736809def0 2024-03-23T14:57:20.730 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.725+0000 7f736e5b0640 1 -- 172.21.15.164:0/2162407906 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f7368144f60 con 0x7f736809def0 2024-03-23T14:57:20.730 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.725+0000 7f7364ff9640 1 -- 172.21.15.164:0/2162407906 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f73600f2620 con 0x7f736809def0 2024-03-23T14:57:20.731 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 1 2024-03-23T14:57:20.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.729+0000 7f736e5b0640 1 -- 172.21.15.164:0/2162407906 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f73540672a0 msgr2=0x7f7354069760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:20.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.729+0000 7f736e5b0640 1 --2- 172.21.15.164:0/2162407906 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f73540672a0 0x7f7354069760 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7368144760 tx=0x7f735c009040 comp rx=0 tx=0).stop 2024-03-23T14:57:20.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.729+0000 7f736e5b0640 1 -- 172.21.15.164:0/2162407906 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f736809def0 msgr2=0x7f736809ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:20.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.729+0000 7f736e5b0640 1 --2- 172.21.15.164:0/2162407906 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f736809def0 0x7f736809ab00 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f736001c7d0 tx=0x7f736001b940 comp rx=0 tx=0).stop 2024-03-23T14:57:20.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.729+0000 7f736e5b0640 1 -- 172.21.15.164:0/2162407906 shutdown_connections 2024-03-23T14:57:20.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.729+0000 7f736e5b0640 1 --2- 172.21.15.164:0/2162407906 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f73680b4e90 0x7f73680b17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:20.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.729+0000 7f736e5b0640 1 --2- 172.21.15.164:0/2162407906 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f73540672a0 0x7f7354069760 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:20.734 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.729+0000 7f736e5b0640 1 --2- 172.21.15.164:0/2162407906 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f73680b4a00 0x7f7368143e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:20.734 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.729+0000 7f736e5b0640 1 --2- 172.21.15.164:0/2162407906 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f736809def0 0x7f736809ab00 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:20.734 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.729+0000 7f736e5b0640 1 -- 172.21.15.164:0/2162407906 >> 172.21.15.164:0/2162407906 conn(0x7f73680b8ca0 msgr2=0x7f73680b7fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:20.734 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.729+0000 7f736e5b0640 1 -- 172.21.15.164:0/2162407906 shutdown_connections 2024-03-23T14:57:20.734 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.729+0000 7f736e5b0640 1 -- 172.21.15.164:0/2162407906 wait complete. 2024-03-23T14:57:20.748 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-23T14:57:20.871 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.865+0000 7f901da7f640 1 Processor -- start 2024-03-23T14:57:20.871 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.865+0000 7f901da7f640 1 -- start start 2024-03-23T14:57:20.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.869+0000 7f901da7f640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f901809def0 0x7f90180ae620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:20.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.869+0000 7f901da7f640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f90180b4e90 0x7f901814e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:20.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.869+0000 7f901da7f640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f901809e2d0 con 0x7f90180b4a00 2024-03-23T14:57:20.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.869+0000 7f901da7f640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f9018146de0 con 0x7f90180b4e90 2024-03-23T14:57:20.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.869+0000 7f901da7f640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 -- 0x7f90180b52e0 con 0x7f901809def0 2024-03-23T14:57:20.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.869+0000 7f9017fff640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f90180b4e90 0x7f901814e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:20.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.869+0000 7f9017fff640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f90180b4e90 0x7f901814e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:60384/0 (socket says 172.21.15.164:60384) 2024-03-23T14:57:20.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.869+0000 7f9017fff640 1 -- 172.21.15.164:0/210329694 learned_addr learned my addr 172.21.15.164:0/210329694 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:20.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.869+0000 7f90177fe640 1 --2- 172.21.15.164:0/210329694 >> v2:172.21.15.173:3300/0 conn(0x7f901809def0 0x7f90180ae620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:20.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.869+0000 7f9017fff640 1 -- 172.21.15.164:0/210329694 >> v2:172.21.15.173:3300/0 conn(0x7f901809def0 msgr2=0x7f90180ae620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:20.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.869+0000 7f9017fff640 1 --2- 172.21.15.164:0/210329694 >> v2:172.21.15.173:3300/0 conn(0x7f901809def0 0x7f90180ae620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:20.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.869+0000 7f9017fff640 1 -- 172.21.15.164:0/210329694 >> v1:172.21.15.173:6789/0 conn(0x7f90180b4a00 legacy=0x7f9018147420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:20.874 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.869+0000 7f9017fff640 1 -- 172.21.15.164:0/210329694 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f901806cb70 con 0x7f90180b4e90 2024-03-23T14:57:20.875 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.869+0000 7f9017fff640 1 --2- 172.21.15.164:0/210329694 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f90180b4e90 0x7f901814e110 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f901806da30 tx=0x7f901001c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23c8b1348ab1beae server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:20.875 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.869+0000 7f90167fc640 1 -- 172.21.15.164:0/210329694 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90100a5570 con 0x7f90180b4e90 2024-03-23T14:57:20.875 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.869+0000 7f90167fc640 1 -- 172.21.15.164:0/210329694 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90100a56d0 con 0x7f90180b4e90 2024-03-23T14:57:20.875 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.869+0000 7f90167fc640 1 -- 172.21.15.164:0/210329694 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90100a5a60 con 0x7f90180b4e90 2024-03-23T14:57:20.876 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.869+0000 7f901da7f640 1 -- 172.21.15.164:0/210329694 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f90180b4e90 msgr2=0x7f901814e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:20.876 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.869+0000 7f901da7f640 1 --2- 172.21.15.164:0/210329694 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f90180b4e90 0x7f901814e110 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f901806da30 tx=0x7f901001c8d0 comp rx=0 tx=0).stop 2024-03-23T14:57:20.876 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.869+0000 7f901da7f640 1 -- 172.21.15.164:0/210329694 shutdown_connections 2024-03-23T14:57:20.876 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.869+0000 7f901da7f640 1 --2- 172.21.15.164:0/210329694 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f90180b4e90 0x7f901814e110 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:20.876 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.869+0000 7f901da7f640 1 --2- 172.21.15.164:0/210329694 >> v2:172.21.15.173:3300/0 conn(0x7f901809def0 0x7f90180ae620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:20.876 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.869+0000 7f901da7f640 1 -- 172.21.15.164:0/210329694 >> 172.21.15.164:0/210329694 conn(0x7f90180b8ca0 msgr2=0x7f90180b90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:20.876 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.869+0000 7f901da7f640 1 -- 172.21.15.164:0/210329694 shutdown_connections 2024-03-23T14:57:20.876 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.869+0000 7f901da7f640 1 -- 172.21.15.164:0/210329694 wait complete. 2024-03-23T14:57:20.878 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.873+0000 7f901da7f640 1 Processor -- start 2024-03-23T14:57:20.878 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.873+0000 7f901da7f640 1 -- start start 2024-03-23T14:57:20.879 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.873+0000 7f901da7f640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f901809def0 0x7f9018143dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:20.879 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.873+0000 7f901da7f640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f90180b4a00 0x7f90181dded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:20.879 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.873+0000 7f901da7f640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f90180b4e90 0x7f901813c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:20.879 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.873+0000 7f901da7f640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f901809b500 con 0x7f90180b4e90 2024-03-23T14:57:20.879 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.873+0000 7f901da7f640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f90181e1440 con 0x7f90180b4a00 2024-03-23T14:57:20.879 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.873+0000 7f901da7f640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f90181e1720 con 0x7f901809def0 2024-03-23T14:57:20.879 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.873+0000 7f9017fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f90180b4e90 0x7f901813c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:20.879 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.873+0000 7f9017fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f90180b4e90 0x7f901813c200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:60390/0 (socket says 172.21.15.164:60390) 2024-03-23T14:57:20.879 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.873+0000 7f9017fff640 1 -- 172.21.15.164:0/3582906429 learned_addr learned my addr 172.21.15.164:0/3582906429 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:20.879 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.873+0000 7f90177fe640 1 --2- 172.21.15.164:0/3582906429 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f901809def0 0x7f9018143dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:20.879 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.873+0000 7f9016ffd640 1 --2- 172.21.15.164:0/3582906429 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f90180b4a00 0x7f90181dded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:20.879 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.873+0000 7f9017fff640 1 -- 172.21.15.164:0/3582906429 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f901809def0 msgr2=0x7f9018143dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:20.879 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.873+0000 7f9017fff640 1 --2- 172.21.15.164:0/3582906429 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f901809def0 0x7f9018143dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:20.879 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.873+0000 7f9017fff640 1 -- 172.21.15.164:0/3582906429 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f90180b4a00 msgr2=0x7f90181dded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:20.880 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.873+0000 7f9017fff640 1 --2- 172.21.15.164:0/3582906429 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f90180b4a00 0x7f90181dded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:20.880 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.873+0000 7f9017fff640 1 -- 172.21.15.164:0/3582906429 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9010009b30 con 0x7f90180b4e90 2024-03-23T14:57:20.880 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.873+0000 7f9017fff640 1 --2- 172.21.15.164:0/3582906429 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f90180b4e90 0x7f901813c200 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f90100194e0 tx=0x7f9010019670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:20.880 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.873+0000 7f8ffffff640 1 -- 172.21.15.164:0/3582906429 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90100a5570 con 0x7f90180b4e90 2024-03-23T14:57:20.880 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.873+0000 7f8ffffff640 1 -- 172.21.15.164:0/3582906429 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90100b8690 con 0x7f90180b4e90 2024-03-23T14:57:20.880 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.873+0000 7f901da7f640 1 -- 172.21.15.164:0/3582906429 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90180addc0 con 0x7f90180b4e90 2024-03-23T14:57:20.880 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.873+0000 7f8ffffff640 1 -- 172.21.15.164:0/3582906429 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90100b8a40 con 0x7f90180b4e90 2024-03-23T14:57:20.880 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.873+0000 7f901da7f640 1 -- 172.21.15.164:0/3582906429 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90181e1ce0 con 0x7f90180b4e90 2024-03-23T14:57:20.880 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.873+0000 7f901da7f640 1 -- 172.21.15.164:0/3582906429 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90181e1ed0 con 0x7f90180b4e90 2024-03-23T14:57:20.881 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.873+0000 7f8ffffff640 1 -- 172.21.15.164:0/3582906429 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7f90100b8ba0 con 0x7f90180b4e90 2024-03-23T14:57:20.881 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.877+0000 7f8ffffff640 1 --2- 172.21.15.164:0/3582906429 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f90040672a0 0x7f9004069760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:20.881 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.877+0000 7f8ffffff640 1 -- 172.21.15.164:0/3582906429 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f9010120420 con 0x7f90180b4e90 2024-03-23T14:57:20.881 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.877+0000 7f90177fe640 1 --2- 172.21.15.164:0/3582906429 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f90040672a0 0x7f9004069760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:20.881 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.877+0000 7f90177fe640 1 --2- 172.21.15.164:0/3582906429 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f90040672a0 0x7f9004069760 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f900800b9d0 tx=0x7f9008008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:20.884 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:20.881+0000 7f8ffffff640 1 -- 172.21.15.164:0/3582906429 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7f90100ed180 con 0x7f90180b4e90 2024-03-23T14:57:21.040 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.037+0000 7f901da7f640 1 -- 172.21.15.164:0/3582906429 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f90180690a0 con 0x7f90180b4e90 2024-03-23T14:57:21.041 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.037+0000 7f8ffffff640 1 -- 172.21.15.164:0/3582906429 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f9010009510 con 0x7f90180b4e90 2024-03-23T14:57:21.041 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 32 2024-03-23T14:57:21.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.037+0000 7f901da7f640 1 -- 172.21.15.164:0/3582906429 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f90040672a0 msgr2=0x7f9004069760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:21.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.037+0000 7f901da7f640 1 --2- 172.21.15.164:0/3582906429 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f90040672a0 0x7f9004069760 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f900800b9d0 tx=0x7f9008008040 comp rx=0 tx=0).stop 2024-03-23T14:57:21.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.037+0000 7f901da7f640 1 -- 172.21.15.164:0/3582906429 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f90180b4e90 msgr2=0x7f901813c200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:21.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.037+0000 7f901da7f640 1 --2- 172.21.15.164:0/3582906429 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f90180b4e90 0x7f901813c200 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f90100194e0 tx=0x7f9010019670 comp rx=0 tx=0).stop 2024-03-23T14:57:21.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.037+0000 7f901da7f640 1 -- 172.21.15.164:0/3582906429 shutdown_connections 2024-03-23T14:57:21.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.037+0000 7f901da7f640 1 --2- 172.21.15.164:0/3582906429 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f90180b4e90 0x7f901813c200 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:21.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.037+0000 7f901da7f640 1 --2- 172.21.15.164:0/3582906429 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f90040672a0 0x7f9004069760 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:21.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.037+0000 7f901da7f640 1 --2- 172.21.15.164:0/3582906429 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f90180b4a00 0x7f90181dded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:21.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.037+0000 7f901da7f640 1 --2- 172.21.15.164:0/3582906429 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f901809def0 0x7f9018143dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:21.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.037+0000 7f901da7f640 1 -- 172.21.15.164:0/3582906429 >> 172.21.15.164:0/3582906429 conn(0x7f90180b8ca0 msgr2=0x7f90181418a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:21.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.041+0000 7f901da7f640 1 -- 172.21.15.164:0/3582906429 shutdown_connections 2024-03-23T14:57:21.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.041+0000 7f901da7f640 1 -- 172.21.15.164:0/3582906429 wait complete. 2024-03-23T14:57:21.060 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-23T14:57:21.183 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.177+0000 7fd95159b640 1 Processor -- start 2024-03-23T14:57:21.184 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.177+0000 7fd95159b640 1 -- start start 2024-03-23T14:57:21.184 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.181+0000 7fd95159b640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd94c0add50 0x7fd94c147770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:21.185 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.181+0000 7fd95159b640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fd94c147cb0 0x7fd94c148140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:21.185 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.181+0000 7fd95159b640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd94c06d650 con 0x7fd94c09def0 2024-03-23T14:57:21.185 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.181+0000 7fd95159b640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 -- 0x7fd94c0af4a0 con 0x7fd94c147cb0 2024-03-23T14:57:21.185 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.181+0000 7fd95159b640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fd94c09e2d0 con 0x7fd94c0add50 2024-03-23T14:57:21.185 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.181+0000 7fd94a7fc640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd94c0add50 0x7fd94c147770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:21.185 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.181+0000 7fd94b7fe640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fd94c147cb0 0x7fd94c148140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:21.185 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.181+0000 7fd94a7fc640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd94c0add50 0x7fd94c147770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.164:60818/0 (socket says 172.21.15.164:60818) 2024-03-23T14:57:21.185 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.181+0000 7fd94a7fc640 1 -- 172.21.15.164:0/2795462619 learned_addr learned my addr 172.21.15.164:0/2795462619 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:21.185 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.181+0000 7fd94affd640 1 --1- 172.21.15.164:0/2795462619 >> v1:172.21.15.173:6789/0 conn(0x7fd94c09def0 0x7fd94c0b45e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.173:6789/0 says I am v1:172.21.15.164:50570/0 (socket says 172.21.15.164:50570) 2024-03-23T14:57:21.186 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.181+0000 7fd94b7fe640 1 -- 172.21.15.164:0/2795462619 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd94c0add50 msgr2=0x7fd94c147770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:21.186 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.181+0000 7fd94b7fe640 1 --2- 172.21.15.164:0/2795462619 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd94c0add50 0x7fd94c147770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:21.186 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.181+0000 7fd94b7fe640 1 -- 172.21.15.164:0/2795462619 >> v1:172.21.15.173:6789/0 conn(0x7fd94c09def0 legacy=0x7fd94c0b45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:21.186 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.181+0000 7fd94b7fe640 1 -- 172.21.15.164:0/2795462619 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd94c0aeca0 con 0x7fd94c147cb0 2024-03-23T14:57:21.186 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.181+0000 7fd94a7fc640 1 --2- 172.21.15.164:0/2795462619 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd94c0add50 0x7fd94c147770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T14:57:21.187 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.181+0000 7fd94b7fe640 1 --2- 172.21.15.164:0/2795462619 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd94c147cb0 0x7fd94c148140 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd944009d90 tx=0x7fd94401c850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ad40bfd636fe59a server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:21.188 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.181+0000 7fd949ffb640 1 -- 172.21.15.164:0/2795462619 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9440a5870 con 0x7fd94c147cb0 2024-03-23T14:57:21.188 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.185+0000 7fd949ffb640 1 -- 172.21.15.164:0/2795462619 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9440a59d0 con 0x7fd94c147cb0 2024-03-23T14:57:21.188 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.185+0000 7fd949ffb640 1 -- 172.21.15.164:0/2795462619 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9440a5d60 con 0x7fd94c147cb0 2024-03-23T14:57:21.188 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.185+0000 7fd95159b640 1 -- 172.21.15.164:0/2795462619 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd94c147cb0 msgr2=0x7fd94c148140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:21.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.185+0000 7fd95159b640 1 --2- 172.21.15.164:0/2795462619 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd94c147cb0 0x7fd94c148140 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd944009d90 tx=0x7fd94401c850 comp rx=0 tx=0).stop 2024-03-23T14:57:21.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.185+0000 7fd95159b640 1 -- 172.21.15.164:0/2795462619 shutdown_connections 2024-03-23T14:57:21.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.185+0000 7fd95159b640 1 --2- 172.21.15.164:0/2795462619 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd94c147cb0 0x7fd94c148140 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:21.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.185+0000 7fd95159b640 1 --2- 172.21.15.164:0/2795462619 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd94c0add50 0x7fd94c147770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:21.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.185+0000 7fd95159b640 1 -- 172.21.15.164:0/2795462619 >> 172.21.15.164:0/2795462619 conn(0x7fd94c0b8ca0 msgr2=0x7fd94c0b90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:21.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.185+0000 7fd95159b640 1 -- 172.21.15.164:0/2795462619 shutdown_connections 2024-03-23T14:57:21.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.185+0000 7fd95159b640 1 -- 172.21.15.164:0/2795462619 wait complete. 2024-03-23T14:57:21.192 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.185+0000 7fd95159b640 1 Processor -- start 2024-03-23T14:57:21.192 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.185+0000 7fd95159b640 1 -- start start 2024-03-23T14:57:21.192 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.185+0000 7fd95159b640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd94c09def0 0x7fd94c139b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:21.192 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.185+0000 7fd95159b640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd94c0add50 0x7fd94c0b3730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:21.192 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.185+0000 7fd95159b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd94c147cb0 0x7fd94c0b1640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:21.192 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.185+0000 7fd95159b640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd94c0b1b80 con 0x7fd94c147cb0 2024-03-23T14:57:21.192 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.185+0000 7fd95159b640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd94c1396d0 con 0x7fd94c09def0 2024-03-23T14:57:21.192 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.185+0000 7fd95159b640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fd94c0b1250 con 0x7fd94c0add50 2024-03-23T14:57:21.192 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.185+0000 7fd94b7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd94c147cb0 0x7fd94c0b1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:21.192 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.185+0000 7fd94b7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd94c147cb0 0x7fd94c0b1640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:60394/0 (socket says 172.21.15.164:60394) 2024-03-23T14:57:21.192 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.185+0000 7fd94b7fe640 1 -- 172.21.15.164:0/3488435061 learned_addr learned my addr 172.21.15.164:0/3488435061 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:21.193 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.185+0000 7fd94affd640 1 --2- 172.21.15.164:0/3488435061 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd94c09def0 0x7fd94c139b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:21.193 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.185+0000 7fd94a7fc640 1 --2- 172.21.15.164:0/3488435061 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd94c0add50 0x7fd94c0b3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:21.193 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.185+0000 7fd94b7fe640 1 -- 172.21.15.164:0/3488435061 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd94c0add50 msgr2=0x7fd94c0b3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:21.193 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.185+0000 7fd94b7fe640 1 --2- 172.21.15.164:0/3488435061 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd94c0add50 0x7fd94c0b3730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:21.193 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.185+0000 7fd94b7fe640 1 -- 172.21.15.164:0/3488435061 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd94c09def0 msgr2=0x7fd94c139b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:21.193 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.185+0000 7fd94b7fe640 1 --2- 172.21.15.164:0/3488435061 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd94c09def0 0x7fd94c139b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:21.193 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.185+0000 7fd94b7fe640 1 -- 172.21.15.164:0/3488435061 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9400094d0 con 0x7fd94c147cb0 2024-03-23T14:57:21.193 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.185+0000 7fd94b7fe640 1 --2- 172.21.15.164:0/3488435061 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd94c147cb0 0x7fd94c0b1640 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fd9440b0040 tx=0x7fd9440195f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:21.193 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.185+0000 7fd9337fe640 1 -- 172.21.15.164:0/3488435061 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9440b9860 con 0x7fd94c147cb0 2024-03-23T14:57:21.193 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.185+0000 7fd9337fe640 1 -- 172.21.15.164:0/3488435061 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9440b99c0 con 0x7fd94c147cb0 2024-03-23T14:57:21.193 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.185+0000 7fd95159b640 1 -- 172.21.15.164:0/3488435061 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd944009a70 con 0x7fd94c147cb0 2024-03-23T14:57:21.193 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.185+0000 7fd9337fe640 1 -- 172.21.15.164:0/3488435061 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9440b9d70 con 0x7fd94c147cb0 2024-03-23T14:57:21.193 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.185+0000 7fd95159b640 1 -- 172.21.15.164:0/3488435061 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd94c0b4e70 con 0x7fd94c147cb0 2024-03-23T14:57:21.194 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.189+0000 7fd95159b640 1 -- 172.21.15.164:0/3488435061 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd94c1392c0 con 0x7fd94c147cb0 2024-03-23T14:57:21.194 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.189+0000 7fd9337fe640 1 -- 172.21.15.164:0/3488435061 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7fd9440ad440 con 0x7fd94c147cb0 2024-03-23T14:57:21.194 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.189+0000 7fd9337fe640 1 --2- 172.21.15.164:0/3488435061 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fd9380672a0 0x7fd938069760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:21.194 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.189+0000 7fd9337fe640 1 -- 172.21.15.164:0/3488435061 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fd9441200d0 con 0x7fd94c147cb0 2024-03-23T14:57:21.195 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.189+0000 7fd94affd640 1 --2- 172.21.15.164:0/3488435061 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fd9380672a0 0x7fd938069760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:21.195 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.189+0000 7fd94affd640 1 --2- 172.21.15.164:0/3488435061 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fd9380672a0 0x7fd938069760 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fd93c000f70 tx=0x7fd93c007e90 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:21.198 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.193+0000 7fd9337fe640 1 -- 172.21.15.164:0/3488435061 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7fd9440ecd20 con 0x7fd94c147cb0 2024-03-23T14:57:21.351 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.345+0000 7fd95159b640 1 -- 172.21.15.164:0/3488435061 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fd94c1394a0 con 0x7fd94c147cb0 2024-03-23T14:57:21.352 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.349+0000 7fd9337fe640 1 -- 172.21.15.164:0/3488435061 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7fd94c1394a0 con 0x7fd94c147cb0 2024-03-23T14:57:21.352 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 64 2024-03-23T14:57:21.354 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.349+0000 7fd95159b640 1 -- 172.21.15.164:0/3488435061 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fd9380672a0 msgr2=0x7fd938069760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:21.354 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.349+0000 7fd95159b640 1 --2- 172.21.15.164:0/3488435061 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fd9380672a0 0x7fd938069760 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fd93c000f70 tx=0x7fd93c007e90 comp rx=0 tx=0).stop 2024-03-23T14:57:21.354 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.349+0000 7fd95159b640 1 -- 172.21.15.164:0/3488435061 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd94c147cb0 msgr2=0x7fd94c0b1640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:21.354 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.349+0000 7fd95159b640 1 --2- 172.21.15.164:0/3488435061 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd94c147cb0 0x7fd94c0b1640 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fd9440b0040 tx=0x7fd9440195f0 comp rx=0 tx=0).stop 2024-03-23T14:57:21.355 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.349+0000 7fd95159b640 1 -- 172.21.15.164:0/3488435061 shutdown_connections 2024-03-23T14:57:21.355 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.349+0000 7fd95159b640 1 --2- 172.21.15.164:0/3488435061 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd94c147cb0 0x7fd94c0b1640 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:21.355 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.349+0000 7fd95159b640 1 --2- 172.21.15.164:0/3488435061 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd94c0add50 0x7fd94c0b3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:21.355 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.349+0000 7fd95159b640 1 --2- 172.21.15.164:0/3488435061 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fd9380672a0 0x7fd938069760 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:21.355 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.349+0000 7fd95159b640 1 --2- 172.21.15.164:0/3488435061 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd94c09def0 0x7fd94c139b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:21.355 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.349+0000 7fd95159b640 1 -- 172.21.15.164:0/3488435061 >> 172.21.15.164:0/3488435061 conn(0x7fd94c0b8ca0 msgr2=0x7fd94c146eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:21.355 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.349+0000 7fd95159b640 1 -- 172.21.15.164:0/3488435061 shutdown_connections 2024-03-23T14:57:21.355 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.349+0000 7fd95159b640 1 -- 172.21.15.164:0/3488435061 wait complete. 2024-03-23T14:57:21.370 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T14:57:21.484 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.481+0000 7f1486f20640 1 Processor -- start 2024-03-23T14:57:21.484 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.481+0000 7f1486f20640 1 -- start start 2024-03-23T14:57:21.486 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.481+0000 7f1486f20640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f14800b0230 0x7f14800b34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:21.486 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.481+0000 7f1486f20640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f148014db70 0x7f148014ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:21.486 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.481+0000 7f1486f20640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14800b51d0 con 0x7f1480148cf0 2024-03-23T14:57:21.486 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.481+0000 7f1486f20640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f14800b0610 con 0x7f14800b0230 2024-03-23T14:57:21.486 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.481+0000 7f1486f20640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f14800b1500 con 0x7f148014db70 2024-03-23T14:57:21.486 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.481+0000 7f1484cbc640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f14800b0230 0x7f14800b34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:21.486 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.481+0000 7f1484cbc640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f14800b0230 0x7f14800b34d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:60404/0 (socket says 172.21.15.164:60404) 2024-03-23T14:57:21.486 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.481+0000 7f1484cbc640 1 -- 172.21.15.164:0/763403429 learned_addr learned my addr 172.21.15.164:0/763403429 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:21.486 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.481+0000 7f14854bd640 1 --2- 172.21.15.164:0/763403429 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f148014db70 0x7f148014ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:21.487 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.481+0000 7f1484cbc640 1 -- 172.21.15.164:0/763403429 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f148014db70 msgr2=0x7f148014ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:21.487 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.481+0000 7f1484cbc640 1 --2- 172.21.15.164:0/763403429 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f148014db70 0x7f148014ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:21.487 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.481+0000 7f1484cbc640 1 -- 172.21.15.164:0/763403429 >> v1:172.21.15.173:6789/0 conn(0x7f1480148cf0 legacy=0x7f1480149180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:21.487 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.481+0000 7f1484cbc640 1 -- 172.21.15.164:0/763403429 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f148006d250 con 0x7f14800b0230 2024-03-23T14:57:21.489 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.485+0000 7f1484cbc640 1 --2- 172.21.15.164:0/763403429 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f14800b0230 0x7f14800b34d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f1470009870 tx=0x7f147001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1802ef37f1432878 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:21.489 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.485+0000 7f147f7fe640 1 -- 172.21.15.164:0/763403429 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14700a6040 con 0x7f14800b0230 2024-03-23T14:57:21.489 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.485+0000 7f147f7fe640 1 -- 172.21.15.164:0/763403429 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14700aa600 con 0x7f14800b0230 2024-03-23T14:57:21.489 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.485+0000 7f147f7fe640 1 -- 172.21.15.164:0/763403429 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14700aa990 con 0x7f14800b0230 2024-03-23T14:57:21.490 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.485+0000 7f1486f20640 1 -- 172.21.15.164:0/763403429 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f14800b0230 msgr2=0x7f14800b34d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:21.490 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.485+0000 7f1486f20640 1 --2- 172.21.15.164:0/763403429 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f14800b0230 0x7f14800b34d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f1470009870 tx=0x7f147001c4a0 comp rx=0 tx=0).stop 2024-03-23T14:57:21.490 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.485+0000 7f1486f20640 1 -- 172.21.15.164:0/763403429 shutdown_connections 2024-03-23T14:57:21.490 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.485+0000 7f1486f20640 1 --2- 172.21.15.164:0/763403429 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f148014db70 0x7f148014ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:21.490 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.485+0000 7f1486f20640 1 --2- 172.21.15.164:0/763403429 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f14800b0230 0x7f14800b34d0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:21.490 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.485+0000 7f1486f20640 1 -- 172.21.15.164:0/763403429 >> 172.21.15.164:0/763403429 conn(0x7f14800b8ca0 msgr2=0x7f14800b90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:21.491 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.485+0000 7f1486f20640 1 -- 172.21.15.164:0/763403429 shutdown_connections 2024-03-23T14:57:21.491 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.485+0000 7f1486f20640 1 -- 172.21.15.164:0/763403429 wait complete. 2024-03-23T14:57:21.493 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.485+0000 7f1486f20640 1 Processor -- start 2024-03-23T14:57:21.493 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.485+0000 7f1486f20640 1 -- start start 2024-03-23T14:57:21.493 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.485+0000 7f1486f20640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f14800b0230 0x7f148009c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:21.493 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.485+0000 7f1486f20640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1480148cf0 0x7f14801e3960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:21.493 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.485+0000 7f1486f20640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f148014db70 0x7f148009af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:21.494 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.485+0000 7f1486f20640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f148013efb0 con 0x7f1480148cf0 2024-03-23T14:57:21.494 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.485+0000 7f1486f20640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f148009c980 con 0x7f14800b0230 2024-03-23T14:57:21.494 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.485+0000 7f1486f20640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f14801e3470 con 0x7f148014db70 2024-03-23T14:57:21.494 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.485+0000 7f147ffff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1480148cf0 0x7f14801e3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:21.494 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.485+0000 7f147ffff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1480148cf0 0x7f14801e3960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:60412/0 (socket says 172.21.15.164:60412) 2024-03-23T14:57:21.494 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.485+0000 7f147ffff640 1 -- 172.21.15.164:0/1289044983 learned_addr learned my addr 172.21.15.164:0/1289044983 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:21.494 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.485+0000 7f1484cbc640 1 --2- 172.21.15.164:0/1289044983 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f14800b0230 0x7f148009c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:21.494 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.485+0000 7f14854bd640 1 --2- 172.21.15.164:0/1289044983 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f148014db70 0x7f148009af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:21.494 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.489+0000 7f147ffff640 1 -- 172.21.15.164:0/1289044983 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f148014db70 msgr2=0x7f148009af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:21.494 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.489+0000 7f147ffff640 1 --2- 172.21.15.164:0/1289044983 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f148014db70 0x7f148009af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:21.494 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.489+0000 7f147ffff640 1 -- 172.21.15.164:0/1289044983 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f14800b0230 msgr2=0x7f148009c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:21.495 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.489+0000 7f147ffff640 1 --2- 172.21.15.164:0/1289044983 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f14800b0230 0x7f148009c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:21.495 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.489+0000 7f147ffff640 1 -- 172.21.15.164:0/1289044983 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14700094d0 con 0x7f1480148cf0 2024-03-23T14:57:21.495 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.489+0000 7f147ffff640 1 --2- 172.21.15.164:0/1289044983 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1480148cf0 0x7f14801e3960 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f147400c420 tx=0x7f147400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:21.495 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.489+0000 7f147d7fa640 1 -- 172.21.15.164:0/1289044983 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14740052d0 con 0x7f1480148cf0 2024-03-23T14:57:21.495 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.489+0000 7f147d7fa640 1 -- 172.21.15.164:0/1289044983 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1474005430 con 0x7f1480148cf0 2024-03-23T14:57:21.495 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.489+0000 7f147d7fa640 1 -- 172.21.15.164:0/1289044983 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1474007260 con 0x7f1480148cf0 2024-03-23T14:57:21.495 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.489+0000 7f1486f20640 1 -- 172.21.15.164:0/1289044983 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14801e3f00 con 0x7f1480148cf0 2024-03-23T14:57:21.495 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.489+0000 7f1486f20640 1 -- 172.21.15.164:0/1289044983 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1480072250 con 0x7f1480148cf0 2024-03-23T14:57:21.495 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.489+0000 7f1486f20640 1 -- 172.21.15.164:0/1289044983 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14801e3630 con 0x7f1480148cf0 2024-03-23T14:57:21.495 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.489+0000 7f147d7fa640 1 -- 172.21.15.164:0/1289044983 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7f14740073c0 con 0x7f1480148cf0 2024-03-23T14:57:21.496 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.489+0000 7f147d7fa640 1 --2- 172.21.15.164:0/1289044983 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f146c0672a0 0x7f146c069760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:21.496 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.489+0000 7f1484cbc640 1 --2- 172.21.15.164:0/1289044983 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f146c0672a0 0x7f146c069760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:21.496 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.489+0000 7f147d7fa640 1 -- 172.21.15.164:0/1289044983 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f1474084290 con 0x7f1480148cf0 2024-03-23T14:57:21.496 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.489+0000 7f1484cbc640 1 --2- 172.21.15.164:0/1289044983 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f146c0672a0 0x7f146c069760 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f14700ae000 tx=0x7f14700b5040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:21.498 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.493+0000 7f147d7fa640 1 -- 172.21.15.164:0/1289044983 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7f1474050f80 con 0x7f1480148cf0 2024-03-23T14:57:21.648 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.645+0000 7f1486f20640 1 -- 172.21.15.164:0/1289044983 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f14800690a0 con 0x7f1480148cf0 2024-03-23T14:57:21.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.645+0000 7f147d7fa640 1 -- 172.21.15.164:0/1289044983 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16629 (secure 0 0 0) 0x7f1474054640 con 0x7f1480148cf0 2024-03-23T14:57:21.650 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-23T14:57:21.650 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":20,"fsid":"45c7911b-b69d-4e68-907b-fbfeb780aa3b","created":"2024-03-23T14:56:34.367633+0000","modified":"2024-03-23T14:57:00.611265+0000","last_up_change":"2024-03-23T14:56:46.297403+0000","last_in_change":"2024-03-23T14:56:36.629884+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T14:56:47.186157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-23T14:56:56.994057+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-23T14:56:57.953862+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"80abdc62-bd0d-4e60-a554-7ed6fd67cf54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":984716690},{"type":"v1","addr":"172.21.15.164:6801","nonce":984716690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":984716690},{"type":"v1","addr":"172.21.15.164:6803","nonce":984716690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":984716690},{"type":"v1","addr":"172.21.15.164:6807","nonce":984716690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":984716690},{"type":"v1","addr":"172.21.15.164:6805","nonce":984716690}]},"public_addr":"172.21.15.164:6801/984716690","cluster_addr":"172.21.15.164:6803/984716690","heartbeat_back_addr":"172.21.15.164:6807/984716690","heartbeat_front_addr":"172.21.15.164:6805/984716690","state":["exists","up"]},{"osd":1,"uuid":"f9001b06-be4d-4f42-a0e6-bcef2afa0d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":1349657239},{"type":"v1","addr":"172.21.15.164:6817","nonce":1349657239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":1349657239},{"type":"v1","addr":"172.21.15.164:6819","nonce":1349657239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":1349657239},{"type":"v1","addr":"172.21.15.164:6823","nonce":1349657239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":1349657239},{"type":"v1","addr":"172.21.15.164:6821","nonce":1349657239}]},"public_addr":"172.21.15.164:6817/1349657239","cluster_addr":"172.21.15.164:6819/1349657239","heartbeat_back_addr":"172.21.15.164:6823/1349657239","heartbeat_front_addr":"172.21.15.164:6821/1349657239","state":["exists","up"]},{"osd":2,"uuid":"fa0aeb9f-8b54-4831-a188-3e361c7980d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":887861252},{"type":"v1","addr":"172.21.15.164:6809","nonce":887861252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":887861252},{"type":"v1","addr":"172.21.15.164:6811","nonce":887861252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":887861252},{"type":"v1","addr":"172.21.15.164:6815","nonce":887861252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":887861252},{"type":"v1","addr":"172.21.15.164:6813","nonce":887861252}]},"public_addr":"172.21.15.164:6809/887861252","cluster_addr":"172.21.15.164:6811/887861252","heartbeat_back_addr":"172.21.15.164:6815/887861252","heartbeat_front_addr":"172.21.15.164:6813/887861252","state":["exists","up"]},{"osd":3,"uuid":"7e18c957-3091-46d5-8e17-8f3354c1726c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":812540271},{"type":"v1","addr":"172.21.15.164:6825","nonce":812540271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":812540271},{"type":"v1","addr":"172.21.15.164:6827","nonce":812540271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":812540271},{"type":"v1","addr":"172.21.15.164:6831","nonce":812540271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":812540271},{"type":"v1","addr":"172.21.15.164:6829","nonce":812540271}]},"public_addr":"172.21.15.164:6825/812540271","cluster_addr":"172.21.15.164:6827/812540271","heartbeat_back_addr":"172.21.15.164:6831/812540271","heartbeat_front_addr":"172.21.15.164:6829/812540271","state":["exists","up"]},{"osd":4,"uuid":"8e83f315-42bf-4471-9ce8-d81948da0f9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":2656857421},{"type":"v1","addr":"172.21.15.173:6825","nonce":2656857421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2656857421},{"type":"v1","addr":"172.21.15.173:6827","nonce":2656857421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2656857421},{"type":"v1","addr":"172.21.15.173:6831","nonce":2656857421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":2656857421},{"type":"v1","addr":"172.21.15.173:6829","nonce":2656857421}]},"public_addr":"172.21.15.173:6825/2656857421","cluster_addr":"172.21.15.173:6827/2656857421","heartbeat_back_addr":"172.21.15.173:6831/2656857421","heartbeat_front_addr":"172.21.15.173:6829/2656857421","state":["exists","up"]},{"osd":5,"uuid":"a230c872-71f9-40ab-8999-defd0162d4ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":3676483185},{"type":"v1","addr":"172.21.15.173:6801","nonce":3676483185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":3676483185},{"type":"v1","addr":"172.21.15.173:6803","nonce":3676483185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":3676483185},{"type":"v1","addr":"172.21.15.173:6807","nonce":3676483185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":3676483185},{"type":"v1","addr":"172.21.15.173:6805","nonce":3676483185}]},"public_addr":"172.21.15.173:6801/3676483185","cluster_addr":"172.21.15.173:6803/3676483185","heartbeat_back_addr":"172.21.15.173:6807/3676483185","heartbeat_front_addr":"172.21.15.173:6805/3676483185","state":["exists","up"]},{"osd":6,"uuid":"6193bfd7-8432-4f7f-a117-a3fcee21af99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":3631684087},{"type":"v1","addr":"172.21.15.173:6809","nonce":3631684087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":3631684087},{"type":"v1","addr":"172.21.15.173:6811","nonce":3631684087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":3631684087},{"type":"v1","addr":"172.21.15.173:6815","nonce":3631684087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":3631684087},{"type":"v1","addr":"172.21.15.173:6813","nonce":3631684087}]},"public_addr":"172.21.15.173:6809/3631684087","cluster_addr":"172.21.15.173:6811/3631684087","heartbeat_back_addr":"172.21.15.173:6815/3631684087","heartbeat_front_addr":"172.21.15.173:6813/3631684087","state":["exists","up"]},{"osd":7,"uuid":"b0f1b80a-afc4-4538-83b9-f081d2dcb8cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":3261114948},{"type":"v1","addr":"172.21.15.173:6817","nonce":3261114948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":3261114948},{"type":"v1","addr":"172.21.15.173:6819","nonce":3261114948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":3261114948},{"type":"v1","addr":"172.21.15.173:6823","nonce":3261114948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":3261114948},{"type":"v1","addr":"172.21.15.173:6821","nonce":3261114948}]},"public_addr":"172.21.15.173:6817/3261114948","cluster_addr":"172.21.15.173:6819/3261114948","heartbeat_back_addr":"172.21.15.173:6823/3261114948","heartbeat_front_addr":"172.21.15.173:6821/3261114948","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.188390+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.159932+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.191101+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.181776+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.224365+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.177515+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.153631+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.139972+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T14:57:21.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.649+0000 7f1486f20640 1 -- 172.21.15.164:0/1289044983 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f146c0672a0 msgr2=0x7f146c069760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:21.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.649+0000 7f1486f20640 1 --2- 172.21.15.164:0/1289044983 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f146c0672a0 0x7f146c069760 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f14700ae000 tx=0x7f14700b5040 comp rx=0 tx=0).stop 2024-03-23T14:57:21.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.649+0000 7f1486f20640 1 -- 172.21.15.164:0/1289044983 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1480148cf0 msgr2=0x7f14801e3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:21.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.649+0000 7f1486f20640 1 --2- 172.21.15.164:0/1289044983 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1480148cf0 0x7f14801e3960 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f147400c420 tx=0x7f147400c8e0 comp rx=0 tx=0).stop 2024-03-23T14:57:21.653 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.649+0000 7f1486f20640 1 -- 172.21.15.164:0/1289044983 shutdown_connections 2024-03-23T14:57:21.653 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.649+0000 7f1486f20640 1 --2- 172.21.15.164:0/1289044983 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f148014db70 0x7f148009af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:21.653 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.649+0000 7f1486f20640 1 --2- 172.21.15.164:0/1289044983 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1480148cf0 0x7f14801e3960 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:21.653 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.649+0000 7f1486f20640 1 --2- 172.21.15.164:0/1289044983 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f146c0672a0 0x7f146c069760 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:21.653 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.649+0000 7f1486f20640 1 --2- 172.21.15.164:0/1289044983 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f14800b0230 0x7f148009c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:21.653 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.649+0000 7f1486f20640 1 -- 172.21.15.164:0/1289044983 >> 172.21.15.164:0/1289044983 conn(0x7f14800b8ca0 msgr2=0x7f1480146140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:21.653 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.649+0000 7f1486f20640 1 -- 172.21.15.164:0/1289044983 shutdown_connections 2024-03-23T14:57:21.653 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.649+0000 7f1486f20640 1 -- 172.21.15.164:0/1289044983 wait complete. 2024-03-23T14:57:21.668 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T14:56:47.186157+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-23T14:56:56.994057+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-23T14:56:57.953862+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-23T14:57:21.668 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-23T14:57:21.792 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.785+0000 7fb42365d640 1 Processor -- start 2024-03-23T14:57:21.792 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.785+0000 7fb42365d640 1 -- start start 2024-03-23T14:57:21.794 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.789+0000 7fb42365d640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fb41c0b6740 0x7fb41c0b41d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:21.794 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.789+0000 7fb42365d640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb41c0b5530 con 0x7fb41c0b4710 2024-03-23T14:57:21.794 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.789+0000 7fb42365d640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb41c0b2a70 con 0x7fb41c0b6360 2024-03-23T14:57:21.794 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.789+0000 7fb42365d640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 -- 0x7fb41c0b5a60 con 0x7fb41c0b6740 2024-03-23T14:57:21.795 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.789+0000 7fb421bfa640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fb41c0b4710 0x7fb41c14fb90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:56904/0 (socket says 172.21.15.164:56904) 2024-03-23T14:57:21.795 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.789+0000 7fb421bfa640 1 -- 172.21.15.164:0/3561366116 learned_addr learned my addr 172.21.15.164:0/3561366116 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-03-23T14:57:21.795 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.789+0000 7fb420bf8640 1 --2- 172.21.15.164:0/3561366116 >> v2:172.21.15.173:3300/0 conn(0x7fb41c0b6740 0x7fb41c0b41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:21.795 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.789+0000 7fb413fff640 1 -- 172.21.15.164:0/3561366116 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3171454725 0 0) 0x7fb41c0b5530 con 0x7fb41c0b4710 2024-03-23T14:57:21.795 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.789+0000 7fb413fff640 1 -- 172.21.15.164:0/3561366116 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb408003530 con 0x7fb41c0b4710 2024-03-23T14:57:21.795 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.789+0000 7fb413fff640 1 -- 172.21.15.164:0/3561366116 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 81987171 0 0) 0x7fb408003530 con 0x7fb41c0b4710 2024-03-23T14:57:21.795 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.789+0000 7fb413fff640 1 -- 172.21.15.164:0/3561366116 >> v2:172.21.15.173:3300/0 conn(0x7fb41c0b6740 msgr2=0x7fb41c0b41d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:21.795 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.789+0000 7fb413fff640 1 --2- 172.21.15.164:0/3561366116 >> v2:172.21.15.173:3300/0 conn(0x7fb41c0b6740 0x7fb41c0b41d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:21.795 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.789+0000 7fb413fff640 1 -- 172.21.15.164:0/3561366116 >> v1:172.21.15.173:6789/0 conn(0x7fb41c0b6360 legacy=0x7fb41c145a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:21.795 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.789+0000 7fb413fff640 1 -- 172.21.15.164:0/3561366116 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb41c0b2800 con 0x7fb41c0b4710 2024-03-23T14:57:21.795 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.789+0000 7fb413fff640 1 -- 172.21.15.164:0/3561366116 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2526390548 0 0) 0x7fb414001a00 con 0x7fb41c0b4710 2024-03-23T14:57:21.795 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.789+0000 7fb420bf8640 1 --2- 172.21.15.164:0/3561366116 >> v2:172.21.15.173:3300/0 conn(0x7fb41c0b6740 0x7fb41c0b41d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T14:57:21.795 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.789+0000 7fb413fff640 1 -- 172.21.15.164:0/3561366116 >> v1:172.21.15.164:6789/0 conn(0x7fb41c0b4710 legacy=0x7fb41c14fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:21.795 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.789+0000 7fb413fff640 1 --2- 172.21.15.164:0/3561366116 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb408003980 0x7fb408003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:21.796 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.789+0000 7fb413fff640 1 --2- 172.21.15.164:0/3561366116 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb4080047d0 0x7fb408004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:21.796 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.789+0000 7fb413fff640 1 --2- 172.21.15.164:0/3561366116 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb408005190 0x7fb40800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:21.796 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.789+0000 7fb413fff640 1 -- 172.21.15.164:0/3561366116 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb41c0b5a60 con 0x7fb408003980 2024-03-23T14:57:21.796 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.789+0000 7fb413fff640 1 -- 172.21.15.164:0/3561366116 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fb40800c130 con 0x7fb408005190 2024-03-23T14:57:21.796 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.789+0000 7fb413fff640 1 -- 172.21.15.164:0/3561366116 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fb40800c2f0 con 0x7fb4080047d0 2024-03-23T14:57:21.796 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.789+0000 7fb4213f9640 1 --2- 172.21.15.164:0/3561366116 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb408003980 0x7fb408003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:21.796 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.789+0000 7fb420bf8640 1 --2- 172.21.15.164:0/3561366116 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb4080047d0 0x7fb408004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:21.796 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.789+0000 7fb421bfa640 1 --2- 172.21.15.164:0/3561366116 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb408005190 0x7fb40800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:21.796 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.789+0000 7fb4213f9640 1 -- 172.21.15.164:0/3561366116 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb4080047d0 msgr2=0x7fb408004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:21.796 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.789+0000 7fb4213f9640 1 --2- 172.21.15.164:0/3561366116 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb4080047d0 0x7fb408004c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:21.796 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.789+0000 7fb4213f9640 1 -- 172.21.15.164:0/3561366116 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb408005190 msgr2=0x7fb40800b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:21.796 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.789+0000 7fb4213f9640 1 --2- 172.21.15.164:0/3561366116 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb408005190 0x7fb40800b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:21.796 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.789+0000 7fb4213f9640 1 -- 172.21.15.164:0/3561366116 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb408003530 con 0x7fb408003980 2024-03-23T14:57:21.796 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.789+0000 7fb421bfa640 1 --2- 172.21.15.164:0/3561366116 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb408005190 0x7fb40800b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T14:57:21.796 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.789+0000 7fb421bfa640 1 -- 172.21.15.164:0/3561366116 reap_dead start 2024-03-23T14:57:21.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.793+0000 7fb4213f9640 1 --2- 172.21.15.164:0/3561366116 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb408003980 0x7fb408003da0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fb418001780 tx=0x7fb41801c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8776a5f6df3b733f server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:21.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.793+0000 7fb413fff640 1 -- 172.21.15.164:0/3561366116 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4180052d0 con 0x7fb408003980 2024-03-23T14:57:21.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.793+0000 7fb413fff640 1 -- 172.21.15.164:0/3561366116 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4180a7050 con 0x7fb408003980 2024-03-23T14:57:21.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.793+0000 7fb413fff640 1 -- 172.21.15.164:0/3561366116 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb418003600 con 0x7fb408003980 2024-03-23T14:57:21.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.793+0000 7fb42365d640 1 -- 172.21.15.164:0/3561366116 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb408003980 msgr2=0x7fb408003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:21.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.793+0000 7fb42365d640 1 --2- 172.21.15.164:0/3561366116 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb408003980 0x7fb408003da0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fb418001780 tx=0x7fb41801c400 comp rx=0 tx=0).stop 2024-03-23T14:57:21.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.793+0000 7fb42365d640 1 -- 172.21.15.164:0/3561366116 shutdown_connections 2024-03-23T14:57:21.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.793+0000 7fb42365d640 1 --2- 172.21.15.164:0/3561366116 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb408003980 0x7fb408003da0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:21.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.793+0000 7fb42365d640 1 -- 172.21.15.164:0/3561366116 >> 172.21.15.164:0/3561366116 conn(0x7fb41c0b7630 msgr2=0x7fb41c0b7a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:21.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.793+0000 7fb42365d640 1 -- 172.21.15.164:0/3561366116 shutdown_connections 2024-03-23T14:57:21.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.793+0000 7fb42365d640 1 -- 172.21.15.164:0/3561366116 wait complete. 2024-03-23T14:57:21.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.793+0000 7fb42365d640 1 Processor -- start 2024-03-23T14:57:21.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.793+0000 7fb42365d640 1 -- start start 2024-03-23T14:57:21.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.793+0000 7fb42365d640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb408003980 0x7fb41c1e1a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:21.801 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.793+0000 7fb42365d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb41c13fbf0 0x7fb41c13a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:21.801 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.793+0000 7fb42365d640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb41c1de510 0x7fb41c1de990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:21.801 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.793+0000 7fb42365d640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb41c13ac40 con 0x7fb41c13fbf0 2024-03-23T14:57:21.801 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.793+0000 7fb42365d640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fb41c1deed0 con 0x7fb408003980 2024-03-23T14:57:21.801 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.793+0000 7fb42365d640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fb41c1401b0 con 0x7fb41c1de510 2024-03-23T14:57:21.801 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.793+0000 7fb420bf8640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb41c13fbf0 0x7fb41c13a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:21.801 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.793+0000 7fb420bf8640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb41c13fbf0 0x7fb41c13a0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:60426/0 (socket says 172.21.15.164:60426) 2024-03-23T14:57:21.801 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.793+0000 7fb420bf8640 1 -- 172.21.15.164:0/2240079766 learned_addr learned my addr 172.21.15.164:0/2240079766 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:21.801 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.793+0000 7fb4213f9640 1 --2- 172.21.15.164:0/2240079766 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb408003980 0x7fb41c1e1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:21.801 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.793+0000 7fb421bfa640 1 --2- 172.21.15.164:0/2240079766 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb41c1de510 0x7fb41c1de990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:21.801 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.793+0000 7fb420bf8640 1 -- 172.21.15.164:0/2240079766 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb41c1de510 msgr2=0x7fb41c1de990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:21.801 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.793+0000 7fb420bf8640 1 --2- 172.21.15.164:0/2240079766 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb41c1de510 0x7fb41c1de990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:21.801 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.793+0000 7fb420bf8640 1 -- 172.21.15.164:0/2240079766 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb408003980 msgr2=0x7fb41c1e1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:21.801 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.793+0000 7fb420bf8640 1 --2- 172.21.15.164:0/2240079766 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb408003980 0x7fb41c1e1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:21.801 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.793+0000 7fb420bf8640 1 -- 172.21.15.164:0/2240079766 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb41c0b4c70 con 0x7fb41c13fbf0 2024-03-23T14:57:21.801 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.793+0000 7fb420bf8640 1 --2- 172.21.15.164:0/2240079766 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb41c13fbf0 0x7fb41c13a0b0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb40c010950 tx=0x7fb40c010e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:21.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.793+0000 7fb411ffb640 1 -- 172.21.15.164:0/2240079766 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb40c00ec20 con 0x7fb41c13fbf0 2024-03-23T14:57:21.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.793+0000 7fb411ffb640 1 -- 172.21.15.164:0/2240079766 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb40c00ed80 con 0x7fb41c13fbf0 2024-03-23T14:57:21.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.793+0000 7fb42365d640 1 -- 172.21.15.164:0/2240079766 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb41c1e2020 con 0x7fb41c13fbf0 2024-03-23T14:57:21.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.793+0000 7fb411ffb640 1 -- 172.21.15.164:0/2240079766 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb40c01c620 con 0x7fb41c13fbf0 2024-03-23T14:57:21.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.793+0000 7fb42365d640 1 -- 172.21.15.164:0/2240079766 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb41c1ddfc0 con 0x7fb41c13fbf0 2024-03-23T14:57:21.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.797+0000 7fb42365d640 1 -- 172.21.15.164:0/2240079766 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb41c0718c0 con 0x7fb41c13fbf0 2024-03-23T14:57:21.803 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.797+0000 7fb411ffb640 1 -- 172.21.15.164:0/2240079766 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7fb40c01c780 con 0x7fb41c13fbf0 2024-03-23T14:57:21.803 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.797+0000 7fb411ffb640 1 --2- 172.21.15.164:0/2240079766 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fb4000672d0 0x7fb400069790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:21.803 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.797+0000 7fb411ffb640 1 -- 172.21.15.164:0/2240079766 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fb40c085660 con 0x7fb41c13fbf0 2024-03-23T14:57:21.803 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.797+0000 7fb4213f9640 1 --2- 172.21.15.164:0/2240079766 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fb4000672d0 0x7fb400069790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:21.803 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.797+0000 7fb4213f9640 1 --2- 172.21.15.164:0/2240079766 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fb4000672d0 0x7fb400069790 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb418001750 tx=0x7fb4180011a0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:21.805 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.801+0000 7fb411ffb640 1 -- 172.21.15.164:0/2240079766 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7fb40c052350 con 0x7fb41c13fbf0 2024-03-23T14:57:21.964 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.961+0000 7fb42365d640 1 -- 172.21.15.164:0/2240079766 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb41c0690a0 con 0x7fb41c13fbf0 2024-03-23T14:57:21.965 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.961+0000 7fb411ffb640 1 -- 172.21.15.164:0/2240079766 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7fb40c055a10 con 0x7fb41c13fbf0 2024-03-23T14:57:21.965 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 1 2024-03-23T14:57:21.967 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.961+0000 7fb42365d640 1 -- 172.21.15.164:0/2240079766 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fb4000672d0 msgr2=0x7fb400069790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:21.967 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.961+0000 7fb42365d640 1 --2- 172.21.15.164:0/2240079766 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fb4000672d0 0x7fb400069790 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb418001750 tx=0x7fb4180011a0 comp rx=0 tx=0).stop 2024-03-23T14:57:21.967 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.961+0000 7fb42365d640 1 -- 172.21.15.164:0/2240079766 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb41c13fbf0 msgr2=0x7fb41c13a0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:21.967 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.961+0000 7fb42365d640 1 --2- 172.21.15.164:0/2240079766 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb41c13fbf0 0x7fb41c13a0b0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb40c010950 tx=0x7fb40c010e10 comp rx=0 tx=0).stop 2024-03-23T14:57:21.967 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.961+0000 7fb42365d640 1 -- 172.21.15.164:0/2240079766 shutdown_connections 2024-03-23T14:57:21.967 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.961+0000 7fb42365d640 1 --2- 172.21.15.164:0/2240079766 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb41c1de510 0x7fb41c1de990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:21.968 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.961+0000 7fb42365d640 1 --2- 172.21.15.164:0/2240079766 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb41c13fbf0 0x7fb41c13a0b0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:21.968 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.961+0000 7fb42365d640 1 --2- 172.21.15.164:0/2240079766 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fb4000672d0 0x7fb400069790 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:21.968 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.961+0000 7fb42365d640 1 --2- 172.21.15.164:0/2240079766 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb408003980 0x7fb41c1e1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:21.968 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.961+0000 7fb42365d640 1 -- 172.21.15.164:0/2240079766 >> 172.21.15.164:0/2240079766 conn(0x7fb41c0b7630 msgr2=0x7fb41c06e090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:21.968 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.961+0000 7fb42365d640 1 -- 172.21.15.164:0/2240079766 shutdown_connections 2024-03-23T14:57:21.968 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:21.961+0000 7fb42365d640 1 -- 172.21.15.164:0/2240079766 wait complete. 2024-03-23T14:57:21.984 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-23T14:57:22.100 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.097+0000 7f7b13161640 1 Processor -- start 2024-03-23T14:57:22.100 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.097+0000 7f7b13161640 1 -- start start 2024-03-23T14:57:22.101 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.097+0000 7f7b13161640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f7b0c14eb80 0x7f7b0c150f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:22.101 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.097+0000 7f7b13161640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b0c0b51d0 con 0x7f7b0c148cf0 2024-03-23T14:57:22.101 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.097+0000 7f7b13161640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b0c0b0610 con 0x7f7b0c0b0230 2024-03-23T14:57:22.102 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.097+0000 7f7b13161640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f7b0c0b1500 con 0x7f7b0c14eb80 2024-03-23T14:57:22.102 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.097+0000 7f7b116fe640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f7b0c14eb80 0x7f7b0c150f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:22.102 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.097+0000 7f7b0bfff640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f7b0c148cf0 0x7f7b0c149180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:56908/0 (socket says 172.21.15.164:56908) 2024-03-23T14:57:22.102 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.097+0000 7f7b0bfff640 1 -- 172.21.15.164:0/3629340552 learned_addr learned my addr 172.21.15.164:0/3629340552 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-03-23T14:57:22.102 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.097+0000 7f7b116fe640 1 -- 172.21.15.164:0/3629340552 >> v1:172.21.15.173:6789/0 conn(0x7f7b0c0b0230 legacy=0x7f7b0c0b34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:22.102 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.097+0000 7f7b116fe640 1 -- 172.21.15.164:0/3629340552 >> v1:172.21.15.164:6789/0 conn(0x7f7b0c148cf0 legacy=0x7f7b0c149180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:22.102 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.097+0000 7f7b0b7fe640 1 -- 172.21.15.164:0/3629340552 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 454767919 0 0) 0x7f7b0c0b51d0 con 0x7f7b0c148cf0 2024-03-23T14:57:22.102 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.097+0000 7f7b116fe640 1 -- 172.21.15.164:0/3629340552 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b0c06d250 con 0x7f7b0c14eb80 2024-03-23T14:57:22.104 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.101+0000 7f7b116fe640 1 --2- 172.21.15.164:0/3629340552 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b0c14eb80 0x7f7b0c150f70 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c0b0800 tx=0x7f7b0401c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f47b6ea5d559315 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:22.104 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.101+0000 7f7b0b7fe640 1 -- 172.21.15.164:0/3629340552 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b040a58c0 con 0x7f7b0c14eb80 2024-03-23T14:57:22.104 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.101+0000 7f7b0b7fe640 1 -- 172.21.15.164:0/3629340552 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b040a5a20 con 0x7f7b0c14eb80 2024-03-23T14:57:22.104 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.101+0000 7f7b0b7fe640 1 -- 172.21.15.164:0/3629340552 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b040a5db0 con 0x7f7b0c14eb80 2024-03-23T14:57:22.104 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.101+0000 7f7b13161640 1 -- 172.21.15.164:0/3629340552 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b0c14eb80 msgr2=0x7f7b0c150f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:22.105 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.101+0000 7f7b13161640 1 --2- 172.21.15.164:0/3629340552 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b0c14eb80 0x7f7b0c150f70 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c0b0800 tx=0x7f7b0401c800 comp rx=0 tx=0).stop 2024-03-23T14:57:22.105 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.101+0000 7f7b13161640 1 -- 172.21.15.164:0/3629340552 shutdown_connections 2024-03-23T14:57:22.105 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.101+0000 7f7b13161640 1 --2- 172.21.15.164:0/3629340552 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b0c14eb80 0x7f7b0c150f70 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:22.105 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.101+0000 7f7b13161640 1 -- 172.21.15.164:0/3629340552 >> 172.21.15.164:0/3629340552 conn(0x7f7b0c0b8ca0 msgr2=0x7f7b0c0b90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:22.105 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.101+0000 7f7b13161640 1 -- 172.21.15.164:0/3629340552 shutdown_connections 2024-03-23T14:57:22.105 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.101+0000 7f7b13161640 1 -- 172.21.15.164:0/3629340552 wait complete. 2024-03-23T14:57:22.105 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.101+0000 7f7b13161640 1 Processor -- start 2024-03-23T14:57:22.105 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.101+0000 7f7b13161640 1 -- start start 2024-03-23T14:57:22.107 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.101+0000 7f7b13161640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7b0c0b0230 0x7f7b0c144bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:22.107 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.101+0000 7f7b13161640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b0c148cf0 0x7f7b0c1404b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:22.107 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.101+0000 7f7b13161640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7b0c14eb80 0x7f7b0c141370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:22.108 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.101+0000 7f7b13161640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f7b0c1459d0 con 0x7f7b0c148cf0 2024-03-23T14:57:22.108 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.101+0000 7f7b13161640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f7b0c13fe20 con 0x7f7b0c14eb80 2024-03-23T14:57:22.108 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.101+0000 7f7b13161640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f7b0c1409f0 con 0x7f7b0c0b0230 2024-03-23T14:57:22.108 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.101+0000 7f7b0bfff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b0c148cf0 0x7f7b0c1404b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:22.108 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.101+0000 7f7b0bfff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b0c148cf0 0x7f7b0c1404b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:60434/0 (socket says 172.21.15.164:60434) 2024-03-23T14:57:22.108 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.101+0000 7f7b0bfff640 1 -- 172.21.15.164:0/272813012 learned_addr learned my addr 172.21.15.164:0/272813012 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:22.108 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.101+0000 7f7b116fe640 1 --2- 172.21.15.164:0/272813012 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7b0c14eb80 0x7f7b0c141370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:22.108 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.101+0000 7f7b10efd640 1 --2- 172.21.15.164:0/272813012 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7b0c0b0230 0x7f7b0c144bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:22.108 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.101+0000 7f7b0bfff640 1 -- 172.21.15.164:0/272813012 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7b0c0b0230 msgr2=0x7f7b0c144bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:22.108 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.101+0000 7f7b0bfff640 1 --2- 172.21.15.164:0/272813012 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7b0c0b0230 0x7f7b0c144bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:22.108 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.101+0000 7f7b0bfff640 1 -- 172.21.15.164:0/272813012 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7b0c14eb80 msgr2=0x7f7b0c141370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:22.108 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.101+0000 7f7b0bfff640 1 --2- 172.21.15.164:0/272813012 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7b0c14eb80 0x7f7b0c141370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:22.108 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.101+0000 7f7b0bfff640 1 -- 172.21.15.164:0/272813012 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b04009a70 con 0x7f7b0c148cf0 2024-03-23T14:57:22.108 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.101+0000 7f7b0bfff640 1 --2- 172.21.15.164:0/272813012 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b0c148cf0 0x7f7b0c1404b0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7b0000c4a0 tx=0x7f7b0000c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:22.108 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.101+0000 7f7b097fa640 1 -- 172.21.15.164:0/272813012 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b000074e0 con 0x7f7b0c148cf0 2024-03-23T14:57:22.109 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.101+0000 7f7b097fa640 1 -- 172.21.15.164:0/272813012 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b00007640 con 0x7f7b0c148cf0 2024-03-23T14:57:22.109 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.101+0000 7f7b13161640 1 -- 172.21.15.164:0/272813012 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b0c0b0f70 con 0x7f7b0c148cf0 2024-03-23T14:57:22.109 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.101+0000 7f7b097fa640 1 -- 172.21.15.164:0/272813012 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b00004a30 con 0x7f7b0c148cf0 2024-03-23T14:57:22.109 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.101+0000 7f7b13161640 1 -- 172.21.15.164:0/272813012 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b0c09c1f0 con 0x7f7b0c148cf0 2024-03-23T14:57:22.109 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.101+0000 7f7b13161640 1 -- 172.21.15.164:0/272813012 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b0c09c050 con 0x7f7b0c148cf0 2024-03-23T14:57:22.109 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.105+0000 7f7b097fa640 1 -- 172.21.15.164:0/272813012 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7f7b00004b90 con 0x7f7b0c148cf0 2024-03-23T14:57:22.110 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.105+0000 7f7b097fa640 1 --2- 172.21.15.164:0/272813012 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f7af80672a0 0x7f7af8069760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:22.110 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.105+0000 7f7b10efd640 1 --2- 172.21.15.164:0/272813012 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f7af80672a0 0x7f7af8069760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:22.110 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.105+0000 7f7b097fa640 1 -- 172.21.15.164:0/272813012 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f7b000842f0 con 0x7f7b0c148cf0 2024-03-23T14:57:22.110 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.105+0000 7f7b10efd640 1 --2- 172.21.15.164:0/272813012 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f7af80672a0 0x7f7af8069760 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c09c590 tx=0x7f7afc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:22.112 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.109+0000 7f7b097fa640 1 -- 172.21.15.164:0/272813012 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7f7b0c09c050 con 0x7f7b0c148cf0 2024-03-23T14:57:22.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.253+0000 7f7b13161640 1 -- 172.21.15.164:0/272813012 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f7b0c0690a0 con 0x7f7b0c148cf0 2024-03-23T14:57:22.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.257+0000 7f7b097fa640 1 -- 172.21.15.164:0/272813012 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f7b00050f30 con 0x7f7b0c148cf0 2024-03-23T14:57:22.260 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 32 2024-03-23T14:57:22.262 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.257+0000 7f7b13161640 1 -- 172.21.15.164:0/272813012 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f7af80672a0 msgr2=0x7f7af8069760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:22.262 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.257+0000 7f7b13161640 1 --2- 172.21.15.164:0/272813012 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f7af80672a0 0x7f7af8069760 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c09c590 tx=0x7f7afc009040 comp rx=0 tx=0).stop 2024-03-23T14:57:22.263 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.257+0000 7f7b13161640 1 -- 172.21.15.164:0/272813012 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b0c148cf0 msgr2=0x7f7b0c1404b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:22.263 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.257+0000 7f7b13161640 1 --2- 172.21.15.164:0/272813012 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b0c148cf0 0x7f7b0c1404b0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7b0000c4a0 tx=0x7f7b0000c960 comp rx=0 tx=0).stop 2024-03-23T14:57:22.263 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.257+0000 7f7b13161640 1 -- 172.21.15.164:0/272813012 shutdown_connections 2024-03-23T14:57:22.263 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.257+0000 7f7b13161640 1 --2- 172.21.15.164:0/272813012 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f7af80672a0 0x7f7af8069760 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:22.263 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.257+0000 7f7b13161640 1 --2- 172.21.15.164:0/272813012 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7b0c14eb80 0x7f7b0c141370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:22.263 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.257+0000 7f7b13161640 1 --2- 172.21.15.164:0/272813012 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b0c148cf0 0x7f7b0c1404b0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:22.263 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.257+0000 7f7b13161640 1 --2- 172.21.15.164:0/272813012 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7b0c0b0230 0x7f7b0c144bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:22.263 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.257+0000 7f7b13161640 1 -- 172.21.15.164:0/272813012 >> 172.21.15.164:0/272813012 conn(0x7f7b0c0b8ca0 msgr2=0x7f7b0c0b43f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:22.263 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.257+0000 7f7b13161640 1 -- 172.21.15.164:0/272813012 shutdown_connections 2024-03-23T14:57:22.263 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.257+0000 7f7b13161640 1 -- 172.21.15.164:0/272813012 wait complete. 2024-03-23T14:57:22.277 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-23T14:57:22.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.393+0000 7fc570c61640 1 Processor -- start 2024-03-23T14:57:22.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.393+0000 7fc570c61640 1 -- start start 2024-03-23T14:57:22.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.393+0000 7fc570c61640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fc56c0add50 0x7fc56c147770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:22.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.393+0000 7fc570c61640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc56c06d650 con 0x7fc56c09def0 2024-03-23T14:57:22.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.393+0000 7fc570c61640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc56c0af4a0 con 0x7fc56c147cb0 2024-03-23T14:57:22.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.393+0000 7fc570c61640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 -- 0x7fc56c09e2d0 con 0x7fc56c0add50 2024-03-23T14:57:22.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.393+0000 7fc56a59c640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fc56c09def0 0x7fc56c0b45e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:56922/0 (socket says 172.21.15.164:56922) 2024-03-23T14:57:22.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.393+0000 7fc56a59c640 1 -- 172.21.15.164:0/4250568298 learned_addr learned my addr 172.21.15.164:0/4250568298 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-03-23T14:57:22.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.393+0000 7fc569d9b640 1 --2- 172.21.15.164:0/4250568298 >> v2:172.21.15.173:3300/0 conn(0x7fc56c0add50 0x7fc56c147770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:22.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.393+0000 7fc56959a640 1 -- 172.21.15.164:0/4250568298 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1415288553 0 0) 0x7fc56c06d650 con 0x7fc56c09def0 2024-03-23T14:57:22.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.393+0000 7fc56959a640 1 -- 172.21.15.164:0/4250568298 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc554003530 con 0x7fc56c09def0 2024-03-23T14:57:22.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.393+0000 7fc56959a640 1 -- 172.21.15.164:0/4250568298 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3582853925 0 0) 0x7fc554003530 con 0x7fc56c09def0 2024-03-23T14:57:22.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.393+0000 7fc56959a640 1 -- 172.21.15.164:0/4250568298 >> v2:172.21.15.173:3300/0 conn(0x7fc56c0add50 msgr2=0x7fc56c147770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:22.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.393+0000 7fc56959a640 1 --2- 172.21.15.164:0/4250568298 >> v2:172.21.15.173:3300/0 conn(0x7fc56c0add50 0x7fc56c147770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:22.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.393+0000 7fc56959a640 1 -- 172.21.15.164:0/4250568298 >> v1:172.21.15.173:6789/0 conn(0x7fc56c147cb0 legacy=0x7fc56c14ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:22.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.393+0000 7fc56959a640 1 -- 172.21.15.164:0/4250568298 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc56c0aeca0 con 0x7fc56c09def0 2024-03-23T14:57:22.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.393+0000 7fc56959a640 1 -- 172.21.15.164:0/4250568298 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2526390548 0 0) 0x7fc564001a00 con 0x7fc56c09def0 2024-03-23T14:57:22.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.393+0000 7fc56959a640 1 -- 172.21.15.164:0/4250568298 >> v1:172.21.15.164:6789/0 conn(0x7fc56c09def0 legacy=0x7fc56c0b45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:22.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.393+0000 7fc56959a640 1 --2- 172.21.15.164:0/4250568298 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc554003980 0x7fc554003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:22.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.393+0000 7fc56959a640 1 --2- 172.21.15.164:0/4250568298 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc5540047d0 0x7fc554004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:22.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.393+0000 7fc56959a640 1 --2- 172.21.15.164:0/4250568298 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc5540052b0 0x7fc55400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:22.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.393+0000 7fc56959a640 1 -- 172.21.15.164:0/4250568298 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc56c09e2d0 con 0x7fc5540052b0 2024-03-23T14:57:22.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.393+0000 7fc56959a640 1 -- 172.21.15.164:0/4250568298 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fc55400c180 con 0x7fc554003980 2024-03-23T14:57:22.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.393+0000 7fc56959a640 1 -- 172.21.15.164:0/4250568298 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fc55400c340 con 0x7fc5540047d0 2024-03-23T14:57:22.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.397+0000 7fc56ad9d640 1 --2- 172.21.15.164:0/4250568298 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc5540052b0 0x7fc55400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:22.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.397+0000 7fc56ad9d640 1 -- 172.21.15.164:0/4250568298 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc5540047d0 msgr2=0x7fc554004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:22.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.397+0000 7fc569d9b640 1 --2- 172.21.15.164:0/4250568298 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc5540047d0 0x7fc554004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:22.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.397+0000 7fc56ad9d640 1 --2- 172.21.15.164:0/4250568298 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc5540047d0 0x7fc554004c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:22.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.397+0000 7fc56ad9d640 1 -- 172.21.15.164:0/4250568298 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc554003980 msgr2=0x7fc554003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:22.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.397+0000 7fc56ad9d640 1 --2- 172.21.15.164:0/4250568298 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc554003980 0x7fc554003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:22.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.397+0000 7fc56ad9d640 1 -- 172.21.15.164:0/4250568298 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc554003530 con 0x7fc5540052b0 2024-03-23T14:57:22.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.397+0000 7fc56ad9d640 1 -- 172.21.15.164:0/4250568298 reap_dead start 2024-03-23T14:57:22.404 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.397+0000 7fc56ad9d640 1 --2- 172.21.15.164:0/4250568298 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc5540052b0 0x7fc55400b990 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc55c001750 tx=0x7fc55c01c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=531287cafed97d7d server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:22.404 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.397+0000 7fc56959a640 1 -- 172.21.15.164:0/4250568298 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc55c0052d0 con 0x7fc5540052b0 2024-03-23T14:57:22.404 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.397+0000 7fc56959a640 1 -- 172.21.15.164:0/4250568298 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc55c0a7050 con 0x7fc5540052b0 2024-03-23T14:57:22.404 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.397+0000 7fc56959a640 1 -- 172.21.15.164:0/4250568298 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc55c003600 con 0x7fc5540052b0 2024-03-23T14:57:22.404 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.397+0000 7fc570c61640 1 -- 172.21.15.164:0/4250568298 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc5540052b0 msgr2=0x7fc55400b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:22.404 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.397+0000 7fc570c61640 1 --2- 172.21.15.164:0/4250568298 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc5540052b0 0x7fc55400b990 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc55c001750 tx=0x7fc55c01c3e0 comp rx=0 tx=0).stop 2024-03-23T14:57:22.404 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.397+0000 7fc570c61640 1 -- 172.21.15.164:0/4250568298 shutdown_connections 2024-03-23T14:57:22.404 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.397+0000 7fc570c61640 1 --2- 172.21.15.164:0/4250568298 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc5540052b0 0x7fc55400b990 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:22.404 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.397+0000 7fc570c61640 1 -- 172.21.15.164:0/4250568298 >> 172.21.15.164:0/4250568298 conn(0x7fc56c0b8ca0 msgr2=0x7fc56c0b90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:22.404 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.397+0000 7fc570c61640 1 -- 172.21.15.164:0/4250568298 shutdown_connections 2024-03-23T14:57:22.404 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.397+0000 7fc570c61640 1 -- 172.21.15.164:0/4250568298 wait complete. 2024-03-23T14:57:22.404 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.401+0000 7fc570c61640 1 Processor -- start 2024-03-23T14:57:22.404 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.401+0000 7fc570c61640 1 -- start start 2024-03-23T14:57:22.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.401+0000 7fc570c61640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc5540052b0 0x7fc56c09b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:22.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.401+0000 7fc570c61640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc56c1415b0 0x7fc56c09c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:22.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.401+0000 7fc570c61640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc56c09cb50 0x7fc56c134bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:22.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.401+0000 7fc570c61640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc56c0b87f0 con 0x7fc56c1415b0 2024-03-23T14:57:22.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.401+0000 7fc570c61640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fc56c09aa50 con 0x7fc56c09cb50 2024-03-23T14:57:22.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.401+0000 7fc570c61640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fc56c09cfd0 con 0x7fc5540052b0 2024-03-23T14:57:22.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.401+0000 7fc569d9b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc56c1415b0 0x7fc56c09c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:22.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.401+0000 7fc569d9b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc56c1415b0 0x7fc56c09c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:60446/0 (socket says 172.21.15.164:60446) 2024-03-23T14:57:22.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.401+0000 7fc569d9b640 1 -- 172.21.15.164:0/547651586 learned_addr learned my addr 172.21.15.164:0/547651586 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:22.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.401+0000 7fc56ad9d640 1 --2- 172.21.15.164:0/547651586 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc56c09cb50 0x7fc56c134bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:22.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.401+0000 7fc569d9b640 1 -- 172.21.15.164:0/547651586 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc5540052b0 msgr2=0x7fc56c09b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:22.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.401+0000 7fc569d9b640 1 --2- 172.21.15.164:0/547651586 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc5540052b0 0x7fc56c09b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:22.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.401+0000 7fc569d9b640 1 -- 172.21.15.164:0/547651586 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc56c09cb50 msgr2=0x7fc56c134bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:22.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.401+0000 7fc569d9b640 1 --2- 172.21.15.164:0/547651586 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc56c09cb50 0x7fc56c134bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:22.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.401+0000 7fc569d9b640 1 -- 172.21.15.164:0/547651586 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc56c09b860 con 0x7fc56c1415b0 2024-03-23T14:57:22.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.401+0000 7fc569d9b640 1 --2- 172.21.15.164:0/547651586 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc56c1415b0 0x7fc56c09c610 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc5600108c0 tx=0x7fc560010d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:22.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.401+0000 7fc55affd640 1 -- 172.21.15.164:0/547651586 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc560009630 con 0x7fc56c1415b0 2024-03-23T14:57:22.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.401+0000 7fc55affd640 1 -- 172.21.15.164:0/547651586 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc56000eb60 con 0x7fc56c1415b0 2024-03-23T14:57:22.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.401+0000 7fc570c61640 1 -- 172.21.15.164:0/547651586 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc56c09bb30 con 0x7fc56c1415b0 2024-03-23T14:57:22.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.401+0000 7fc55affd640 1 -- 172.21.15.164:0/547651586 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc56001c400 con 0x7fc56c1415b0 2024-03-23T14:57:22.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.401+0000 7fc570c61640 1 -- 172.21.15.164:0/547651586 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc56c145190 con 0x7fc56c1415b0 2024-03-23T14:57:22.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.401+0000 7fc570c61640 1 -- 172.21.15.164:0/547651586 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc56c14fd60 con 0x7fc56c1415b0 2024-03-23T14:57:22.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.401+0000 7fc55affd640 1 -- 172.21.15.164:0/547651586 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7fc56001c560 con 0x7fc56c1415b0 2024-03-23T14:57:22.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.401+0000 7fc55affd640 1 --2- 172.21.15.164:0/547651586 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fc54c0672a0 0x7fc54c069760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:22.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.401+0000 7fc55affd640 1 -- 172.21.15.164:0/547651586 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fc560085580 con 0x7fc56c1415b0 2024-03-23T14:57:22.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.401+0000 7fc56a59c640 1 --2- 172.21.15.164:0/547651586 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fc54c0672a0 0x7fc54c069760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:22.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.401+0000 7fc56a59c640 1 --2- 172.21.15.164:0/547651586 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fc54c0672a0 0x7fc54c069760 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc564001360 tx=0x7fc564007bc0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:22.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.405+0000 7fc55affd640 1 -- 172.21.15.164:0/547651586 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7fc5600521c0 con 0x7fc56c1415b0 2024-03-23T14:57:22.565 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.561+0000 7fc570c61640 1 -- 172.21.15.164:0/547651586 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc56c0690a0 con 0x7fc56c1415b0 2024-03-23T14:57:22.565 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.561+0000 7fc55affd640 1 -- 172.21.15.164:0/547651586 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7fc560055880 con 0x7fc56c1415b0 2024-03-23T14:57:22.566 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 64 2024-03-23T14:57:22.568 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.561+0000 7fc570c61640 1 -- 172.21.15.164:0/547651586 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fc54c0672a0 msgr2=0x7fc54c069760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:22.568 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.561+0000 7fc570c61640 1 --2- 172.21.15.164:0/547651586 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fc54c0672a0 0x7fc54c069760 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc564001360 tx=0x7fc564007bc0 comp rx=0 tx=0).stop 2024-03-23T14:57:22.568 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.561+0000 7fc570c61640 1 -- 172.21.15.164:0/547651586 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc56c1415b0 msgr2=0x7fc56c09c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:22.568 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.561+0000 7fc570c61640 1 --2- 172.21.15.164:0/547651586 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc56c1415b0 0x7fc56c09c610 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc5600108c0 tx=0x7fc560010d80 comp rx=0 tx=0).stop 2024-03-23T14:57:22.568 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.561+0000 7fc570c61640 1 -- 172.21.15.164:0/547651586 shutdown_connections 2024-03-23T14:57:22.568 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.561+0000 7fc570c61640 1 --2- 172.21.15.164:0/547651586 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fc54c0672a0 0x7fc54c069760 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:22.568 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.561+0000 7fc570c61640 1 --2- 172.21.15.164:0/547651586 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc56c09cb50 0x7fc56c134bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:22.568 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.561+0000 7fc570c61640 1 --2- 172.21.15.164:0/547651586 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc56c1415b0 0x7fc56c09c610 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:22.569 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.561+0000 7fc570c61640 1 --2- 172.21.15.164:0/547651586 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc5540052b0 0x7fc56c09b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:22.569 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.561+0000 7fc570c61640 1 -- 172.21.15.164:0/547651586 >> 172.21.15.164:0/547651586 conn(0x7fc56c0b8ca0 msgr2=0x7fc56c145ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:22.569 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.565+0000 7fc570c61640 1 -- 172.21.15.164:0/547651586 shutdown_connections 2024-03-23T14:57:22.569 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.565+0000 7fc570c61640 1 -- 172.21.15.164:0/547651586 wait complete. 2024-03-23T14:57:22.608 INFO:tasks.cephfs_test_runner:Starting test: test_cd_with_args (tasks.cephfs.test_cephfs_shell.TestCD) Test that when cd is issued with an argument, CWD is changed 2024-03-23T14:57:22.608 INFO:tasks.cephfs_test_runner:test_cd_with_args (tasks.cephfs.test_cephfs_shell.TestCD) 2024-03-23T14:57:22.608 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T14:57:22.715 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.709+0000 7f0d2afbc640 1 Processor -- start 2024-03-23T14:57:22.715 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.709+0000 7f0d2afbc640 1 -- start start 2024-03-23T14:57:22.718 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.713+0000 7f0d2afbc640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0d24148cf0 0x7f0d24149180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:22.718 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.713+0000 7f0d2afbc640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f0d2414db70 0x7f0d2414ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:22.718 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.713+0000 7f0d2afbc640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d240b51d0 con 0x7f0d240b0230 2024-03-23T14:57:22.718 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.713+0000 7f0d2afbc640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 -- 0x7f0d240b0610 con 0x7f0d2414db70 2024-03-23T14:57:22.718 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.713+0000 7f0d2afbc640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f0d240b1500 con 0x7f0d24148cf0 2024-03-23T14:57:22.718 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.713+0000 7f0d28d58640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f0d240b0230 0x7f0d240b34d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:42420/0 (socket says 172.21.15.164:42420) 2024-03-23T14:57:22.718 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.713+0000 7f0d28d58640 1 -- 172.21.15.164:0/685039651 learned_addr learned my addr 172.21.15.164:0/685039651 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-03-23T14:57:22.718 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.713+0000 7f0d29559640 1 --2- 172.21.15.164:0/685039651 >> v2:172.21.15.173:3300/0 conn(0x7f0d2414db70 0x7f0d2414ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:22.718 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.713+0000 7f0d23fff640 1 --2- 172.21.15.164:0/685039651 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0d24148cf0 0x7f0d24149180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:22.718 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.713+0000 7f0d237fe640 1 -- 172.21.15.164:0/685039651 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1308029940 0 0) 0x7f0d240b51d0 con 0x7f0d240b0230 2024-03-23T14:57:22.718 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.713+0000 7f0d237fe640 1 -- 172.21.15.164:0/685039651 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0d0c003530 con 0x7f0d240b0230 2024-03-23T14:57:22.719 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.713+0000 7f0d237fe640 1 -- 172.21.15.164:0/685039651 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2878281756 0 0) 0x7f0d0c003530 con 0x7f0d240b0230 2024-03-23T14:57:22.719 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.713+0000 7f0d237fe640 1 -- 172.21.15.164:0/685039651 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0d24148cf0 msgr2=0x7f0d24149180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:22.719 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.713+0000 7f0d237fe640 1 --2- 172.21.15.164:0/685039651 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0d24148cf0 0x7f0d24149180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:22.719 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.713+0000 7f0d237fe640 1 -- 172.21.15.164:0/685039651 >> v2:172.21.15.173:3300/0 conn(0x7f0d2414db70 msgr2=0x7f0d2414ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:22.719 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.713+0000 7f0d237fe640 1 --2- 172.21.15.164:0/685039651 >> v2:172.21.15.173:3300/0 conn(0x7f0d2414db70 0x7f0d2414ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:22.719 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.713+0000 7f0d237fe640 1 -- 172.21.15.164:0/685039651 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d2406d250 con 0x7f0d240b0230 2024-03-23T14:57:22.719 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.713+0000 7f0d237fe640 1 -- 172.21.15.164:0/685039651 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2526390548 0 0) 0x7f0d18001680 con 0x7f0d240b0230 2024-03-23T14:57:22.719 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.713+0000 7f0d237fe640 1 -- 172.21.15.164:0/685039651 >> v1:172.21.15.164:6789/0 conn(0x7f0d240b0230 legacy=0x7f0d240b34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:22.719 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.713+0000 7f0d237fe640 1 --2- 172.21.15.164:0/685039651 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0d0c003980 0x7f0d0c003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:22.719 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.713+0000 7f0d237fe640 1 --2- 172.21.15.164:0/685039651 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0d0c0047d0 0x7f0d0c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:22.719 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.713+0000 7f0d237fe640 1 --2- 172.21.15.164:0/685039651 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0d24148cf0 0x7f0d0c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:22.719 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.713+0000 7f0d237fe640 1 -- 172.21.15.164:0/685039651 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0d240b0610 con 0x7f0d0c003980 2024-03-23T14:57:22.720 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.713+0000 7f0d237fe640 1 -- 172.21.15.164:0/685039651 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f0d240b1500 con 0x7f0d0c0047d0 2024-03-23T14:57:22.720 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.713+0000 7f0d237fe640 1 -- 172.21.15.164:0/685039651 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f0d0c00b9c0 con 0x7f0d24148cf0 2024-03-23T14:57:22.720 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.713+0000 7f0d28d58640 1 --2- 172.21.15.164:0/685039651 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0d0c003980 0x7f0d0c003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:22.720 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.713+0000 7f0d23fff640 1 --2- 172.21.15.164:0/685039651 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0d0c0047d0 0x7f0d0c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:22.720 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.713+0000 7f0d28d58640 1 -- 172.21.15.164:0/685039651 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0d24148cf0 msgr2=0x7f0d0c005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:22.720 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.713+0000 7f0d28d58640 1 --2- 172.21.15.164:0/685039651 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0d24148cf0 0x7f0d0c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:22.720 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.713+0000 7f0d28d58640 1 -- 172.21.15.164:0/685039651 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0d0c0047d0 msgr2=0x7f0d0c004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:22.720 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.713+0000 7f0d28d58640 1 --2- 172.21.15.164:0/685039651 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0d0c0047d0 0x7f0d0c004c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:22.720 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.713+0000 7f0d28d58640 1 -- 172.21.15.164:0/685039651 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d0c003530 con 0x7f0d0c003980 2024-03-23T14:57:22.724 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.713+0000 7f0d28d58640 1 --2- 172.21.15.164:0/685039651 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0d0c003980 0x7f0d0c003da0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f0d1800c290 tx=0x7f0d1801cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8703105518408d9a server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:22.724 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.713+0000 7f0d237fe640 1 -- 172.21.15.164:0/685039651 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d180058e0 con 0x7f0d0c003980 2024-03-23T14:57:22.724 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.713+0000 7f0d237fe640 1 -- 172.21.15.164:0/685039651 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d18005a40 con 0x7f0d0c003980 2024-03-23T14:57:22.724 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.713+0000 7f0d237fe640 1 -- 172.21.15.164:0/685039651 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d18005df0 con 0x7f0d0c003980 2024-03-23T14:57:22.725 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.717+0000 7f0d2afbc640 1 -- 172.21.15.164:0/685039651 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0d0c003980 msgr2=0x7f0d0c003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:22.725 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.717+0000 7f0d2afbc640 1 --2- 172.21.15.164:0/685039651 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0d0c003980 0x7f0d0c003da0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f0d1800c290 tx=0x7f0d1801cc40 comp rx=0 tx=0).stop 2024-03-23T14:57:22.725 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.717+0000 7f0d29559640 1 -- 172.21.15.164:0/685039651 reap_dead start 2024-03-23T14:57:22.725 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.717+0000 7f0d2afbc640 1 -- 172.21.15.164:0/685039651 shutdown_connections 2024-03-23T14:57:22.725 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.717+0000 7f0d2afbc640 1 -- 172.21.15.164:0/685039651 >> 172.21.15.164:0/685039651 conn(0x7f0d240b8ca0 msgr2=0x7f0d240b90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:22.725 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.717+0000 7f0d2afbc640 1 -- 172.21.15.164:0/685039651 shutdown_connections 2024-03-23T14:57:22.725 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.717+0000 7f0d2afbc640 1 -- 172.21.15.164:0/685039651 wait complete. 2024-03-23T14:57:22.725 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.717+0000 7f0d2afbc640 1 Processor -- start 2024-03-23T14:57:22.726 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.717+0000 7f0d2afbc640 1 -- start start 2024-03-23T14:57:22.726 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.717+0000 7f0d2afbc640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0d0c003980 0x7f0d2414af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:22.726 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.717+0000 7f0d2afbc640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0d2414b650 0x7f0d240ae5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:22.726 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.717+0000 7f0d2afbc640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0d241db680 0x7f0d2409d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:22.726 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.717+0000 7f0d2afbc640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0d240adda0 con 0x7f0d241db680 2024-03-23T14:57:22.726 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.717+0000 7f0d2afbc640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f0d240ae080 con 0x7f0d2414b650 2024-03-23T14:57:22.726 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.717+0000 7f0d2afbc640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f0d240af8f0 con 0x7f0d0c003980 2024-03-23T14:57:22.726 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.717+0000 7f0d29559640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0d241db680 0x7f0d2409d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:22.726 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.717+0000 7f0d29559640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0d241db680 0x7f0d2409d130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:49066/0 (socket says 172.21.15.164:49066) 2024-03-23T14:57:22.727 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.717+0000 7f0d29559640 1 -- 172.21.15.164:0/928145677 learned_addr learned my addr 172.21.15.164:0/928145677 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:22.727 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.717+0000 7f0d28d58640 1 --2- 172.21.15.164:0/928145677 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0d0c003980 0x7f0d2414af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:22.727 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.717+0000 7f0d23fff640 1 --2- 172.21.15.164:0/928145677 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0d2414b650 0x7f0d240ae5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:22.727 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.717+0000 7f0d29559640 1 -- 172.21.15.164:0/928145677 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0d0c003980 msgr2=0x7f0d2414af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:22.727 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.717+0000 7f0d29559640 1 --2- 172.21.15.164:0/928145677 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0d0c003980 0x7f0d2414af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:22.727 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.717+0000 7f0d29559640 1 -- 172.21.15.164:0/928145677 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0d2414b650 msgr2=0x7f0d240ae5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:22.727 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.717+0000 7f0d29559640 1 --2- 172.21.15.164:0/928145677 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0d2414b650 0x7f0d240ae5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:22.727 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.717+0000 7f0d29559640 1 -- 172.21.15.164:0/928145677 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d240b0f70 con 0x7f0d241db680 2024-03-23T14:57:22.727 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.717+0000 7f0d28d58640 1 --2- 172.21.15.164:0/928145677 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0d0c003980 0x7f0d2414af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T14:57:22.727 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.717+0000 7f0d29559640 1 --2- 172.21.15.164:0/928145677 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0d241db680 0x7f0d2409d130 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f0d240afea0 tx=0x7f0d1c00b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:22.727 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.717+0000 7f0d217fa640 1 -- 172.21.15.164:0/928145677 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d1c010040 con 0x7f0d241db680 2024-03-23T14:57:22.728 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.717+0000 7f0d217fa640 1 -- 172.21.15.164:0/928145677 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d1c00bce0 con 0x7f0d241db680 2024-03-23T14:57:22.728 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.717+0000 7f0d2afbc640 1 -- 172.21.15.164:0/928145677 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d240b81d0 con 0x7f0d241db680 2024-03-23T14:57:22.728 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.717+0000 7f0d217fa640 1 -- 172.21.15.164:0/928145677 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d1c014640 con 0x7f0d241db680 2024-03-23T14:57:22.728 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.717+0000 7f0d2afbc640 1 -- 172.21.15.164:0/928145677 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d240ada30 con 0x7f0d241db680 2024-03-23T14:57:22.728 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.717+0000 7f0d2afbc640 1 -- 172.21.15.164:0/928145677 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d240690a0 con 0x7f0d241db680 2024-03-23T14:57:22.728 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.721+0000 7f0d217fa640 1 -- 172.21.15.164:0/928145677 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7f0d1c0147a0 con 0x7f0d241db680 2024-03-23T14:57:22.728 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.721+0000 7f0d217fa640 1 --2- 172.21.15.164:0/928145677 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f0d100672a0 0x7f0d10069760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:22.728 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.721+0000 7f0d217fa640 1 -- 172.21.15.164:0/928145677 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f0d1c0857e0 con 0x7f0d241db680 2024-03-23T14:57:22.728 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.721+0000 7f0d28d58640 1 --2- 172.21.15.164:0/928145677 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f0d100672a0 0x7f0d10069760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:22.729 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.721+0000 7f0d28d58640 1 --2- 172.21.15.164:0/928145677 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f0d100672a0 0x7f0d10069760 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0d180aa000 tx=0x7f0d18001a10 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:22.729 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.725+0000 7f0d217fa640 1 -- 172.21.15.164:0/928145677 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7f0d1c052380 con 0x7f0d241db680 2024-03-23T14:57:22.874 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.869+0000 7f0d2afbc640 1 -- 172.21.15.164:0/928145677 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0d240adee0 con 0x7f0d241db680 2024-03-23T14:57:22.877 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.873+0000 7f0d217fa640 1 -- 172.21.15.164:0/928145677 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16629 (secure 0 0 0) 0x7f0d240adee0 con 0x7f0d241db680 2024-03-23T14:57:22.877 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-23T14:57:22.878 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":20,"fsid":"45c7911b-b69d-4e68-907b-fbfeb780aa3b","created":"2024-03-23T14:56:34.367633+0000","modified":"2024-03-23T14:57:00.611265+0000","last_up_change":"2024-03-23T14:56:46.297403+0000","last_in_change":"2024-03-23T14:56:36.629884+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T14:56:47.186157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-23T14:56:56.994057+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-23T14:56:57.953862+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"80abdc62-bd0d-4e60-a554-7ed6fd67cf54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":984716690},{"type":"v1","addr":"172.21.15.164:6801","nonce":984716690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":984716690},{"type":"v1","addr":"172.21.15.164:6803","nonce":984716690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":984716690},{"type":"v1","addr":"172.21.15.164:6807","nonce":984716690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":984716690},{"type":"v1","addr":"172.21.15.164:6805","nonce":984716690}]},"public_addr":"172.21.15.164:6801/984716690","cluster_addr":"172.21.15.164:6803/984716690","heartbeat_back_addr":"172.21.15.164:6807/984716690","heartbeat_front_addr":"172.21.15.164:6805/984716690","state":["exists","up"]},{"osd":1,"uuid":"f9001b06-be4d-4f42-a0e6-bcef2afa0d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":1349657239},{"type":"v1","addr":"172.21.15.164:6817","nonce":1349657239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":1349657239},{"type":"v1","addr":"172.21.15.164:6819","nonce":1349657239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":1349657239},{"type":"v1","addr":"172.21.15.164:6823","nonce":1349657239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":1349657239},{"type":"v1","addr":"172.21.15.164:6821","nonce":1349657239}]},"public_addr":"172.21.15.164:6817/1349657239","cluster_addr":"172.21.15.164:6819/1349657239","heartbeat_back_addr":"172.21.15.164:6823/1349657239","heartbeat_front_addr":"172.21.15.164:6821/1349657239","state":["exists","up"]},{"osd":2,"uuid":"fa0aeb9f-8b54-4831-a188-3e361c7980d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":887861252},{"type":"v1","addr":"172.21.15.164:6809","nonce":887861252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":887861252},{"type":"v1","addr":"172.21.15.164:6811","nonce":887861252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":887861252},{"type":"v1","addr":"172.21.15.164:6815","nonce":887861252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":887861252},{"type":"v1","addr":"172.21.15.164:6813","nonce":887861252}]},"public_addr":"172.21.15.164:6809/887861252","cluster_addr":"172.21.15.164:6811/887861252","heartbeat_back_addr":"172.21.15.164:6815/887861252","heartbeat_front_addr":"172.21.15.164:6813/887861252","state":["exists","up"]},{"osd":3,"uuid":"7e18c957-3091-46d5-8e17-8f3354c1726c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":812540271},{"type":"v1","addr":"172.21.15.164:6825","nonce":812540271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":812540271},{"type":"v1","addr":"172.21.15.164:6827","nonce":812540271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":812540271},{"type":"v1","addr":"172.21.15.164:6831","nonce":812540271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":812540271},{"type":"v1","addr":"172.21.15.164:6829","nonce":812540271}]},"public_addr":"172.21.15.164:6825/812540271","cluster_addr":"172.21.15.164:6827/812540271","heartbeat_back_addr":"172.21.15.164:6831/812540271","heartbeat_front_addr":"172.21.15.164:6829/812540271","state":["exists","up"]},{"osd":4,"uuid":"8e83f315-42bf-4471-9ce8-d81948da0f9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":2656857421},{"type":"v1","addr":"172.21.15.173:6825","nonce":2656857421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2656857421},{"type":"v1","addr":"172.21.15.173:6827","nonce":2656857421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2656857421},{"type":"v1","addr":"172.21.15.173:6831","nonce":2656857421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":2656857421},{"type":"v1","addr":"172.21.15.173:6829","nonce":2656857421}]},"public_addr":"172.21.15.173:6825/2656857421","cluster_addr":"172.21.15.173:6827/2656857421","heartbeat_back_addr":"172.21.15.173:6831/2656857421","heartbeat_front_addr":"172.21.15.173:6829/2656857421","state":["exists","up"]},{"osd":5,"uuid":"a230c872-71f9-40ab-8999-defd0162d4ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":3676483185},{"type":"v1","addr":"172.21.15.173:6801","nonce":3676483185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":3676483185},{"type":"v1","addr":"172.21.15.173:6803","nonce":3676483185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":3676483185},{"type":"v1","addr":"172.21.15.173:6807","nonce":3676483185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":3676483185},{"type":"v1","addr":"172.21.15.173:6805","nonce":3676483185}]},"public_addr":"172.21.15.173:6801/3676483185","cluster_addr":"172.21.15.173:6803/3676483185","heartbeat_back_addr":"172.21.15.173:6807/3676483185","heartbeat_front_addr":"172.21.15.173:6805/3676483185","state":["exists","up"]},{"osd":6,"uuid":"6193bfd7-8432-4f7f-a117-a3fcee21af99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":3631684087},{"type":"v1","addr":"172.21.15.173:6809","nonce":3631684087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":3631684087},{"type":"v1","addr":"172.21.15.173:6811","nonce":3631684087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":3631684087},{"type":"v1","addr":"172.21.15.173:6815","nonce":3631684087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":3631684087},{"type":"v1","addr":"172.21.15.173:6813","nonce":3631684087}]},"public_addr":"172.21.15.173:6809/3631684087","cluster_addr":"172.21.15.173:6811/3631684087","heartbeat_back_addr":"172.21.15.173:6815/3631684087","heartbeat_front_addr":"172.21.15.173:6813/3631684087","state":["exists","up"]},{"osd":7,"uuid":"b0f1b80a-afc4-4538-83b9-f081d2dcb8cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":3261114948},{"type":"v1","addr":"172.21.15.173:6817","nonce":3261114948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":3261114948},{"type":"v1","addr":"172.21.15.173:6819","nonce":3261114948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":3261114948},{"type":"v1","addr":"172.21.15.173:6823","nonce":3261114948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":3261114948},{"type":"v1","addr":"172.21.15.173:6821","nonce":3261114948}]},"public_addr":"172.21.15.173:6817/3261114948","cluster_addr":"172.21.15.173:6819/3261114948","heartbeat_back_addr":"172.21.15.173:6823/3261114948","heartbeat_front_addr":"172.21.15.173:6821/3261114948","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.188390+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.159932+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.191101+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.181776+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.224365+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.177515+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.153631+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.139972+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T14:57:22.880 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.873+0000 7f0d2afbc640 1 -- 172.21.15.164:0/928145677 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f0d100672a0 msgr2=0x7f0d10069760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:22.880 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.873+0000 7f0d2afbc640 1 --2- 172.21.15.164:0/928145677 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f0d100672a0 0x7f0d10069760 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0d180aa000 tx=0x7f0d18001a10 comp rx=0 tx=0).stop 2024-03-23T14:57:22.880 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.873+0000 7f0d2afbc640 1 -- 172.21.15.164:0/928145677 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0d241db680 msgr2=0x7f0d2409d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:22.880 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.873+0000 7f0d2afbc640 1 --2- 172.21.15.164:0/928145677 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0d241db680 0x7f0d2409d130 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f0d240afea0 tx=0x7f0d1c00b480 comp rx=0 tx=0).stop 2024-03-23T14:57:22.880 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.873+0000 7f0d2afbc640 1 -- 172.21.15.164:0/928145677 shutdown_connections 2024-03-23T14:57:22.880 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.873+0000 7f0d2afbc640 1 --2- 172.21.15.164:0/928145677 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0d241db680 0x7f0d2409d130 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:22.880 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.873+0000 7f0d2afbc640 1 --2- 172.21.15.164:0/928145677 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f0d100672a0 0x7f0d10069760 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:22.880 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.873+0000 7f0d2afbc640 1 --2- 172.21.15.164:0/928145677 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0d2414b650 0x7f0d240ae5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:22.880 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.873+0000 7f0d2afbc640 1 --2- 172.21.15.164:0/928145677 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0d0c003980 0x7f0d2414af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:22.880 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.873+0000 7f0d2afbc640 1 -- 172.21.15.164:0/928145677 >> 172.21.15.164:0/928145677 conn(0x7f0d240b8ca0 msgr2=0x7f0d240b43f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:22.880 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.873+0000 7f0d2afbc640 1 -- 172.21.15.164:0/928145677 shutdown_connections 2024-03-23T14:57:22.881 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:22.873+0000 7f0d2afbc640 1 -- 172.21.15.164:0/928145677 wait complete. 2024-03-23T14:57:22.895 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T14:56:47.186157+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-23T14:56:56.994057+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-23T14:56:57.953862+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-23T14:57:22.895 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-23T14:57:23.016 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.009+0000 7fd592f28640 1 Processor -- start 2024-03-23T14:57:23.016 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.013+0000 7fd592f28640 1 -- start start 2024-03-23T14:57:23.019 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.013+0000 7fd592f28640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd58c09def0 0x7fd58c0ae620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:23.019 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.013+0000 7fd592f28640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd58c0b5270 con 0x7fd58c0b4e60 2024-03-23T14:57:23.019 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.013+0000 7fd592f28640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd58c09e2d0 con 0x7fd58c0b4a00 2024-03-23T14:57:23.019 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.013+0000 7fd592f28640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fd58c146de0 con 0x7fd58c09def0 2024-03-23T14:57:23.019 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.013+0000 7fd5914c5640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fd58c0b4e60 0x7fd58c14e110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:42434/0 (socket says 172.21.15.164:42434) 2024-03-23T14:57:23.019 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.013+0000 7fd5914c5640 1 -- 172.21.15.164:0/2527460837 learned_addr learned my addr 172.21.15.164:0/2527460837 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-03-23T14:57:23.019 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.013+0000 7fd590cc4640 1 --2- 172.21.15.164:0/2527460837 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd58c09def0 0x7fd58c0ae620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:23.019 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.013+0000 7fd58b7fe640 1 -- 172.21.15.164:0/2527460837 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 641283487 0 0) 0x7fd58c0b5270 con 0x7fd58c0b4e60 2024-03-23T14:57:23.019 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.013+0000 7fd58b7fe640 1 -- 172.21.15.164:0/2527460837 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd574003450 con 0x7fd58c0b4e60 2024-03-23T14:57:23.019 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.013+0000 7fd58b7fe640 1 -- 172.21.15.164:0/2527460837 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2473816981 0 0) 0x7fd574003450 con 0x7fd58c0b4e60 2024-03-23T14:57:23.020 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.013+0000 7fd58b7fe640 1 -- 172.21.15.164:0/2527460837 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd58c09def0 msgr2=0x7fd58c0ae620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:23.020 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.013+0000 7fd58b7fe640 1 --2- 172.21.15.164:0/2527460837 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd58c09def0 0x7fd58c0ae620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:23.020 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.013+0000 7fd58b7fe640 1 -- 172.21.15.164:0/2527460837 >> v1:172.21.15.173:6789/0 conn(0x7fd58c0b4a00 legacy=0x7fd58c147420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:23.020 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.013+0000 7fd58b7fe640 1 -- 172.21.15.164:0/2527460837 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd58c06cb70 con 0x7fd58c0b4e60 2024-03-23T14:57:23.020 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.013+0000 7fd58b7fe640 1 -- 172.21.15.164:0/2527460837 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2526390548 0 0) 0x7fd57c001680 con 0x7fd58c0b4e60 2024-03-23T14:57:23.020 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.013+0000 7fd58b7fe640 1 -- 172.21.15.164:0/2527460837 >> v1:172.21.15.164:6789/0 conn(0x7fd58c0b4e60 legacy=0x7fd58c14e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:23.020 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.013+0000 7fd58b7fe640 1 --2- 172.21.15.164:0/2527460837 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd574003aa0 0x7fd574003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:23.020 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.013+0000 7fd58b7fe640 1 --2- 172.21.15.164:0/2527460837 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd574008860 0x7fd57400ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:23.020 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.013+0000 7fd58b7fe640 1 --2- 172.21.15.164:0/2527460837 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd58c09def0 0x7fd57400d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:23.020 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.013+0000 7fd58b7fe640 1 -- 172.21.15.164:0/2527460837 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd57400dc00 con 0x7fd574008860 2024-03-23T14:57:23.020 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.013+0000 7fd58b7fe640 1 -- 172.21.15.164:0/2527460837 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd57400dd70 con 0x7fd574003aa0 2024-03-23T14:57:23.020 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.013+0000 7fd58b7fe640 1 -- 172.21.15.164:0/2527460837 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fd57400df30 con 0x7fd58c09def0 2024-03-23T14:57:23.020 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.013+0000 7fd58bfff640 1 --2- 172.21.15.164:0/2527460837 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd574008860 0x7fd57400ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:23.020 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.013+0000 7fd590cc4640 1 --2- 172.21.15.164:0/2527460837 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd574003aa0 0x7fd574003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:23.021 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.013+0000 7fd5914c5640 1 --2- 172.21.15.164:0/2527460837 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd58c09def0 0x7fd57400d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:23.021 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.013+0000 7fd58bfff640 1 -- 172.21.15.164:0/2527460837 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd58c09def0 msgr2=0x7fd57400d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:23.021 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.013+0000 7fd58bfff640 1 --2- 172.21.15.164:0/2527460837 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd58c09def0 0x7fd57400d4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:23.021 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.013+0000 7fd58bfff640 1 -- 172.21.15.164:0/2527460837 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd574003aa0 msgr2=0x7fd574003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:23.021 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.013+0000 7fd58bfff640 1 --2- 172.21.15.164:0/2527460837 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd574003aa0 0x7fd574003e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:23.021 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.013+0000 7fd58bfff640 1 -- 172.21.15.164:0/2527460837 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd574003450 con 0x7fd574008860 2024-03-23T14:57:23.021 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.013+0000 7fd590cc4640 1 --2- 172.21.15.164:0/2527460837 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd574003aa0 0x7fd574003e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T14:57:23.024 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.017+0000 7fd58bfff640 1 --2- 172.21.15.164:0/2527460837 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd574008860 0x7fd57400ac50 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fd580001f60 tx=0x7fd58001eff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7cd5b863735121b server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:23.024 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.017+0000 7fd58b7fe640 1 -- 172.21.15.164:0/2527460837 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5800a8c10 con 0x7fd574008860 2024-03-23T14:57:23.024 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.017+0000 7fd58b7fe640 1 -- 172.21.15.164:0/2527460837 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5800a8d70 con 0x7fd574008860 2024-03-23T14:57:23.024 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.017+0000 7fd58b7fe640 1 -- 172.21.15.164:0/2527460837 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5800b0680 con 0x7fd574008860 2024-03-23T14:57:23.024 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.017+0000 7fd592f28640 1 -- 172.21.15.164:0/2527460837 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd574008860 msgr2=0x7fd57400ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:23.024 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.017+0000 7fd592f28640 1 --2- 172.21.15.164:0/2527460837 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd574008860 0x7fd57400ac50 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fd580001f60 tx=0x7fd58001eff0 comp rx=0 tx=0).stop 2024-03-23T14:57:23.024 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.017+0000 7fd5914c5640 1 -- 172.21.15.164:0/2527460837 reap_dead start 2024-03-23T14:57:23.024 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.017+0000 7fd592f28640 1 -- 172.21.15.164:0/2527460837 shutdown_connections 2024-03-23T14:57:23.024 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.017+0000 7fd592f28640 1 -- 172.21.15.164:0/2527460837 >> 172.21.15.164:0/2527460837 conn(0x7fd58c0b8ca0 msgr2=0x7fd58c0b90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:23.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.017+0000 7fd592f28640 1 -- 172.21.15.164:0/2527460837 shutdown_connections 2024-03-23T14:57:23.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.017+0000 7fd592f28640 1 -- 172.21.15.164:0/2527460837 wait complete. 2024-03-23T14:57:23.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.017+0000 7fd592f28640 1 Processor -- start 2024-03-23T14:57:23.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.017+0000 7fd592f28640 1 -- start start 2024-03-23T14:57:23.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.017+0000 7fd592f28640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd574008860 0x7fd58c09d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:23.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.017+0000 7fd592f28640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd58c09c350 0x7fd58c143730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:23.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.017+0000 7fd592f28640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd58c143c70 0x7fd58c09b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:23.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.017+0000 7fd592f28640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd58c071fa0 con 0x7fd574008860 2024-03-23T14:57:23.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.017+0000 7fd592f28640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd58c1d83c0 con 0x7fd58c143c70 2024-03-23T14:57:23.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.017+0000 7fd592f28640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fd58c071c60 con 0x7fd58c09c350 2024-03-23T14:57:23.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.017+0000 7fd590cc4640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd574008860 0x7fd58c09d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:23.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.017+0000 7fd590cc4640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd574008860 0x7fd58c09d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:49080/0 (socket says 172.21.15.164:49080) 2024-03-23T14:57:23.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.017+0000 7fd590cc4640 1 -- 172.21.15.164:0/3265592786 learned_addr learned my addr 172.21.15.164:0/3265592786 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:23.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.017+0000 7fd58bfff640 1 --2- 172.21.15.164:0/3265592786 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd58c09c350 0x7fd58c143730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:23.025 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.017+0000 7fd5914c5640 1 --2- 172.21.15.164:0/3265592786 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd58c143c70 0x7fd58c09b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:23.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.017+0000 7fd590cc4640 1 -- 172.21.15.164:0/3265592786 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd58c09c350 msgr2=0x7fd58c143730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:23.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.017+0000 7fd590cc4640 1 --2- 172.21.15.164:0/3265592786 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd58c09c350 0x7fd58c143730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:23.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.017+0000 7fd590cc4640 1 -- 172.21.15.164:0/3265592786 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd58c143c70 msgr2=0x7fd58c09b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:23.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.017+0000 7fd590cc4640 1 --2- 172.21.15.164:0/3265592786 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd58c143c70 0x7fd58c09b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:23.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.017+0000 7fd590cc4640 1 -- 172.21.15.164:0/3265592786 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd58c0addc0 con 0x7fd574008860 2024-03-23T14:57:23.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.017+0000 7fd58bfff640 1 --2- 172.21.15.164:0/3265592786 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd58c09c350 0x7fd58c143730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T14:57:23.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.017+0000 7fd590cc4640 1 --2- 172.21.15.164:0/3265592786 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd574008860 0x7fd58c09d570 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fd5840116a0 tx=0x7fd584011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:23.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.017+0000 7fd5897fa640 1 -- 172.21.15.164:0/3265592786 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd58400f9c0 con 0x7fd574008860 2024-03-23T14:57:23.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.017+0000 7fd5897fa640 1 -- 172.21.15.164:0/3265592786 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd58400fb20 con 0x7fd574008860 2024-03-23T14:57:23.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.017+0000 7fd592f28640 1 -- 172.21.15.164:0/3265592786 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd58c070bc0 con 0x7fd574008860 2024-03-23T14:57:23.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.017+0000 7fd5897fa640 1 -- 172.21.15.164:0/3265592786 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd58400fed0 con 0x7fd574008860 2024-03-23T14:57:23.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.017+0000 7fd592f28640 1 -- 172.21.15.164:0/3265592786 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd58c09cdf0 con 0x7fd574008860 2024-03-23T14:57:23.026 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.021+0000 7fd592f28640 1 -- 172.21.15.164:0/3265592786 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd58c0690a0 con 0x7fd574008860 2024-03-23T14:57:23.027 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.021+0000 7fd5897fa640 1 -- 172.21.15.164:0/3265592786 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7fd584015460 con 0x7fd574008860 2024-03-23T14:57:23.027 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.021+0000 7fd5897fa640 1 --2- 172.21.15.164:0/3265592786 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fd5780672a0 0x7fd578069760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:23.027 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.021+0000 7fd58bfff640 1 --2- 172.21.15.164:0/3265592786 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fd5780672a0 0x7fd578069760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:23.027 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.021+0000 7fd5897fa640 1 -- 172.21.15.164:0/3265592786 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fd584085940 con 0x7fd574008860 2024-03-23T14:57:23.027 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.021+0000 7fd58bfff640 1 --2- 172.21.15.164:0/3265592786 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fd5780672a0 0x7fd578069760 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd5800b2000 tx=0x7fd5800aa070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:23.029 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.025+0000 7fd5897fa640 1 -- 172.21.15.164:0/3265592786 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7fd584052580 con 0x7fd574008860 2024-03-23T14:57:23.177 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.173+0000 7fd592f28640 1 -- 172.21.15.164:0/3265592786 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd58c0b8700 con 0x7fd574008860 2024-03-23T14:57:23.178 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.173+0000 7fd5897fa640 1 -- 172.21.15.164:0/3265592786 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7fd584055c40 con 0x7fd574008860 2024-03-23T14:57:23.178 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 1 2024-03-23T14:57:23.180 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.177+0000 7fd592f28640 1 -- 172.21.15.164:0/3265592786 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fd5780672a0 msgr2=0x7fd578069760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:23.180 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.177+0000 7fd592f28640 1 --2- 172.21.15.164:0/3265592786 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fd5780672a0 0x7fd578069760 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd5800b2000 tx=0x7fd5800aa070 comp rx=0 tx=0).stop 2024-03-23T14:57:23.180 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.177+0000 7fd592f28640 1 -- 172.21.15.164:0/3265592786 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd574008860 msgr2=0x7fd58c09d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:23.180 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.177+0000 7fd592f28640 1 --2- 172.21.15.164:0/3265592786 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd574008860 0x7fd58c09d570 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fd5840116a0 tx=0x7fd584011b60 comp rx=0 tx=0).stop 2024-03-23T14:57:23.180 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.177+0000 7fd592f28640 1 -- 172.21.15.164:0/3265592786 shutdown_connections 2024-03-23T14:57:23.181 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.177+0000 7fd592f28640 1 --2- 172.21.15.164:0/3265592786 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fd5780672a0 0x7fd578069760 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:23.181 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.177+0000 7fd592f28640 1 --2- 172.21.15.164:0/3265592786 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd58c143c70 0x7fd58c09b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:23.181 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.177+0000 7fd592f28640 1 --2- 172.21.15.164:0/3265592786 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd58c09c350 0x7fd58c143730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:23.181 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.177+0000 7fd592f28640 1 --2- 172.21.15.164:0/3265592786 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd574008860 0x7fd58c09d570 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:23.181 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.177+0000 7fd592f28640 1 -- 172.21.15.164:0/3265592786 >> 172.21.15.164:0/3265592786 conn(0x7fd58c0b8ca0 msgr2=0x7fd58c0b33d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:23.181 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.177+0000 7fd592f28640 1 -- 172.21.15.164:0/3265592786 shutdown_connections 2024-03-23T14:57:23.181 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.177+0000 7fd592f28640 1 -- 172.21.15.164:0/3265592786 wait complete. 2024-03-23T14:57:23.196 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-23T14:57:23.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.305+0000 7febdcbd1640 1 Processor -- start 2024-03-23T14:57:23.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.305+0000 7febdcbd1640 1 -- start start 2024-03-23T14:57:23.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.305+0000 7febdcbd1640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7febd80b9e80 0x7febd80b5590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:23.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.305+0000 7febdcbd1640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7febd80b9190 0x7febd814a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:23.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.305+0000 7febdcbd1640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7febd80b4280 con 0x7febd814aed0 2024-03-23T14:57:23.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.305+0000 7febdcbd1640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7febd80b48b0 con 0x7febd80b9e80 2024-03-23T14:57:23.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.305+0000 7febdcbd1640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 -- 0x7febd80b5f20 con 0x7febd80b9190 2024-03-23T14:57:23.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.305+0000 7febd659c640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7febd80b9e80 0x7febd80b5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:23.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.305+0000 7febd659c640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7febd80b9e80 0x7febd80b5590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:49084/0 (socket says 172.21.15.164:49084) 2024-03-23T14:57:23.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.305+0000 7febd659c640 1 -- 172.21.15.164:0/4036848661 learned_addr learned my addr 172.21.15.164:0/4036848661 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:23.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.305+0000 7febcffff640 1 --2- 172.21.15.164:0/4036848661 >> v2:172.21.15.173:3300/0 conn(0x7febd80b9190 0x7febd814a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:23.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.305+0000 7febd659c640 1 -- 172.21.15.164:0/4036848661 >> v2:172.21.15.173:3300/0 conn(0x7febd80b9190 msgr2=0x7febd814a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:23.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.305+0000 7febd659c640 1 --2- 172.21.15.164:0/4036848661 >> v2:172.21.15.173:3300/0 conn(0x7febd80b9190 0x7febd814a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:23.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.305+0000 7febd659c640 1 -- 172.21.15.164:0/4036848661 >> v1:172.21.15.173:6789/0 conn(0x7febd814aed0 legacy=0x7febd8140f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:23.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.305+0000 7febd659c640 1 -- 172.21.15.164:0/4036848661 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febd80b4cd0 con 0x7febd80b9e80 2024-03-23T14:57:23.312 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.309+0000 7febd659c640 1 --2- 172.21.15.164:0/4036848661 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7febd80b9e80 0x7febd80b5590 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7febd80ba4c0 tx=0x7febc801c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61234b55a3c23319 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:23.312 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.309+0000 7febd5d9b640 1 -- 172.21.15.164:0/4036848661 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc80ac620 con 0x7febd80b9e80 2024-03-23T14:57:23.312 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.309+0000 7febd5d9b640 1 -- 172.21.15.164:0/4036848661 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febc80ac780 con 0x7febd80b9e80 2024-03-23T14:57:23.312 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.309+0000 7febd5d9b640 1 -- 172.21.15.164:0/4036848661 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc80acb10 con 0x7febd80b9e80 2024-03-23T14:57:23.313 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.309+0000 7febdcbd1640 1 -- 172.21.15.164:0/4036848661 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7febd80b9e80 msgr2=0x7febd80b5590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:23.313 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.309+0000 7febdcbd1640 1 --2- 172.21.15.164:0/4036848661 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7febd80b9e80 0x7febd80b5590 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7febd80ba4c0 tx=0x7febc801c3c0 comp rx=0 tx=0).stop 2024-03-23T14:57:23.313 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.309+0000 7febdcbd1640 1 -- 172.21.15.164:0/4036848661 shutdown_connections 2024-03-23T14:57:23.313 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.309+0000 7febdcbd1640 1 --2- 172.21.15.164:0/4036848661 >> v2:172.21.15.173:3300/0 conn(0x7febd80b9190 0x7febd814a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:23.313 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.309+0000 7febdcbd1640 1 --2- 172.21.15.164:0/4036848661 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7febd80b9e80 0x7febd80b5590 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:23.313 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.309+0000 7febdcbd1640 1 -- 172.21.15.164:0/4036848661 >> 172.21.15.164:0/4036848661 conn(0x7febd802c3c0 msgr2=0x7febd802c7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:23.314 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.309+0000 7febdcbd1640 1 -- 172.21.15.164:0/4036848661 shutdown_connections 2024-03-23T14:57:23.314 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.309+0000 7febdcbd1640 1 -- 172.21.15.164:0/4036848661 wait complete. 2024-03-23T14:57:23.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.309+0000 7febdcbd1640 1 Processor -- start 2024-03-23T14:57:23.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.309+0000 7febdcbd1640 1 -- start start 2024-03-23T14:57:23.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.309+0000 7febdcbd1640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7febd80b9190 0x7febd8139f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:23.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.309+0000 7febdcbd1640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7febd80b9e80 0x7febd8149a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:23.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.309+0000 7febdcbd1640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7febd814aed0 0x7febd81e2d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:23.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.309+0000 7febdcbd1640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7febd81e3250 con 0x7febd80b9e80 2024-03-23T14:57:23.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.309+0000 7febdcbd1640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7febd81433c0 con 0x7febd814aed0 2024-03-23T14:57:23.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.309+0000 7febdcbd1640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7febd81424c0 con 0x7febd80b9190 2024-03-23T14:57:23.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.309+0000 7febcffff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7febd80b9e80 0x7febd8149a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:23.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.309+0000 7febcffff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7febd80b9e80 0x7febd8149a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:49098/0 (socket says 172.21.15.164:49098) 2024-03-23T14:57:23.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.309+0000 7febcffff640 1 -- 172.21.15.164:0/172291759 learned_addr learned my addr 172.21.15.164:0/172291759 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:23.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.309+0000 7febd659c640 1 --2- 172.21.15.164:0/172291759 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7febd80b9190 0x7febd8139f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:23.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.309+0000 7febd6d9d640 1 --2- 172.21.15.164:0/172291759 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7febd814aed0 0x7febd81e2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:23.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.309+0000 7febcffff640 1 -- 172.21.15.164:0/172291759 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7febd80b9190 msgr2=0x7febd8139f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:23.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.309+0000 7febcffff640 1 --2- 172.21.15.164:0/172291759 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7febd80b9190 0x7febd8139f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:23.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.309+0000 7febcffff640 1 -- 172.21.15.164:0/172291759 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7febd814aed0 msgr2=0x7febd81e2d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:23.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.309+0000 7febcffff640 1 --2- 172.21.15.164:0/172291759 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7febd814aed0 0x7febd81e2d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:23.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.309+0000 7febcffff640 1 -- 172.21.15.164:0/172291759 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febc80094d0 con 0x7febd80b9e80 2024-03-23T14:57:23.318 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.309+0000 7febcffff640 1 --2- 172.21.15.164:0/172291759 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7febd80b9e80 0x7febd8149a20 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7febc000d800 tx=0x7febc000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:23.318 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.309+0000 7febceffd640 1 -- 172.21.15.164:0/172291759 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc000b6c0 con 0x7febd80b9e80 2024-03-23T14:57:23.318 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.309+0000 7febceffd640 1 -- 172.21.15.164:0/172291759 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febc000b840 con 0x7febd80b9e80 2024-03-23T14:57:23.318 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.309+0000 7febdcbd1640 1 -- 172.21.15.164:0/172291759 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febd8139a30 con 0x7febd80b9e80 2024-03-23T14:57:23.318 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.309+0000 7febceffd640 1 -- 172.21.15.164:0/172291759 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc000bbf0 con 0x7febd80b9e80 2024-03-23T14:57:23.318 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.309+0000 7febdcbd1640 1 -- 172.21.15.164:0/172291759 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febd814a700 con 0x7febd80b9e80 2024-03-23T14:57:23.318 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.313+0000 7febdcbd1640 1 -- 172.21.15.164:0/172291759 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febd8142320 con 0x7febd80b9e80 2024-03-23T14:57:23.318 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.313+0000 7febceffd640 1 -- 172.21.15.164:0/172291759 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7febc000bd50 con 0x7febd80b9e80 2024-03-23T14:57:23.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.313+0000 7febceffd640 1 --2- 172.21.15.164:0/172291759 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7febb80672a0 0x7febb8069760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:23.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.313+0000 7febceffd640 1 -- 172.21.15.164:0/172291759 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7febc0085540 con 0x7febd80b9e80 2024-03-23T14:57:23.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.313+0000 7febd659c640 1 --2- 172.21.15.164:0/172291759 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7febb80672a0 0x7febb8069760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:23.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.313+0000 7febd659c640 1 --2- 172.21.15.164:0/172291759 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7febb80672a0 0x7febb8069760 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7febc80ae000 tx=0x7febc8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:23.321 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.317+0000 7febceffd640 1 -- 172.21.15.164:0/172291759 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7febd8142320 con 0x7febd80b9e80 2024-03-23T14:57:23.476 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.473+0000 7febdcbd1640 1 -- 172.21.15.164:0/172291759 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7febd8139c70 con 0x7febd80b9e80 2024-03-23T14:57:23.477 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.473+0000 7febceffd640 1 -- 172.21.15.164:0/172291759 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7febd8139c70 con 0x7febd80b9e80 2024-03-23T14:57:23.477 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 32 2024-03-23T14:57:23.479 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.473+0000 7febdcbd1640 1 -- 172.21.15.164:0/172291759 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7febb80672a0 msgr2=0x7febb8069760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:23.480 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.473+0000 7febdcbd1640 1 --2- 172.21.15.164:0/172291759 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7febb80672a0 0x7febb8069760 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7febc80ae000 tx=0x7febc8008040 comp rx=0 tx=0).stop 2024-03-23T14:57:23.480 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.473+0000 7febdcbd1640 1 -- 172.21.15.164:0/172291759 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7febd80b9e80 msgr2=0x7febd8149a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:23.480 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.473+0000 7febdcbd1640 1 --2- 172.21.15.164:0/172291759 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7febd80b9e80 0x7febd8149a20 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7febc000d800 tx=0x7febc000dcc0 comp rx=0 tx=0).stop 2024-03-23T14:57:23.480 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.473+0000 7febdcbd1640 1 -- 172.21.15.164:0/172291759 shutdown_connections 2024-03-23T14:57:23.480 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.473+0000 7febdcbd1640 1 --2- 172.21.15.164:0/172291759 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7febb80672a0 0x7febb8069760 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:23.480 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.473+0000 7febdcbd1640 1 --2- 172.21.15.164:0/172291759 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7febd814aed0 0x7febd81e2d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:23.480 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.473+0000 7febdcbd1640 1 --2- 172.21.15.164:0/172291759 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7febd80b9e80 0x7febd8149a20 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:23.480 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.473+0000 7febdcbd1640 1 --2- 172.21.15.164:0/172291759 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7febd80b9190 0x7febd8139f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:23.480 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.473+0000 7febdcbd1640 1 -- 172.21.15.164:0/172291759 >> 172.21.15.164:0/172291759 conn(0x7febd802c3c0 msgr2=0x7febd80bab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:23.480 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.473+0000 7febdcbd1640 1 -- 172.21.15.164:0/172291759 shutdown_connections 2024-03-23T14:57:23.480 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.473+0000 7febdcbd1640 1 -- 172.21.15.164:0/172291759 wait complete. 2024-03-23T14:57:23.495 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-23T14:57:23.608 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.605+0000 7f9c880ef640 1 Processor -- start 2024-03-23T14:57:23.608 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.605+0000 7f9c880ef640 1 -- start start 2024-03-23T14:57:23.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.605+0000 7f9c880ef640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f9c800adc90 0x7f9c800ae070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:23.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.605+0000 7f9c880ef640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9c800b4ba0 0x7f9c8006bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:23.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.605+0000 7f9c880ef640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c8006d510 con 0x7f9c800b34d0 2024-03-23T14:57:23.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.605+0000 7f9c880ef640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 -- 0x7f9c80072e70 con 0x7f9c800adc90 2024-03-23T14:57:23.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.605+0000 7f9c880ef640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f9c80072c60 con 0x7f9c800b4ba0 2024-03-23T14:57:23.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.605+0000 7f9c8668c640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f9c800b34d0 0x7f9c800b3920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:42440/0 (socket says 172.21.15.164:42440) 2024-03-23T14:57:23.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.605+0000 7f9c8668c640 1 -- 172.21.15.164:0/668578170 learned_addr learned my addr 172.21.15.164:0/668578170 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-03-23T14:57:23.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.605+0000 7f9c8568a640 1 --2- 172.21.15.164:0/668578170 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9c800b4ba0 0x7f9c8006bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:23.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.605+0000 7f9c85e8b640 1 --2- 172.21.15.164:0/668578170 >> v2:172.21.15.173:3300/0 conn(0x7f9c800adc90 0x7f9c800ae070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:23.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.605+0000 7f9c84e89640 1 -- 172.21.15.164:0/668578170 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1994160869 0 0) 0x7f9c8006d510 con 0x7f9c800b34d0 2024-03-23T14:57:23.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.605+0000 7f9c84e89640 1 -- 172.21.15.164:0/668578170 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9c6c003530 con 0x7f9c800b34d0 2024-03-23T14:57:23.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.605+0000 7f9c84e89640 1 -- 172.21.15.164:0/668578170 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 153701747 0 0) 0x7f9c6c003530 con 0x7f9c800b34d0 2024-03-23T14:57:23.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.605+0000 7f9c84e89640 1 -- 172.21.15.164:0/668578170 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9c800b4ba0 msgr2=0x7f9c8006bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:23.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.605+0000 7f9c84e89640 1 --2- 172.21.15.164:0/668578170 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9c800b4ba0 0x7f9c8006bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:23.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.605+0000 7f9c84e89640 1 -- 172.21.15.164:0/668578170 >> v2:172.21.15.173:3300/0 conn(0x7f9c800adc90 msgr2=0x7f9c800ae070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:23.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.605+0000 7f9c84e89640 1 --2- 172.21.15.164:0/668578170 >> v2:172.21.15.173:3300/0 conn(0x7f9c800adc90 0x7f9c800ae070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:23.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.605+0000 7f9c84e89640 1 -- 172.21.15.164:0/668578170 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c800af180 con 0x7f9c800b34d0 2024-03-23T14:57:23.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.605+0000 7f9c84e89640 1 -- 172.21.15.164:0/668578170 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2526390548 0 0) 0x7f9c78001a00 con 0x7f9c800b34d0 2024-03-23T14:57:23.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.605+0000 7f9c84e89640 1 -- 172.21.15.164:0/668578170 >> v1:172.21.15.164:6789/0 conn(0x7f9c800b34d0 legacy=0x7f9c800b3920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:23.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.605+0000 7f9c84e89640 1 --2- 172.21.15.164:0/668578170 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9c6c003980 0x7f9c6c003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:23.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.605+0000 7f9c84e89640 1 --2- 172.21.15.164:0/668578170 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9c6c0047d0 0x7f9c6c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:23.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.605+0000 7f9c84e89640 1 --2- 172.21.15.164:0/668578170 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9c800b4ba0 0x7f9c6c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:23.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.605+0000 7f9c84e89640 1 -- 172.21.15.164:0/668578170 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9c80072e70 con 0x7f9c6c0047d0 2024-03-23T14:57:23.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.605+0000 7f9c84e89640 1 -- 172.21.15.164:0/668578170 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f9c80072c60 con 0x7f9c6c003980 2024-03-23T14:57:23.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.605+0000 7f9c84e89640 1 -- 172.21.15.164:0/668578170 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f9c6c00b990 con 0x7f9c800b4ba0 2024-03-23T14:57:23.613 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.605+0000 7f9c8568a640 1 --2- 172.21.15.164:0/668578170 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9c6c0047d0 0x7f9c6c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:23.613 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.605+0000 7f9c85e8b640 1 --2- 172.21.15.164:0/668578170 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9c6c003980 0x7f9c6c003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:23.613 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.605+0000 7f9c8568a640 1 -- 172.21.15.164:0/668578170 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9c800b4ba0 msgr2=0x7f9c6c005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:23.613 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.605+0000 7f9c8668c640 1 --2- 172.21.15.164:0/668578170 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9c800b4ba0 0x7f9c6c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:23.613 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.605+0000 7f9c8568a640 1 --2- 172.21.15.164:0/668578170 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9c800b4ba0 0x7f9c6c005190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:23.613 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.605+0000 7f9c8568a640 1 -- 172.21.15.164:0/668578170 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9c6c003980 msgr2=0x7f9c6c003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:23.613 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.605+0000 7f9c8568a640 1 --2- 172.21.15.164:0/668578170 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9c6c003980 0x7f9c6c003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:23.613 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.605+0000 7f9c8568a640 1 -- 172.21.15.164:0/668578170 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c6c003530 con 0x7f9c6c0047d0 2024-03-23T14:57:23.616 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.609+0000 7f9c8568a640 1 --2- 172.21.15.164:0/668578170 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9c6c0047d0 0x7f9c6c004c50 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c009b10 tx=0x7f9c7c01ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b74474d901979ae7 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:23.616 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.609+0000 7f9c84e89640 1 -- 172.21.15.164:0/668578170 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c7c0a6450 con 0x7f9c6c0047d0 2024-03-23T14:57:23.616 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.609+0000 7f9c84e89640 1 -- 172.21.15.164:0/668578170 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c7c0a65b0 con 0x7f9c6c0047d0 2024-03-23T14:57:23.616 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.609+0000 7f9c84e89640 1 -- 172.21.15.164:0/668578170 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c7c0a6960 con 0x7f9c6c0047d0 2024-03-23T14:57:23.616 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.609+0000 7f9c880ef640 1 -- 172.21.15.164:0/668578170 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9c6c0047d0 msgr2=0x7f9c6c004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:23.616 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.609+0000 7f9c880ef640 1 --2- 172.21.15.164:0/668578170 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9c6c0047d0 0x7f9c6c004c50 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c009b10 tx=0x7f9c7c01ca40 comp rx=0 tx=0).stop 2024-03-23T14:57:23.616 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.609+0000 7f9c8668c640 1 -- 172.21.15.164:0/668578170 reap_dead start 2024-03-23T14:57:23.617 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.609+0000 7f9c880ef640 1 -- 172.21.15.164:0/668578170 shutdown_connections 2024-03-23T14:57:23.617 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.609+0000 7f9c880ef640 1 -- 172.21.15.164:0/668578170 >> 172.21.15.164:0/668578170 conn(0x7f9c800b8ca0 msgr2=0x7f9c800b90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:23.617 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.609+0000 7f9c880ef640 1 -- 172.21.15.164:0/668578170 shutdown_connections 2024-03-23T14:57:23.617 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.609+0000 7f9c880ef640 1 -- 172.21.15.164:0/668578170 wait complete. 2024-03-23T14:57:23.617 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.609+0000 7f9c880ef640 1 Processor -- start 2024-03-23T14:57:23.617 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.609+0000 7f9c880ef640 1 -- start start 2024-03-23T14:57:23.617 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.609+0000 7f9c880ef640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9c6c0047d0 0x7f9c80154cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:23.617 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.609+0000 7f9c880ef640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9c8009ce30 0x7f9c8009d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:23.617 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.609+0000 7f9c880ef640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9c801e4b50 0x7f9c801e4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:23.617 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.609+0000 7f9c880ef640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9c80154890 con 0x7f9c8009ce30 2024-03-23T14:57:23.617 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.609+0000 7f9c880ef640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f9c801e5510 con 0x7f9c801e4b50 2024-03-23T14:57:23.617 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.609+0000 7f9c880ef640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f9c80154540 con 0x7f9c6c0047d0 2024-03-23T14:57:23.617 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.609+0000 7f9c8568a640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9c8009ce30 0x7f9c8009d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:23.617 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.609+0000 7f9c8568a640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9c8009ce30 0x7f9c8009d290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:49122/0 (socket says 172.21.15.164:49122) 2024-03-23T14:57:23.617 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.609+0000 7f9c8568a640 1 -- 172.21.15.164:0/94929644 learned_addr learned my addr 172.21.15.164:0/94929644 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:23.618 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.609+0000 7f9c8668c640 1 --2- 172.21.15.164:0/94929644 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9c801e4b50 0x7f9c801e4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:23.618 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.609+0000 7f9c85e8b640 1 --2- 172.21.15.164:0/94929644 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9c6c0047d0 0x7f9c80154cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:23.618 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.609+0000 7f9c8568a640 1 -- 172.21.15.164:0/94929644 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9c6c0047d0 msgr2=0x7f9c80154cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:23.618 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.609+0000 7f9c8568a640 1 --2- 172.21.15.164:0/94929644 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9c6c0047d0 0x7f9c80154cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:23.618 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.609+0000 7f9c8568a640 1 -- 172.21.15.164:0/94929644 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9c801e4b50 msgr2=0x7f9c801e4fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:23.618 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.609+0000 7f9c8568a640 1 --2- 172.21.15.164:0/94929644 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9c801e4b50 0x7f9c801e4fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:23.618 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.609+0000 7f9c8568a640 1 -- 172.21.15.164:0/94929644 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c801539f0 con 0x7f9c8009ce30 2024-03-23T14:57:23.618 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.609+0000 7f9c8668c640 1 --2- 172.21.15.164:0/94929644 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9c801e4b50 0x7f9c801e4fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T14:57:23.618 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.609+0000 7f9c8568a640 1 --2- 172.21.15.164:0/94929644 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9c8009ce30 0x7f9c8009d290 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c009b10 tx=0x7f9c6c005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:23.618 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.609+0000 7f9c767fc640 1 -- 172.21.15.164:0/94929644 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c7c0a6d20 con 0x7f9c8009ce30 2024-03-23T14:57:23.618 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.609+0000 7f9c767fc640 1 -- 172.21.15.164:0/94929644 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c7c0a6e80 con 0x7f9c8009ce30 2024-03-23T14:57:23.618 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.609+0000 7f9c880ef640 1 -- 172.21.15.164:0/94929644 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c8009b480 con 0x7f9c8009ce30 2024-03-23T14:57:23.618 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.609+0000 7f9c767fc640 1 -- 172.21.15.164:0/94929644 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c7c004260 con 0x7f9c8009ce30 2024-03-23T14:57:23.618 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.609+0000 7f9c880ef640 1 -- 172.21.15.164:0/94929644 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c801554c0 con 0x7f9c8009ce30 2024-03-23T14:57:23.619 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.613+0000 7f9c880ef640 1 -- 172.21.15.164:0/94929644 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c8009b670 con 0x7f9c8009ce30 2024-03-23T14:57:23.619 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.613+0000 7f9c767fc640 1 -- 172.21.15.164:0/94929644 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7f9c7c0043c0 con 0x7f9c8009ce30 2024-03-23T14:57:23.619 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.613+0000 7f9c767fc640 1 --2- 172.21.15.164:0/94929644 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f9c640672d0 0x7f9c64069790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:23.620 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.613+0000 7f9c767fc640 1 -- 172.21.15.164:0/94929644 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f9c7c11ef00 con 0x7f9c8009ce30 2024-03-23T14:57:23.620 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.613+0000 7f9c85e8b640 1 --2- 172.21.15.164:0/94929644 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f9c640672d0 0x7f9c64069790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:23.620 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.613+0000 7f9c85e8b640 1 --2- 172.21.15.164:0/94929644 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f9c640672d0 0x7f9c64069790 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f9c700099f0 tx=0x7f9c70008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:23.621 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.617+0000 7f9c767fc640 1 -- 172.21.15.164:0/94929644 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7f9c7c0ebb40 con 0x7f9c8009ce30 2024-03-23T14:57:23.772 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.769+0000 7f9c880ef640 1 -- 172.21.15.164:0/94929644 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f9c80154b00 con 0x7f9c8009ce30 2024-03-23T14:57:23.772 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.769+0000 7f9c767fc640 1 -- 172.21.15.164:0/94929644 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f9c7c0ef200 con 0x7f9c8009ce30 2024-03-23T14:57:23.773 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 64 2024-03-23T14:57:23.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.769+0000 7f9c880ef640 1 -- 172.21.15.164:0/94929644 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f9c640672d0 msgr2=0x7f9c64069790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:23.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.769+0000 7f9c880ef640 1 --2- 172.21.15.164:0/94929644 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f9c640672d0 0x7f9c64069790 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f9c700099f0 tx=0x7f9c70008040 comp rx=0 tx=0).stop 2024-03-23T14:57:23.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.769+0000 7f9c880ef640 1 -- 172.21.15.164:0/94929644 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9c8009ce30 msgr2=0x7f9c8009d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:23.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.769+0000 7f9c880ef640 1 --2- 172.21.15.164:0/94929644 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9c8009ce30 0x7f9c8009d290 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c009b10 tx=0x7f9c6c005800 comp rx=0 tx=0).stop 2024-03-23T14:57:23.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.769+0000 7f9c880ef640 1 -- 172.21.15.164:0/94929644 shutdown_connections 2024-03-23T14:57:23.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.769+0000 7f9c880ef640 1 --2- 172.21.15.164:0/94929644 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f9c640672d0 0x7f9c64069790 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:23.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.769+0000 7f9c880ef640 1 --2- 172.21.15.164:0/94929644 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9c801e4b50 0x7f9c801e4fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:23.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.769+0000 7f9c880ef640 1 --2- 172.21.15.164:0/94929644 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9c8009ce30 0x7f9c8009d290 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:23.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.769+0000 7f9c880ef640 1 --2- 172.21.15.164:0/94929644 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9c6c0047d0 0x7f9c80154cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:23.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.769+0000 7f9c880ef640 1 -- 172.21.15.164:0/94929644 >> 172.21.15.164:0/94929644 conn(0x7f9c800b8ca0 msgr2=0x7f9c800b5cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:23.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.769+0000 7f9c880ef640 1 -- 172.21.15.164:0/94929644 shutdown_connections 2024-03-23T14:57:23.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.769+0000 7f9c880ef640 1 -- 172.21.15.164:0/94929644 wait complete. 2024-03-23T14:57:23.790 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_cephfs_shell.TestCD.test_cd_with_args' 2024-03-23T14:57:23.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.893+0000 7fbaa4aca640 1 Processor -- start 2024-03-23T14:57:23.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.893+0000 7fbaa4aca640 1 -- start start 2024-03-23T14:57:23.898 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.893+0000 7fbaa4aca640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fbaa00b6740 0x7fbaa00b41d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:23.898 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.893+0000 7fbaa4aca640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbaa00b4710 0x7fbaa014fb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:23.898 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.893+0000 7fbaa4aca640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbaa00b5530 con 0x7fbaa00b6360 2024-03-23T14:57:23.898 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.893+0000 7fbaa4aca640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fbaa00b5a60 con 0x7fbaa00b6740 2024-03-23T14:57:23.898 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.893+0000 7fbaa4aca640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fbaa0071260 con 0x7fbaa00b4710 2024-03-23T14:57:23.898 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.893+0000 7fba9dd9b640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fbaa00b6740 0x7fbaa00b41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:23.898 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.893+0000 7fba9dd9b640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fbaa00b6740 0x7fbaa00b41d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:49138/0 (socket says 172.21.15.164:49138) 2024-03-23T14:57:23.898 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.893+0000 7fba9dd9b640 1 -- 172.21.15.164:0/2712806090 learned_addr learned my addr 172.21.15.164:0/2712806090 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:23.898 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.893+0000 7fba9dd9b640 1 -- 172.21.15.164:0/2712806090 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbaa00b4710 msgr2=0x7fbaa014fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:23.898 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.893+0000 7fba9dd9b640 1 --2- 172.21.15.164:0/2712806090 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbaa00b4710 0x7fbaa014fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:23.898 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.893+0000 7fba9dd9b640 1 -- 172.21.15.164:0/2712806090 >> v1:172.21.15.173:6789/0 conn(0x7fbaa00b6360 legacy=0x7fbaa0145a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:23.898 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.893+0000 7fba9dd9b640 1 -- 172.21.15.164:0/2712806090 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbaa00b2a70 con 0x7fbaa00b6740 2024-03-23T14:57:23.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.897+0000 7fba9dd9b640 1 --2- 172.21.15.164:0/2712806090 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbaa00b6740 0x7fbaa00b41d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fba94009870 tx=0x7fba9401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a7a7726125fa244 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:23.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.897+0000 7fba9d59a640 1 -- 172.21.15.164:0/2712806090 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba940ab600 con 0x7fbaa00b6740 2024-03-23T14:57:23.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.897+0000 7fba9d59a640 1 -- 172.21.15.164:0/2712806090 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba940a7050 con 0x7fbaa00b6740 2024-03-23T14:57:23.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.897+0000 7fba9d59a640 1 -- 172.21.15.164:0/2712806090 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba940ab980 con 0x7fbaa00b6740 2024-03-23T14:57:23.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.897+0000 7fbaa4aca640 1 -- 172.21.15.164:0/2712806090 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbaa00b6740 msgr2=0x7fbaa00b41d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:23.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.897+0000 7fbaa4aca640 1 --2- 172.21.15.164:0/2712806090 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbaa00b6740 0x7fbaa00b41d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fba94009870 tx=0x7fba9401c490 comp rx=0 tx=0).stop 2024-03-23T14:57:23.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.897+0000 7fbaa4aca640 1 -- 172.21.15.164:0/2712806090 shutdown_connections 2024-03-23T14:57:23.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.897+0000 7fbaa4aca640 1 --2- 172.21.15.164:0/2712806090 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbaa00b4710 0x7fbaa014fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:23.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.897+0000 7fbaa4aca640 1 --2- 172.21.15.164:0/2712806090 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbaa00b6740 0x7fbaa00b41d0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:23.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.897+0000 7fbaa4aca640 1 -- 172.21.15.164:0/2712806090 >> 172.21.15.164:0/2712806090 conn(0x7fbaa00b7630 msgr2=0x7fbaa00b7a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:23.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.897+0000 7fbaa4aca640 1 -- 172.21.15.164:0/2712806090 shutdown_connections 2024-03-23T14:57:23.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.897+0000 7fbaa4aca640 1 -- 172.21.15.164:0/2712806090 wait complete. 2024-03-23T14:57:23.903 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.897+0000 7fbaa4aca640 1 Processor -- start 2024-03-23T14:57:23.903 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.897+0000 7fbaa4aca640 1 -- start start 2024-03-23T14:57:23.903 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.897+0000 7fbaa4aca640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbaa00b4710 0x7fbaa01e35a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:23.903 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.897+0000 7fbaa4aca640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbaa00b6360 0x7fbaa006b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:23.903 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.897+0000 7fbaa4aca640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbaa00b6740 0x7fbaa006b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:23.903 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.897+0000 7fbaa4aca640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fbaa01e40b0 con 0x7fbaa00b4710 2024-03-23T14:57:23.904 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.897+0000 7fbaa4aca640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fbaa01e3ae0 con 0x7fbaa00b6740 2024-03-23T14:57:23.904 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.897+0000 7fbaa4aca640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fbaa00afac0 con 0x7fbaa00b6360 2024-03-23T14:57:23.904 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.897+0000 7fba9e59c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbaa00b4710 0x7fbaa01e35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:23.904 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.897+0000 7fba9e59c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbaa00b4710 0x7fbaa01e35a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:49152/0 (socket says 172.21.15.164:49152) 2024-03-23T14:57:23.904 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.897+0000 7fba9e59c640 1 -- 172.21.15.164:0/2905585052 learned_addr learned my addr 172.21.15.164:0/2905585052 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:23.904 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.897+0000 7fba9dd9b640 1 --2- 172.21.15.164:0/2905585052 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbaa00b6360 0x7fbaa006b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:23.904 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.897+0000 7fba9ed9d640 1 --2- 172.21.15.164:0/2905585052 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbaa00b6740 0x7fbaa006b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:23.904 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.897+0000 7fba9e59c640 1 -- 172.21.15.164:0/2905585052 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbaa00b6360 msgr2=0x7fbaa006b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:23.904 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.897+0000 7fba9e59c640 1 --2- 172.21.15.164:0/2905585052 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbaa00b6360 0x7fbaa006b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:23.904 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.897+0000 7fba9e59c640 1 -- 172.21.15.164:0/2905585052 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbaa00b6740 msgr2=0x7fbaa006b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:23.904 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.897+0000 7fba9e59c640 1 --2- 172.21.15.164:0/2905585052 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbaa00b6740 0x7fbaa006b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:23.904 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.897+0000 7fba9e59c640 1 -- 172.21.15.164:0/2905585052 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba940094d0 con 0x7fbaa00b4710 2024-03-23T14:57:23.904 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.897+0000 7fba9e59c640 1 --2- 172.21.15.164:0/2905585052 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbaa00b4710 0x7fbaa01e35a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fba9000c480 tx=0x7fba9000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:23.904 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.897+0000 7fba8effd640 1 -- 172.21.15.164:0/2905585052 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba90012830 con 0x7fbaa00b4710 2024-03-23T14:57:23.904 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.897+0000 7fba8effd640 1 -- 172.21.15.164:0/2905585052 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba900129b0 con 0x7fbaa00b4710 2024-03-23T14:57:23.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.897+0000 7fba8effd640 1 -- 172.21.15.164:0/2905585052 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba90012d60 con 0x7fbaa00b4710 2024-03-23T14:57:23.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.897+0000 7fbaa4aca640 1 -- 172.21.15.164:0/2905585052 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbaa006cbc0 con 0x7fbaa00b4710 2024-03-23T14:57:23.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.897+0000 7fbaa4aca640 1 -- 172.21.15.164:0/2905585052 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbaa00b08c0 con 0x7fbaa00b4710 2024-03-23T14:57:23.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.901+0000 7fbaa4aca640 1 -- 172.21.15.164:0/2905585052 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbaa00b5fd0 con 0x7fbaa00b4710 2024-03-23T14:57:23.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.901+0000 7fba8effd640 1 -- 172.21.15.164:0/2905585052 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7fba9001c460 con 0x7fbaa00b4710 2024-03-23T14:57:23.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.901+0000 7fba8effd640 1 --2- 172.21.15.164:0/2905585052 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fba800672a0 0x7fba80069760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:23.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.901+0000 7fba9dd9b640 1 --2- 172.21.15.164:0/2905585052 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fba800672a0 0x7fba80069760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:23.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.901+0000 7fba8effd640 1 -- 172.21.15.164:0/2905585052 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fba90084e10 con 0x7fbaa00b4710 2024-03-23T14:57:23.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.901+0000 7fba9dd9b640 1 --2- 172.21.15.164:0/2905585052 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fba800672a0 0x7fba80069760 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fba940ad000 tx=0x7fba940a6000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:23.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:23.905+0000 7fba8effd640 1 -- 172.21.15.164:0/2905585052 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7fba90051a50 con 0x7fbaa00b4710 2024-03-23T14:57:24.108 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.105+0000 7fbaa4aca640 1 -- 172.21.15.164:0/2905585052 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_cephfs_shell.TestCD.test_cd_with_args"]} v 0) -- 0x7fbaa00690a0 con 0x7fbaa00b4710 2024-03-23T14:57:24.240 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.237+0000 7fba8effd640 1 -- 172.21.15.164:0/2905585052 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_cephfs_shell.TestCD.test_cd_with_args"]}]=0 v48) ==== 137+0+0 (secure 0 0 0) 0x7fba90055110 con 0x7fbaa00b4710 2024-03-23T14:57:24.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.241+0000 7fbaa4aca640 1 -- 172.21.15.164:0/2905585052 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fba800672a0 msgr2=0x7fba80069760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:24.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.241+0000 7fbaa4aca640 1 --2- 172.21.15.164:0/2905585052 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fba800672a0 0x7fba80069760 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fba940ad000 tx=0x7fba940a6000 comp rx=0 tx=0).stop 2024-03-23T14:57:24.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.241+0000 7fbaa4aca640 1 -- 172.21.15.164:0/2905585052 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbaa00b4710 msgr2=0x7fbaa01e35a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:24.245 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.241+0000 7fbaa4aca640 1 --2- 172.21.15.164:0/2905585052 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbaa00b4710 0x7fbaa01e35a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fba9000c480 tx=0x7fba9000c940 comp rx=0 tx=0).stop 2024-03-23T14:57:24.245 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.241+0000 7fbaa4aca640 1 -- 172.21.15.164:0/2905585052 shutdown_connections 2024-03-23T14:57:24.245 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.241+0000 7fbaa4aca640 1 --2- 172.21.15.164:0/2905585052 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fba800672a0 0x7fba80069760 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:24.245 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.241+0000 7fbaa4aca640 1 --2- 172.21.15.164:0/2905585052 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbaa00b6740 0x7fbaa006b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:24.245 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.241+0000 7fbaa4aca640 1 --2- 172.21.15.164:0/2905585052 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbaa00b6360 0x7fbaa006b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:24.245 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.241+0000 7fbaa4aca640 1 --2- 172.21.15.164:0/2905585052 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbaa00b4710 0x7fbaa01e35a0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:24.245 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.241+0000 7fbaa4aca640 1 -- 172.21.15.164:0/2905585052 >> 172.21.15.164:0/2905585052 conn(0x7fbaa00b7630 msgr2=0x7fbaa00713b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:24.245 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.241+0000 7fbaa4aca640 1 -- 172.21.15.164:0/2905585052 shutdown_connections 2024-03-23T14:57:24.245 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.241+0000 7fbaa4aca640 1 -- 172.21.15.164:0/2905585052 wait complete. 2024-03-23T14:57:24.263 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-03-23T14:57:24.388 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.385+0000 7f94896ad640 1 Processor -- start 2024-03-23T14:57:24.388 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.385+0000 7f94896ad640 1 -- start start 2024-03-23T14:57:24.389 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.385+0000 7f94896ad640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f94840b5ab0 0x7f9484141180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:24.389 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.385+0000 7f94896ad640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f94841416c0 0x7f9484151ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:24.389 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.385+0000 7f94896ad640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9484147b10 con 0x7f94840b9e60 2024-03-23T14:57:24.389 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.385+0000 7f94896ad640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f94840b6070 con 0x7f94840b5ab0 2024-03-23T14:57:24.390 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.385+0000 7f94896ad640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f94840b3e20 con 0x7f94841416c0 2024-03-23T14:57:24.390 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.385+0000 7f947bfff640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f94840b5ab0 0x7f9484141180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:24.390 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.385+0000 7f947bfff640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f94840b5ab0 0x7f9484141180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:49158/0 (socket says 172.21.15.164:49158) 2024-03-23T14:57:24.390 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.385+0000 7f947bfff640 1 -- 172.21.15.164:0/3750647196 learned_addr learned my addr 172.21.15.164:0/3750647196 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:24.390 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.385+0000 7f94837fe640 1 --2- 172.21.15.164:0/3750647196 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f94841416c0 0x7f9484151ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:24.390 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.385+0000 7f947bfff640 1 -- 172.21.15.164:0/3750647196 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f94841416c0 msgr2=0x7f9484151ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:24.390 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.385+0000 7f947bfff640 1 --2- 172.21.15.164:0/3750647196 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f94841416c0 0x7f9484151ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:24.390 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.385+0000 7f947bfff640 1 -- 172.21.15.164:0/3750647196 >> v1:172.21.15.173:6789/0 conn(0x7f94840b9e60 legacy=0x7f9484147400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:24.390 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.385+0000 7f947bfff640 1 -- 172.21.15.164:0/3750647196 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94840b3b80 con 0x7f94840b5ab0 2024-03-23T14:57:24.391 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.385+0000 7f947bfff640 1 --2- 172.21.15.164:0/3750647196 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f94840b5ab0 0x7f9484141180 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f946c009870 tx=0x7f946c01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d368d29a40155981 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:24.391 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.385+0000 7f94827fc640 1 -- 172.21.15.164:0/3750647196 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f946c0ab650 con 0x7f94840b5ab0 2024-03-23T14:57:24.391 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.385+0000 7f94827fc640 1 -- 172.21.15.164:0/3750647196 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f946c0ab7b0 con 0x7f94840b5ab0 2024-03-23T14:57:24.391 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.385+0000 7f94827fc640 1 -- 172.21.15.164:0/3750647196 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f946c0abb40 con 0x7f94840b5ab0 2024-03-23T14:57:24.392 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.389+0000 7f94896ad640 1 -- 172.21.15.164:0/3750647196 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f94840b5ab0 msgr2=0x7f9484141180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:24.392 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.389+0000 7f94896ad640 1 --2- 172.21.15.164:0/3750647196 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f94840b5ab0 0x7f9484141180 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f946c009870 tx=0x7f946c01c490 comp rx=0 tx=0).stop 2024-03-23T14:57:24.392 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.389+0000 7f94896ad640 1 -- 172.21.15.164:0/3750647196 shutdown_connections 2024-03-23T14:57:24.392 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.389+0000 7f94896ad640 1 --2- 172.21.15.164:0/3750647196 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f94841416c0 0x7f9484151ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:24.392 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.389+0000 7f94896ad640 1 --2- 172.21.15.164:0/3750647196 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f94840b5ab0 0x7f9484141180 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:24.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.389+0000 7f94896ad640 1 -- 172.21.15.164:0/3750647196 >> 172.21.15.164:0/3750647196 conn(0x7f9484072cd0 msgr2=0x7f94840730e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:24.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.389+0000 7f94896ad640 1 -- 172.21.15.164:0/3750647196 shutdown_connections 2024-03-23T14:57:24.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.389+0000 7f94896ad640 1 -- 172.21.15.164:0/3750647196 wait complete. 2024-03-23T14:57:24.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.389+0000 7f94896ad640 1 Processor -- start 2024-03-23T14:57:24.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.389+0000 7f94896ad640 1 -- start start 2024-03-23T14:57:24.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.389+0000 7f94896ad640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f94840b5ab0 0x7f94841490a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:24.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.389+0000 7f94896ad640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f94840b9e60 0x7f94841e03f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:24.396 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.389+0000 7f94896ad640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f94841416c0 0x7f9484148560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:24.396 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.389+0000 7f94896ad640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f94841dff60 con 0x7f94841416c0 2024-03-23T14:57:24.396 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.389+0000 7f94896ad640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f948414adf0 con 0x7f94840b5ab0 2024-03-23T14:57:24.396 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.389+0000 7f94896ad640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f948413aa80 con 0x7f94840b9e60 2024-03-23T14:57:24.396 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.389+0000 7f94837fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f94841416c0 0x7f9484148560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:24.396 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.389+0000 7f94837fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f94841416c0 0x7f9484148560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:49160/0 (socket says 172.21.15.164:49160) 2024-03-23T14:57:24.396 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.389+0000 7f94837fe640 1 -- 172.21.15.164:0/3140195117 learned_addr learned my addr 172.21.15.164:0/3140195117 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:24.396 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.389+0000 7f947bfff640 1 --2- 172.21.15.164:0/3140195117 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f94840b9e60 0x7f94841e03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:24.396 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.389+0000 7f9482ffd640 1 --2- 172.21.15.164:0/3140195117 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f94840b5ab0 0x7f94841490a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:24.396 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.389+0000 7f94837fe640 1 -- 172.21.15.164:0/3140195117 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f94840b9e60 msgr2=0x7f94841e03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:24.396 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.389+0000 7f94837fe640 1 --2- 172.21.15.164:0/3140195117 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f94840b9e60 0x7f94841e03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:24.396 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.389+0000 7f94837fe640 1 -- 172.21.15.164:0/3140195117 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f94840b5ab0 msgr2=0x7f94841490a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:24.396 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.389+0000 7f94837fe640 1 --2- 172.21.15.164:0/3140195117 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f94840b5ab0 0x7f94841490a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:24.396 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.389+0000 7f94837fe640 1 -- 172.21.15.164:0/3140195117 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f946c0094d0 con 0x7f94841416c0 2024-03-23T14:57:24.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.389+0000 7f947bfff640 1 --2- 172.21.15.164:0/3140195117 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f94840b9e60 0x7f94841e03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T14:57:24.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.389+0000 7f94837fe640 1 --2- 172.21.15.164:0/3140195117 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f94841416c0 0x7f9484148560 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f947c007e90 tx=0x7f947c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:24.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.389+0000 7f947b7fe640 1 -- 172.21.15.164:0/3140195117 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f947c014420 con 0x7f94841416c0 2024-03-23T14:57:24.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.389+0000 7f947b7fe640 1 -- 172.21.15.164:0/3140195117 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f947c014580 con 0x7f94841416c0 2024-03-23T14:57:24.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.389+0000 7f94896ad640 1 -- 172.21.15.164:0/3140195117 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9484139b10 con 0x7f94841416c0 2024-03-23T14:57:24.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.389+0000 7f947b7fe640 1 -- 172.21.15.164:0/3140195117 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f947c014930 con 0x7f94841416c0 2024-03-23T14:57:24.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.389+0000 7f94896ad640 1 -- 172.21.15.164:0/3140195117 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f948414a910 con 0x7f94841416c0 2024-03-23T14:57:24.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.389+0000 7f94896ad640 1 -- 172.21.15.164:0/3140195117 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94841e0110 con 0x7f94841416c0 2024-03-23T14:57:24.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.393+0000 7f947b7fe640 1 -- 172.21.15.164:0/3140195117 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7f947c014a90 con 0x7f94841416c0 2024-03-23T14:57:24.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.393+0000 7f947b7fe640 1 --2- 172.21.15.164:0/3140195117 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f94640672a0 0x7f9464069760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:24.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.393+0000 7f947b7fe640 1 -- 172.21.15.164:0/3140195117 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f947c085880 con 0x7f94841416c0 2024-03-23T14:57:24.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.393+0000 7f9482ffd640 1 --2- 172.21.15.164:0/3140195117 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f94640672a0 0x7f9464069760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:24.398 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.393+0000 7f9482ffd640 1 --2- 172.21.15.164:0/3140195117 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f94640672a0 0x7f9464069760 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9474003d60 tx=0x7f9474009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:24.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.397+0000 7f947b7fe640 1 -- 172.21.15.164:0/3140195117 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7f94841e0110 con 0x7f94841416c0 2024-03-23T14:57:24.558 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.553+0000 7f94896ad640 1 -- 172.21.15.164:0/3140195117 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f94841e0110 con 0x7f94841416c0 2024-03-23T14:57:24.563 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.557+0000 7f947b7fe640 1 -- 172.21.15.164:0/3140195117 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) ==== 122+0+0 (secure 0 0 0) 0x7f94841e0110 con 0x7f94841416c0 2024-03-23T14:57:24.566 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.561+0000 7f94896ad640 1 -- 172.21.15.164:0/3140195117 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f94640672a0 msgr2=0x7f9464069760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:24.566 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.561+0000 7f94896ad640 1 --2- 172.21.15.164:0/3140195117 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f94640672a0 0x7f9464069760 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9474003d60 tx=0x7f9474009040 comp rx=0 tx=0).stop 2024-03-23T14:57:24.566 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.561+0000 7f94896ad640 1 -- 172.21.15.164:0/3140195117 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f94841416c0 msgr2=0x7f9484148560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:24.566 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.561+0000 7f94896ad640 1 --2- 172.21.15.164:0/3140195117 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f94841416c0 0x7f9484148560 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f947c007e90 tx=0x7f947c00e4b0 comp rx=0 tx=0).stop 2024-03-23T14:57:24.566 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.561+0000 7f94896ad640 1 -- 172.21.15.164:0/3140195117 shutdown_connections 2024-03-23T14:57:24.566 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.561+0000 7f94896ad640 1 --2- 172.21.15.164:0/3140195117 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f94841416c0 0x7f9484148560 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:24.566 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.561+0000 7f94896ad640 1 --2- 172.21.15.164:0/3140195117 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f94840b9e60 0x7f94841e03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:24.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.561+0000 7f94896ad640 1 --2- 172.21.15.164:0/3140195117 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f94640672a0 0x7f9464069760 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:24.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.561+0000 7f94896ad640 1 --2- 172.21.15.164:0/3140195117 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f94840b5ab0 0x7f94841490a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:24.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.561+0000 7f94896ad640 1 -- 172.21.15.164:0/3140195117 >> 172.21.15.164:0/3140195117 conn(0x7f9484072cd0 msgr2=0x7f94840bcbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:24.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.561+0000 7f94896ad640 1 -- 172.21.15.164:0/3140195117 shutdown_connections 2024-03-23T14:57:24.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.561+0000 7f94896ad640 1 -- 172.21.15.164:0/3140195117 wait complete. 2024-03-23T14:57:24.581 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-23T14:57:24.582 DEBUG:teuthology.orchestra.run.smithi164:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-23T14:57:24.621 DEBUG:teuthology.orchestra.run.smithi173:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-23T14:57:24.655 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-23T14:57:24.656 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-23T14:57:24.656 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T14:57:24.672 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-23T14:57:24.673 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T14:57:24.720 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T14:57:24.880 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.873+0000 7f9eafc7b640 1 Processor -- start 2024-03-23T14:57:24.880 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.877+0000 7f9eafc7b640 1 -- start start 2024-03-23T14:57:24.880 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.877+0000 7f9eafc7b640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f9ea80b4a70 0x7f9ea8140f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:24.880 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.877+0000 7f9eafc7b640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ea80b4300 con 0x7f9ea80b56f0 2024-03-23T14:57:24.880 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.877+0000 7f9eafc7b640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ea80b45e0 con 0x7f9ea8141470 2024-03-23T14:57:24.881 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.877+0000 7f9eafc7b640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 -- 0x7f9ea80b5fa0 con 0x7f9ea80b4a70 2024-03-23T14:57:24.881 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.877+0000 7f9eada17640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f9ea80b56f0 0x7f9ea814a8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:42450/0 (socket says 172.21.15.164:42450) 2024-03-23T14:57:24.881 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.877+0000 7f9eada17640 1 -- 172.21.15.164:0/1276375339 learned_addr learned my addr 172.21.15.164:0/1276375339 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-03-23T14:57:24.883 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.877+0000 7f9eaca15640 1 -- 172.21.15.164:0/1276375339 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2737545395 0 0) 0x7f9ea80b4300 con 0x7f9ea80b56f0 2024-03-23T14:57:24.883 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.877+0000 7f9eaca15640 1 -- 172.21.15.164:0/1276375339 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9e94003530 con 0x7f9ea80b56f0 2024-03-23T14:57:24.883 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.877+0000 7f9ead216640 1 --2- 172.21.15.164:0/1276375339 >> v2:172.21.15.173:3300/0 conn(0x7f9ea80b4a70 0x7f9ea8140f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:24.883 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.877+0000 7f9eaca15640 1 -- 172.21.15.164:0/1276375339 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 665165749 0 0) 0x7f9e94003530 con 0x7f9ea80b56f0 2024-03-23T14:57:24.883 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.877+0000 7f9eaca15640 1 -- 172.21.15.164:0/1276375339 >> v2:172.21.15.173:3300/0 conn(0x7f9ea80b4a70 msgr2=0x7f9ea8140f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:24.883 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.877+0000 7f9eaca15640 1 --2- 172.21.15.164:0/1276375339 >> v2:172.21.15.173:3300/0 conn(0x7f9ea80b4a70 0x7f9ea8140f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:24.883 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.877+0000 7f9eaca15640 1 -- 172.21.15.164:0/1276375339 >> v1:172.21.15.173:6789/0 conn(0x7f9ea8141470 legacy=0x7f9ea8151c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:24.883 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.877+0000 7f9eaca15640 1 -- 172.21.15.164:0/1276375339 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ea80bd7e0 con 0x7f9ea80b56f0 2024-03-23T14:57:24.883 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.877+0000 7f9eaca15640 1 -- 172.21.15.164:0/1276375339 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2526390548 0 0) 0x7f9e98001680 con 0x7f9ea80b56f0 2024-03-23T14:57:24.883 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.877+0000 7f9eaca15640 1 -- 172.21.15.164:0/1276375339 >> v1:172.21.15.164:6789/0 conn(0x7f9ea80b56f0 legacy=0x7f9ea814a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:24.884 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.877+0000 7f9eaca15640 1 --2- 172.21.15.164:0/1276375339 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9e94003980 0x7f9e94003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:24.884 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.877+0000 7f9eaca15640 1 --2- 172.21.15.164:0/1276375339 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9e940047d0 0x7f9e94004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:24.884 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.877+0000 7f9eaca15640 1 --2- 172.21.15.164:0/1276375339 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9e94005190 0x7f9e9400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:24.884 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.877+0000 7f9eaca15640 1 -- 172.21.15.164:0/1276375339 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9e9400c0b0 con 0x7f9e94003980 2024-03-23T14:57:24.884 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.877+0000 7f9eaca15640 1 -- 172.21.15.164:0/1276375339 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f9e9400c220 con 0x7f9e940047d0 2024-03-23T14:57:24.884 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.877+0000 7f9eaca15640 1 -- 172.21.15.164:0/1276375339 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f9e9400c3e0 con 0x7f9e94005190 2024-03-23T14:57:24.884 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.877+0000 7f9ead216640 1 --2- 172.21.15.164:0/1276375339 >> v2:172.21.15.173:3300/0 conn(0x7f9ea80b4a70 0x7f9ea8140f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T14:57:24.884 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.877+0000 7f9eada17640 1 --2- 172.21.15.164:0/1276375339 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9e94003980 0x7f9e94003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:24.884 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.877+0000 7f9eae218640 1 --2- 172.21.15.164:0/1276375339 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9e94005190 0x7f9e9400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:24.884 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.877+0000 7f9eada17640 1 -- 172.21.15.164:0/1276375339 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9e94005190 msgr2=0x7f9e9400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:24.884 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.877+0000 7f9eada17640 1 --2- 172.21.15.164:0/1276375339 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9e94005190 0x7f9e9400b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:24.884 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.877+0000 7f9eada17640 1 -- 172.21.15.164:0/1276375339 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9e940047d0 msgr2=0x7f9e94004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:24.884 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.877+0000 7f9eada17640 1 --2- 172.21.15.164:0/1276375339 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9e940047d0 0x7f9e94004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:24.884 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.877+0000 7f9eada17640 1 -- 172.21.15.164:0/1276375339 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e94003530 con 0x7f9e94003980 2024-03-23T14:57:24.885 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.877+0000 7f9eae218640 1 -- 172.21.15.164:0/1276375339 reap_dead start 2024-03-23T14:57:24.887 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.881+0000 7f9eada17640 1 --2- 172.21.15.164:0/1276375339 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9e94003980 0x7f9e94003da0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f9e9800c240 tx=0x7f9e9801cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=908fc852dd14c0d2 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:24.888 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.881+0000 7f9eaca15640 1 -- 172.21.15.164:0/1276375339 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e98005ae0 con 0x7f9e94003980 2024-03-23T14:57:24.888 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.881+0000 7f9eaca15640 1 -- 172.21.15.164:0/1276375339 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e98005c40 con 0x7f9e94003980 2024-03-23T14:57:24.888 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.881+0000 7f9eaca15640 1 -- 172.21.15.164:0/1276375339 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e980a5a60 con 0x7f9e94003980 2024-03-23T14:57:24.888 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.881+0000 7f9eafc7b640 1 -- 172.21.15.164:0/1276375339 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9e94003980 msgr2=0x7f9e94003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:24.888 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.881+0000 7f9eafc7b640 1 --2- 172.21.15.164:0/1276375339 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9e94003980 0x7f9e94003da0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f9e9800c240 tx=0x7f9e9801cc30 comp rx=0 tx=0).stop 2024-03-23T14:57:24.888 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.881+0000 7f9eafc7b640 1 -- 172.21.15.164:0/1276375339 shutdown_connections 2024-03-23T14:57:24.888 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.881+0000 7f9eafc7b640 1 --2- 172.21.15.164:0/1276375339 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9e94003980 0x7f9e94003da0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:24.888 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.881+0000 7f9eafc7b640 1 -- 172.21.15.164:0/1276375339 >> 172.21.15.164:0/1276375339 conn(0x7f9ea802c3c0 msgr2=0x7f9ea802c7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:24.888 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.881+0000 7f9eafc7b640 1 -- 172.21.15.164:0/1276375339 shutdown_connections 2024-03-23T14:57:24.888 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.881+0000 7f9eafc7b640 1 -- 172.21.15.164:0/1276375339 wait complete. 2024-03-23T14:57:24.888 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.881+0000 7f9eafc7b640 1 Processor -- start 2024-03-23T14:57:24.888 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.881+0000 7f9eafc7b640 1 -- start start 2024-03-23T14:57:24.888 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.881+0000 7f9eafc7b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9e94003980 0x7f9ea8144510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:24.889 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.881+0000 7f9eafc7b640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9ea8146cf0 0x7f9ea8145c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:24.889 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.881+0000 7f9eafc7b640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9ea8146150 0x7f9ea81e32e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:24.889 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.881+0000 7f9eafc7b640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9ea8148250 con 0x7f9e94003980 2024-03-23T14:57:24.889 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.881+0000 7f9eafc7b640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f9ea8147d00 con 0x7f9ea8146cf0 2024-03-23T14:57:24.889 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.881+0000 7f9eafc7b640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f9ea81465d0 con 0x7f9ea8146150 2024-03-23T14:57:24.889 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.881+0000 7f9eada17640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9e94003980 0x7f9ea8144510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:24.889 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.881+0000 7f9eada17640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9e94003980 0x7f9ea8144510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:49182/0 (socket says 172.21.15.164:49182) 2024-03-23T14:57:24.889 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.881+0000 7f9eada17640 1 -- 172.21.15.164:0/2507229666 learned_addr learned my addr 172.21.15.164:0/2507229666 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:24.889 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.881+0000 7f9ead216640 1 --2- 172.21.15.164:0/2507229666 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9ea8146cf0 0x7f9ea8145c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:24.889 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.881+0000 7f9eae218640 1 --2- 172.21.15.164:0/2507229666 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9ea8146150 0x7f9ea81e32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:24.889 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.881+0000 7f9eada17640 1 -- 172.21.15.164:0/2507229666 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9ea8146150 msgr2=0x7f9ea81e32e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:24.889 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.881+0000 7f9eada17640 1 --2- 172.21.15.164:0/2507229666 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9ea8146150 0x7f9ea81e32e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:24.889 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.881+0000 7f9eada17640 1 -- 172.21.15.164:0/2507229666 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9ea8146cf0 msgr2=0x7f9ea8145c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:24.889 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.881+0000 7f9eada17640 1 --2- 172.21.15.164:0/2507229666 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9ea8146cf0 0x7f9ea8145c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:24.889 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.881+0000 7f9eada17640 1 -- 172.21.15.164:0/2507229666 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ea8147520 con 0x7f9e94003980 2024-03-23T14:57:24.890 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.881+0000 7f9eada17640 1 --2- 172.21.15.164:0/2507229666 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9e94003980 0x7f9ea8144510 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f9e98098540 tx=0x7f9e94005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:24.890 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.881+0000 7f9e9e7fc640 1 -- 172.21.15.164:0/2507229666 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e980b4070 con 0x7f9e94003980 2024-03-23T14:57:24.890 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.881+0000 7f9e9e7fc640 1 -- 172.21.15.164:0/2507229666 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e98001580 con 0x7f9e94003980 2024-03-23T14:57:24.890 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.881+0000 7f9eafc7b640 1 -- 172.21.15.164:0/2507229666 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ea81436f0 con 0x7f9e94003980 2024-03-23T14:57:24.890 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.881+0000 7f9e9e7fc640 1 -- 172.21.15.164:0/2507229666 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e980a5a40 con 0x7f9e94003980 2024-03-23T14:57:24.890 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.881+0000 7f9eafc7b640 1 -- 172.21.15.164:0/2507229666 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ea8147170 con 0x7f9e94003980 2024-03-23T14:57:24.890 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.885+0000 7f9eafc7b640 1 -- 172.21.15.164:0/2507229666 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ea8145550 con 0x7f9e94003980 2024-03-23T14:57:24.891 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.885+0000 7f9e9e7fc640 1 -- 172.21.15.164:0/2507229666 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7f9e980b7070 con 0x7f9e94003980 2024-03-23T14:57:24.891 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.885+0000 7f9e9e7fc640 1 --2- 172.21.15.164:0/2507229666 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f9e8c0672d0 0x7f9e8c069790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:24.891 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.885+0000 7f9e9e7fc640 1 -- 172.21.15.164:0/2507229666 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f9e9811e920 con 0x7f9e94003980 2024-03-23T14:57:24.891 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.885+0000 7f9ead216640 1 --2- 172.21.15.164:0/2507229666 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f9e8c0672d0 0x7f9e8c069790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:24.891 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.885+0000 7f9ead216640 1 --2- 172.21.15.164:0/2507229666 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f9e8c0672d0 0x7f9e8c069790 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f9ea4006aa0 tx=0x7f9ea4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:24.892 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:24.889+0000 7f9e9e7fc640 1 -- 172.21.15.164:0/2507229666 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7f9e980eb640 con 0x7f9e94003980 2024-03-23T14:57:25.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.041+0000 7f9eafc7b640 1 -- 172.21.15.164:0/2507229666 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9ea8145a60 con 0x7f9e94003980 2024-03-23T14:57:25.049 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.045+0000 7f9e9e7fc640 1 -- 172.21.15.164:0/2507229666 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16629 (secure 0 0 0) 0x7f9e980be090 con 0x7f9e94003980 2024-03-23T14:57:25.050 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-23T14:57:25.050 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":20,"fsid":"45c7911b-b69d-4e68-907b-fbfeb780aa3b","created":"2024-03-23T14:56:34.367633+0000","modified":"2024-03-23T14:57:00.611265+0000","last_up_change":"2024-03-23T14:56:46.297403+0000","last_in_change":"2024-03-23T14:56:36.629884+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T14:56:47.186157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-23T14:56:56.994057+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-23T14:56:57.953862+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"80abdc62-bd0d-4e60-a554-7ed6fd67cf54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":984716690},{"type":"v1","addr":"172.21.15.164:6801","nonce":984716690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":984716690},{"type":"v1","addr":"172.21.15.164:6803","nonce":984716690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":984716690},{"type":"v1","addr":"172.21.15.164:6807","nonce":984716690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":984716690},{"type":"v1","addr":"172.21.15.164:6805","nonce":984716690}]},"public_addr":"172.21.15.164:6801/984716690","cluster_addr":"172.21.15.164:6803/984716690","heartbeat_back_addr":"172.21.15.164:6807/984716690","heartbeat_front_addr":"172.21.15.164:6805/984716690","state":["exists","up"]},{"osd":1,"uuid":"f9001b06-be4d-4f42-a0e6-bcef2afa0d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":1349657239},{"type":"v1","addr":"172.21.15.164:6817","nonce":1349657239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":1349657239},{"type":"v1","addr":"172.21.15.164:6819","nonce":1349657239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":1349657239},{"type":"v1","addr":"172.21.15.164:6823","nonce":1349657239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":1349657239},{"type":"v1","addr":"172.21.15.164:6821","nonce":1349657239}]},"public_addr":"172.21.15.164:6817/1349657239","cluster_addr":"172.21.15.164:6819/1349657239","heartbeat_back_addr":"172.21.15.164:6823/1349657239","heartbeat_front_addr":"172.21.15.164:6821/1349657239","state":["exists","up"]},{"osd":2,"uuid":"fa0aeb9f-8b54-4831-a188-3e361c7980d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":887861252},{"type":"v1","addr":"172.21.15.164:6809","nonce":887861252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":887861252},{"type":"v1","addr":"172.21.15.164:6811","nonce":887861252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":887861252},{"type":"v1","addr":"172.21.15.164:6815","nonce":887861252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":887861252},{"type":"v1","addr":"172.21.15.164:6813","nonce":887861252}]},"public_addr":"172.21.15.164:6809/887861252","cluster_addr":"172.21.15.164:6811/887861252","heartbeat_back_addr":"172.21.15.164:6815/887861252","heartbeat_front_addr":"172.21.15.164:6813/887861252","state":["exists","up"]},{"osd":3,"uuid":"7e18c957-3091-46d5-8e17-8f3354c1726c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":812540271},{"type":"v1","addr":"172.21.15.164:6825","nonce":812540271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":812540271},{"type":"v1","addr":"172.21.15.164:6827","nonce":812540271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":812540271},{"type":"v1","addr":"172.21.15.164:6831","nonce":812540271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":812540271},{"type":"v1","addr":"172.21.15.164:6829","nonce":812540271}]},"public_addr":"172.21.15.164:6825/812540271","cluster_addr":"172.21.15.164:6827/812540271","heartbeat_back_addr":"172.21.15.164:6831/812540271","heartbeat_front_addr":"172.21.15.164:6829/812540271","state":["exists","up"]},{"osd":4,"uuid":"8e83f315-42bf-4471-9ce8-d81948da0f9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":2656857421},{"type":"v1","addr":"172.21.15.173:6825","nonce":2656857421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2656857421},{"type":"v1","addr":"172.21.15.173:6827","nonce":2656857421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2656857421},{"type":"v1","addr":"172.21.15.173:6831","nonce":2656857421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":2656857421},{"type":"v1","addr":"172.21.15.173:6829","nonce":2656857421}]},"public_addr":"172.21.15.173:6825/2656857421","cluster_addr":"172.21.15.173:6827/2656857421","heartbeat_back_addr":"172.21.15.173:6831/2656857421","heartbeat_front_addr":"172.21.15.173:6829/2656857421","state":["exists","up"]},{"osd":5,"uuid":"a230c872-71f9-40ab-8999-defd0162d4ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":3676483185},{"type":"v1","addr":"172.21.15.173:6801","nonce":3676483185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":3676483185},{"type":"v1","addr":"172.21.15.173:6803","nonce":3676483185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":3676483185},{"type":"v1","addr":"172.21.15.173:6807","nonce":3676483185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":3676483185},{"type":"v1","addr":"172.21.15.173:6805","nonce":3676483185}]},"public_addr":"172.21.15.173:6801/3676483185","cluster_addr":"172.21.15.173:6803/3676483185","heartbeat_back_addr":"172.21.15.173:6807/3676483185","heartbeat_front_addr":"172.21.15.173:6805/3676483185","state":["exists","up"]},{"osd":6,"uuid":"6193bfd7-8432-4f7f-a117-a3fcee21af99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":3631684087},{"type":"v1","addr":"172.21.15.173:6809","nonce":3631684087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":3631684087},{"type":"v1","addr":"172.21.15.173:6811","nonce":3631684087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":3631684087},{"type":"v1","addr":"172.21.15.173:6815","nonce":3631684087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":3631684087},{"type":"v1","addr":"172.21.15.173:6813","nonce":3631684087}]},"public_addr":"172.21.15.173:6809/3631684087","cluster_addr":"172.21.15.173:6811/3631684087","heartbeat_back_addr":"172.21.15.173:6815/3631684087","heartbeat_front_addr":"172.21.15.173:6813/3631684087","state":["exists","up"]},{"osd":7,"uuid":"b0f1b80a-afc4-4538-83b9-f081d2dcb8cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":3261114948},{"type":"v1","addr":"172.21.15.173:6817","nonce":3261114948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":3261114948},{"type":"v1","addr":"172.21.15.173:6819","nonce":3261114948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":3261114948},{"type":"v1","addr":"172.21.15.173:6823","nonce":3261114948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":3261114948},{"type":"v1","addr":"172.21.15.173:6821","nonce":3261114948}]},"public_addr":"172.21.15.173:6817/3261114948","cluster_addr":"172.21.15.173:6819/3261114948","heartbeat_back_addr":"172.21.15.173:6823/3261114948","heartbeat_front_addr":"172.21.15.173:6821/3261114948","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.188390+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.159932+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.191101+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.181776+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.224365+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.177515+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.153631+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.139972+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T14:57:25.052 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.045+0000 7f9eafc7b640 1 -- 172.21.15.164:0/2507229666 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f9e8c0672d0 msgr2=0x7f9e8c069790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:25.052 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.045+0000 7f9eafc7b640 1 --2- 172.21.15.164:0/2507229666 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f9e8c0672d0 0x7f9e8c069790 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f9ea4006aa0 tx=0x7f9ea4008040 comp rx=0 tx=0).stop 2024-03-23T14:57:25.052 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.045+0000 7f9eafc7b640 1 -- 172.21.15.164:0/2507229666 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9e94003980 msgr2=0x7f9ea8144510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:25.052 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.045+0000 7f9eafc7b640 1 --2- 172.21.15.164:0/2507229666 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9e94003980 0x7f9ea8144510 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f9e98098540 tx=0x7f9e94005800 comp rx=0 tx=0).stop 2024-03-23T14:57:25.052 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.045+0000 7f9eafc7b640 1 -- 172.21.15.164:0/2507229666 shutdown_connections 2024-03-23T14:57:25.052 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.045+0000 7f9eafc7b640 1 --2- 172.21.15.164:0/2507229666 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9ea8146150 0x7f9ea81e32e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:25.052 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.045+0000 7f9eafc7b640 1 --2- 172.21.15.164:0/2507229666 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f9e8c0672d0 0x7f9e8c069790 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:25.052 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.045+0000 7f9eafc7b640 1 --2- 172.21.15.164:0/2507229666 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9ea8146cf0 0x7f9ea8145c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:25.052 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.045+0000 7f9eafc7b640 1 --2- 172.21.15.164:0/2507229666 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9e94003980 0x7f9ea8144510 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:25.052 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.045+0000 7f9eafc7b640 1 -- 172.21.15.164:0/2507229666 >> 172.21.15.164:0/2507229666 conn(0x7f9ea802c3c0 msgr2=0x7f9ea814afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:25.052 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.045+0000 7f9eafc7b640 1 -- 172.21.15.164:0/2507229666 shutdown_connections 2024-03-23T14:57:25.052 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.045+0000 7f9eafc7b640 1 -- 172.21.15.164:0/2507229666 wait complete. 2024-03-23T14:57:25.067 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T14:56:47.186157+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-23T14:56:56.994057+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-23T14:56:57.953862+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-23T14:57:25.067 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-23T14:57:25.179 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.173+0000 7feac891d640 1 Processor -- start 2024-03-23T14:57:25.179 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.173+0000 7feac891d640 1 -- start start 2024-03-23T14:57:25.182 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.177+0000 7feac891d640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7feac406f8e0 0x7feac41459b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:25.182 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.177+0000 7feac891d640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feac4145240 con 0x7feac40b9ac0 2024-03-23T14:57:25.182 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.177+0000 7feac891d640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feac4070050 con 0x7feac4145ef0 2024-03-23T14:57:25.182 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.177+0000 7feac891d640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 -- 0x7feac40b3b50 con 0x7feac406f8e0 2024-03-23T14:57:25.182 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.177+0000 7feac259c640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7feac40b9ac0 0x7feac413f240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:42458/0 (socket says 172.21.15.164:42458) 2024-03-23T14:57:25.182 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.177+0000 7feac259c640 1 -- 172.21.15.164:0/3886993758 learned_addr learned my addr 172.21.15.164:0/3886993758 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-03-23T14:57:25.182 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.177+0000 7feab9d9b640 1 --2- 172.21.15.164:0/3886993758 >> v2:172.21.15.173:3300/0 conn(0x7feac406f8e0 0x7feac41459b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:25.182 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.177+0000 7feac1d9b640 1 -- 172.21.15.164:0/3886993758 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2766993407 0 0) 0x7feac4145240 con 0x7feac40b9ac0 2024-03-23T14:57:25.183 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.177+0000 7feac1d9b640 1 -- 172.21.15.164:0/3886993758 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feab0003530 con 0x7feac40b9ac0 2024-03-23T14:57:25.183 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.177+0000 7feac1d9b640 1 -- 172.21.15.164:0/3886993758 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3798524171 0 0) 0x7feab0003530 con 0x7feac40b9ac0 2024-03-23T14:57:25.183 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.177+0000 7feac1d9b640 1 -- 172.21.15.164:0/3886993758 >> v2:172.21.15.173:3300/0 conn(0x7feac406f8e0 msgr2=0x7feac41459b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:25.183 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.177+0000 7feac1d9b640 1 --2- 172.21.15.164:0/3886993758 >> v2:172.21.15.173:3300/0 conn(0x7feac406f8e0 0x7feac41459b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:25.183 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.177+0000 7feac1d9b640 1 -- 172.21.15.164:0/3886993758 >> v1:172.21.15.173:6789/0 conn(0x7feac4145ef0 legacy=0x7feac414fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:25.183 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.177+0000 7feac1d9b640 1 -- 172.21.15.164:0/3886993758 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feac413f950 con 0x7feac40b9ac0 2024-03-23T14:57:25.183 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.177+0000 7feac1d9b640 1 -- 172.21.15.164:0/3886993758 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2526390548 0 0) 0x7feab4001680 con 0x7feac40b9ac0 2024-03-23T14:57:25.183 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.177+0000 7feac1d9b640 1 -- 172.21.15.164:0/3886993758 >> v1:172.21.15.164:6789/0 conn(0x7feac40b9ac0 legacy=0x7feac413f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:25.183 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.177+0000 7feac1d9b640 1 --2- 172.21.15.164:0/3886993758 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feab00039d0 0x7feab0003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:25.183 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.177+0000 7feac1d9b640 1 --2- 172.21.15.164:0/3886993758 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7feab00047d0 0x7feab0004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:25.183 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.177+0000 7feac1d9b640 1 --2- 172.21.15.164:0/3886993758 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feab00051e0 0x7feab000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:25.183 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.177+0000 7feac1d9b640 1 -- 172.21.15.164:0/3886993758 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7feab000c100 con 0x7feab00051e0 2024-03-23T14:57:25.183 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.177+0000 7feac1d9b640 1 -- 172.21.15.164:0/3886993758 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7feab000c270 con 0x7feab00039d0 2024-03-23T14:57:25.183 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.177+0000 7feac1d9b640 1 -- 172.21.15.164:0/3886993758 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7feab000c430 con 0x7feab00047d0 2024-03-23T14:57:25.184 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.177+0000 7feac2d9d640 1 --2- 172.21.15.164:0/3886993758 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feab00051e0 0x7feab000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:25.184 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.177+0000 7feac259c640 1 --2- 172.21.15.164:0/3886993758 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feab00039d0 0x7feab0003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:25.184 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.177+0000 7feab9d9b640 1 --2- 172.21.15.164:0/3886993758 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7feab00047d0 0x7feab0004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:25.184 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.177+0000 7feac2d9d640 1 -- 172.21.15.164:0/3886993758 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7feab00047d0 msgr2=0x7feab0004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:25.184 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.177+0000 7feac2d9d640 1 --2- 172.21.15.164:0/3886993758 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7feab00047d0 0x7feab0004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:25.184 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.177+0000 7feac2d9d640 1 -- 172.21.15.164:0/3886993758 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feab00039d0 msgr2=0x7feab0003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:25.184 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.177+0000 7feac2d9d640 1 --2- 172.21.15.164:0/3886993758 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feab00039d0 0x7feab0003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:25.184 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.177+0000 7feac2d9d640 1 -- 172.21.15.164:0/3886993758 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feab0003530 con 0x7feab00051e0 2024-03-23T14:57:25.187 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.177+0000 7feac2d9d640 1 -- 172.21.15.164:0/3886993758 reap_dead start 2024-03-23T14:57:25.188 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.177+0000 7feac2d9d640 1 --2- 172.21.15.164:0/3886993758 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feab00051e0 0x7feab000b990 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7feabc001e40 tx=0x7feabc01c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92907d314dd6e6b1 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:25.188 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.177+0000 7feac1d9b640 1 -- 172.21.15.164:0/3886993758 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feabc003b80 con 0x7feab00051e0 2024-03-23T14:57:25.188 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.177+0000 7feac1d9b640 1 -- 172.21.15.164:0/3886993758 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feabc003ce0 con 0x7feab00051e0 2024-03-23T14:57:25.188 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.177+0000 7feac1d9b640 1 -- 172.21.15.164:0/3886993758 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feabc0052f0 con 0x7feab00051e0 2024-03-23T14:57:25.188 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.181+0000 7feac891d640 1 -- 172.21.15.164:0/3886993758 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feab00051e0 msgr2=0x7feab000b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:25.188 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.181+0000 7feac891d640 1 --2- 172.21.15.164:0/3886993758 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feab00051e0 0x7feab000b990 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7feabc001e40 tx=0x7feabc01c740 comp rx=0 tx=0).stop 2024-03-23T14:57:25.188 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.181+0000 7feac891d640 1 -- 172.21.15.164:0/3886993758 shutdown_connections 2024-03-23T14:57:25.188 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.181+0000 7feac891d640 1 --2- 172.21.15.164:0/3886993758 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feab00051e0 0x7feab000b990 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:25.188 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.181+0000 7feac891d640 1 -- 172.21.15.164:0/3886993758 >> 172.21.15.164:0/3886993758 conn(0x7feac406b930 msgr2=0x7feac40b9150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:25.188 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.181+0000 7feac891d640 1 -- 172.21.15.164:0/3886993758 shutdown_connections 2024-03-23T14:57:25.188 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.181+0000 7feac891d640 1 -- 172.21.15.164:0/3886993758 wait complete. 2024-03-23T14:57:25.188 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.181+0000 7feac891d640 1 Processor -- start 2024-03-23T14:57:25.188 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.181+0000 7feac891d640 1 -- start start 2024-03-23T14:57:25.188 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.181+0000 7feac891d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feab00051e0 0x7feac4131160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:25.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.181+0000 7feac891d640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7feac40b8a80 0x7feac4132210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:25.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.181+0000 7feac891d640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feac4132ab0 0x7feac41dba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:25.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.181+0000 7feac891d640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7feac4131de0 con 0x7feab00051e0 2024-03-23T14:57:25.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.181+0000 7feac891d640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7feac40b8f00 con 0x7feac4132ab0 2024-03-23T14:57:25.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.181+0000 7feac891d640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7feac4133480 con 0x7feac40b8a80 2024-03-23T14:57:25.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.181+0000 7feac259c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feab00051e0 0x7feac4131160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:25.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.181+0000 7feac259c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feab00051e0 0x7feac4131160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:49200/0 (socket says 172.21.15.164:49200) 2024-03-23T14:57:25.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.181+0000 7feac259c640 1 -- 172.21.15.164:0/1180242371 learned_addr learned my addr 172.21.15.164:0/1180242371 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:25.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.181+0000 7feab9d9b640 1 --2- 172.21.15.164:0/1180242371 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7feac40b8a80 0x7feac4132210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:25.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.181+0000 7feac2d9d640 1 --2- 172.21.15.164:0/1180242371 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feac4132ab0 0x7feac41dba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:25.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.181+0000 7feac259c640 1 -- 172.21.15.164:0/1180242371 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7feac40b8a80 msgr2=0x7feac4132210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:25.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.181+0000 7feac259c640 1 --2- 172.21.15.164:0/1180242371 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7feac40b8a80 0x7feac4132210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:25.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.181+0000 7feac259c640 1 -- 172.21.15.164:0/1180242371 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feac4132ab0 msgr2=0x7feac41dba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:25.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.181+0000 7feac259c640 1 --2- 172.21.15.164:0/1180242371 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feac4132ab0 0x7feac41dba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:25.189 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.181+0000 7feac259c640 1 -- 172.21.15.164:0/1180242371 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feac4070af0 con 0x7feab00051e0 2024-03-23T14:57:25.190 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.181+0000 7feac259c640 1 --2- 172.21.15.164:0/1180242371 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feab00051e0 0x7feac4131160 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7feab40118b0 tx=0x7feab4011d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:25.190 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.181+0000 7feabb7fe640 1 -- 172.21.15.164:0/1180242371 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feab400fb60 con 0x7feab00051e0 2024-03-23T14:57:25.190 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.181+0000 7feabb7fe640 1 -- 172.21.15.164:0/1180242371 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feab400fcc0 con 0x7feab00051e0 2024-03-23T14:57:25.190 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.181+0000 7feac891d640 1 -- 172.21.15.164:0/1180242371 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feac4072790 con 0x7feab00051e0 2024-03-23T14:57:25.190 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.181+0000 7feabb7fe640 1 -- 172.21.15.164:0/1180242371 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feab4015650 con 0x7feab00051e0 2024-03-23T14:57:25.190 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.181+0000 7feac891d640 1 -- 172.21.15.164:0/1180242371 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feac40733e0 con 0x7feab00051e0 2024-03-23T14:57:25.190 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.181+0000 7feac891d640 1 -- 172.21.15.164:0/1180242371 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feac4072cb0 con 0x7feab00051e0 2024-03-23T14:57:25.191 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.185+0000 7feabb7fe640 1 -- 172.21.15.164:0/1180242371 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7feab40157b0 con 0x7feab00051e0 2024-03-23T14:57:25.191 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.185+0000 7feabb7fe640 1 --2- 172.21.15.164:0/1180242371 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7feaa40672a0 0x7feaa4069760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:25.191 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.185+0000 7feabb7fe640 1 -- 172.21.15.164:0/1180242371 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7feab4088bc0 con 0x7feab00051e0 2024-03-23T14:57:25.191 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.185+0000 7feab9d9b640 1 --2- 172.21.15.164:0/1180242371 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7feaa40672a0 0x7feaa4069760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:25.191 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.185+0000 7feab9d9b640 1 --2- 172.21.15.164:0/1180242371 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7feaa40672a0 0x7feaa4069760 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7feac4142450 tx=0x7feaac008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:25.193 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.189+0000 7feabb7fe640 1 -- 172.21.15.164:0/1180242371 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7feab40557e0 con 0x7feab00051e0 2024-03-23T14:57:25.339 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.333+0000 7feac891d640 1 -- 172.21.15.164:0/1180242371 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7feac40725f0 con 0x7feab00051e0 2024-03-23T14:57:25.340 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.337+0000 7feabb7fe640 1 -- 172.21.15.164:0/1180242371 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7feac40725f0 con 0x7feab00051e0 2024-03-23T14:57:25.340 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 1 2024-03-23T14:57:25.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.337+0000 7feac891d640 1 -- 172.21.15.164:0/1180242371 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7feaa40672a0 msgr2=0x7feaa4069760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:25.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.337+0000 7feac891d640 1 --2- 172.21.15.164:0/1180242371 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7feaa40672a0 0x7feaa4069760 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7feac4142450 tx=0x7feaac008040 comp rx=0 tx=0).stop 2024-03-23T14:57:25.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.337+0000 7feac891d640 1 -- 172.21.15.164:0/1180242371 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feab00051e0 msgr2=0x7feac4131160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:25.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.337+0000 7feac891d640 1 --2- 172.21.15.164:0/1180242371 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feab00051e0 0x7feac4131160 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7feab40118b0 tx=0x7feab4011d70 comp rx=0 tx=0).stop 2024-03-23T14:57:25.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.337+0000 7feac891d640 1 -- 172.21.15.164:0/1180242371 shutdown_connections 2024-03-23T14:57:25.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.337+0000 7feac891d640 1 --2- 172.21.15.164:0/1180242371 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7feaa40672a0 0x7feaa4069760 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:25.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.337+0000 7feac891d640 1 --2- 172.21.15.164:0/1180242371 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feac4132ab0 0x7feac41dba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:25.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.337+0000 7feac891d640 1 --2- 172.21.15.164:0/1180242371 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7feac40b8a80 0x7feac4132210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:25.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.337+0000 7feac891d640 1 --2- 172.21.15.164:0/1180242371 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feab00051e0 0x7feac4131160 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:25.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.337+0000 7feac891d640 1 -- 172.21.15.164:0/1180242371 >> 172.21.15.164:0/1180242371 conn(0x7feac406b930 msgr2=0x7feac40b3a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:25.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.337+0000 7feac891d640 1 -- 172.21.15.164:0/1180242371 shutdown_connections 2024-03-23T14:57:25.343 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.337+0000 7feac891d640 1 -- 172.21.15.164:0/1180242371 wait complete. 2024-03-23T14:57:25.357 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-23T14:57:25.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.465+0000 7f2b833f6640 1 Processor -- start 2024-03-23T14:57:25.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.465+0000 7f2b833f6640 1 -- start start 2024-03-23T14:57:25.472 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.469+0000 7f2b833f6640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f2b7c0b56f0 0x7f2b7c14a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:25.473 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.469+0000 7f2b833f6640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f2b7c0b4a70 0x7f2b7c140f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:25.473 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.469+0000 7f2b833f6640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b7c0b4300 con 0x7f2b7c141470 2024-03-23T14:57:25.473 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.469+0000 7f2b833f6640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f2b7c0b5fa0 con 0x7f2b7c0b56f0 2024-03-23T14:57:25.473 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.469+0000 7f2b833f6640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 -- 0x7f2b7c0b45e0 con 0x7f2b7c0b4a70 2024-03-23T14:57:25.473 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.469+0000 7f2b81192640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f2b7c0b56f0 0x7f2b7c14a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:25.473 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.469+0000 7f2b81192640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f2b7c0b56f0 0x7f2b7c14a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:49210/0 (socket says 172.21.15.164:49210) 2024-03-23T14:57:25.473 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.469+0000 7f2b81192640 1 -- 172.21.15.164:0/41401662 learned_addr learned my addr 172.21.15.164:0/41401662 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:25.473 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.469+0000 7f2b81993640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f2b7c141470 0x7f2b7c0b75f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:42460/0 (socket says 172.21.15.164:42460) 2024-03-23T14:57:25.473 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.469+0000 7f2b80991640 1 --2- 172.21.15.164:0/41401662 >> v2:172.21.15.173:3300/0 conn(0x7f2b7c0b4a70 0x7f2b7c140f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:25.473 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.469+0000 7f2b81192640 1 -- 172.21.15.164:0/41401662 >> v2:172.21.15.173:3300/0 conn(0x7f2b7c0b4a70 msgr2=0x7f2b7c140f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:25.473 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.469+0000 7f2b81192640 1 --2- 172.21.15.164:0/41401662 >> v2:172.21.15.173:3300/0 conn(0x7f2b7c0b4a70 0x7f2b7c140f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:25.473 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.469+0000 7f2b81192640 1 -- 172.21.15.164:0/41401662 >> v1:172.21.15.164:6789/0 conn(0x7f2b7c141470 legacy=0x7f2b7c0b75f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:25.474 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.469+0000 7f2b81192640 1 -- 172.21.15.164:0/41401662 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b7c14ade0 con 0x7f2b7c0b56f0 2024-03-23T14:57:25.474 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.469+0000 7f2b81192640 1 --2- 172.21.15.164:0/41401662 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2b7c0b56f0 0x7f2b7c14a8a0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f2b7c0bdd50 tx=0x7f2b7801c3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6abfa52ceced7bc0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:25.475 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.469+0000 7f2b73fff640 1 -- 172.21.15.164:0/41401662 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b78004230 con 0x7f2b7c0b56f0 2024-03-23T14:57:25.475 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.469+0000 7f2b73fff640 1 -- 172.21.15.164:0/41401662 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b780a7050 con 0x7f2b7c0b56f0 2024-03-23T14:57:25.475 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.469+0000 7f2b73fff640 1 -- 172.21.15.164:0/41401662 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b78003620 con 0x7f2b7c0b56f0 2024-03-23T14:57:25.476 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.469+0000 7f2b833f6640 1 -- 172.21.15.164:0/41401662 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2b7c0b56f0 msgr2=0x7f2b7c14a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:25.476 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.469+0000 7f2b833f6640 1 --2- 172.21.15.164:0/41401662 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2b7c0b56f0 0x7f2b7c14a8a0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f2b7c0bdd50 tx=0x7f2b7801c3b0 comp rx=0 tx=0).stop 2024-03-23T14:57:25.476 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.469+0000 7f2b833f6640 1 -- 172.21.15.164:0/41401662 shutdown_connections 2024-03-23T14:57:25.476 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.469+0000 7f2b833f6640 1 --2- 172.21.15.164:0/41401662 >> v2:172.21.15.173:3300/0 conn(0x7f2b7c0b4a70 0x7f2b7c140f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:25.476 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.469+0000 7f2b833f6640 1 --2- 172.21.15.164:0/41401662 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2b7c0b56f0 0x7f2b7c14a8a0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:25.476 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.469+0000 7f2b833f6640 1 -- 172.21.15.164:0/41401662 >> 172.21.15.164:0/41401662 conn(0x7f2b7c02c3c0 msgr2=0x7f2b7c02c7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:25.476 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.469+0000 7f2b833f6640 1 -- 172.21.15.164:0/41401662 shutdown_connections 2024-03-23T14:57:25.476 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.469+0000 7f2b833f6640 1 -- 172.21.15.164:0/41401662 wait complete. 2024-03-23T14:57:25.479 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.473+0000 7f2b833f6640 1 Processor -- start 2024-03-23T14:57:25.479 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.473+0000 7f2b833f6640 1 -- start start 2024-03-23T14:57:25.479 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.473+0000 7f2b833f6640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2b7c0b4a70 0x7f2b7c1480e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:25.479 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.473+0000 7f2b833f6640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2b7c0b56f0 0x7f2b7c144750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:25.479 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.473+0000 7f2b833f6640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2b7c141470 0x7f2b7c1dc960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:25.479 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.473+0000 7f2b833f6640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2b7c147cc0 con 0x7f2b7c0b56f0 2024-03-23T14:57:25.479 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.473+0000 7f2b833f6640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f2b7c1dcea0 con 0x7f2b7c141470 2024-03-23T14:57:25.479 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.473+0000 7f2b833f6640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f2b7c147970 con 0x7f2b7c0b4a70 2024-03-23T14:57:25.479 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.473+0000 7f2b80991640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2b7c0b56f0 0x7f2b7c144750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:25.479 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.473+0000 7f2b80991640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2b7c0b56f0 0x7f2b7c144750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:49214/0 (socket says 172.21.15.164:49214) 2024-03-23T14:57:25.479 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.473+0000 7f2b80991640 1 -- 172.21.15.164:0/857521454 learned_addr learned my addr 172.21.15.164:0/857521454 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:25.479 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.473+0000 7f2b81192640 1 --2- 172.21.15.164:0/857521454 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2b7c0b4a70 0x7f2b7c1480e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:25.479 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.473+0000 7f2b81993640 1 --2- 172.21.15.164:0/857521454 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2b7c141470 0x7f2b7c1dc960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:25.480 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.473+0000 7f2b80991640 1 -- 172.21.15.164:0/857521454 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2b7c0b4a70 msgr2=0x7f2b7c1480e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:25.480 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.473+0000 7f2b80991640 1 --2- 172.21.15.164:0/857521454 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2b7c0b4a70 0x7f2b7c1480e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:25.480 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.473+0000 7f2b80991640 1 -- 172.21.15.164:0/857521454 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2b7c141470 msgr2=0x7f2b7c1dc960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:25.480 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.473+0000 7f2b80991640 1 --2- 172.21.15.164:0/857521454 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2b7c141470 0x7f2b7c1dc960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:25.480 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.473+0000 7f2b80991640 1 -- 172.21.15.164:0/857521454 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b780094d0 con 0x7f2b7c0b56f0 2024-03-23T14:57:25.480 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.473+0000 7f2b81192640 1 --2- 172.21.15.164:0/857521454 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2b7c0b4a70 0x7f2b7c1480e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T14:57:25.480 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.473+0000 7f2b81993640 1 --2- 172.21.15.164:0/857521454 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2b7c141470 0x7f2b7c1dc960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T14:57:25.480 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.473+0000 7f2b80991640 1 --2- 172.21.15.164:0/857521454 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2b7c0b56f0 0x7f2b7c144750 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f2b6c00d830 tx=0x7f2b6c00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:25.480 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.473+0000 7f2b71ffb640 1 -- 172.21.15.164:0/857521454 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b6c00b6c0 con 0x7f2b7c0b56f0 2024-03-23T14:57:25.480 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.473+0000 7f2b71ffb640 1 -- 172.21.15.164:0/857521454 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b6c00b860 con 0x7f2b7c0b56f0 2024-03-23T14:57:25.480 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.473+0000 7f2b833f6640 1 -- 172.21.15.164:0/857521454 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b7c143ff0 con 0x7f2b7c0b56f0 2024-03-23T14:57:25.480 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.473+0000 7f2b71ffb640 1 -- 172.21.15.164:0/857521454 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b6c00bc10 con 0x7f2b7c0b56f0 2024-03-23T14:57:25.480 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.473+0000 7f2b833f6640 1 -- 172.21.15.164:0/857521454 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b7c146db0 con 0x7f2b7c0b56f0 2024-03-23T14:57:25.480 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.473+0000 7f2b833f6640 1 -- 172.21.15.164:0/857521454 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b7c147e00 con 0x7f2b7c0b56f0 2024-03-23T14:57:25.481 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.473+0000 7f2b71ffb640 1 -- 172.21.15.164:0/857521454 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7f2b6c01e460 con 0x7f2b7c0b56f0 2024-03-23T14:57:25.481 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.477+0000 7f2b71ffb640 1 --2- 172.21.15.164:0/857521454 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f2b600672a0 0x7f2b60069760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:25.481 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.477+0000 7f2b81192640 1 --2- 172.21.15.164:0/857521454 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f2b600672a0 0x7f2b60069760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:25.481 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.477+0000 7f2b71ffb640 1 -- 172.21.15.164:0/857521454 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f2b6c085760 con 0x7f2b7c0b56f0 2024-03-23T14:57:25.481 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.477+0000 7f2b81192640 1 --2- 172.21.15.164:0/857521454 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f2b600672a0 0x7f2b60069760 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f2b780ad000 tx=0x7f2b780a6000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:25.484 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.481+0000 7f2b71ffb640 1 -- 172.21.15.164:0/857521454 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7f2b7c147e00 con 0x7f2b7c0b56f0 2024-03-23T14:57:25.632 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.629+0000 7f2b833f6640 1 -- 172.21.15.164:0/857521454 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f2b7c147e00 con 0x7f2b7c0b56f0 2024-03-23T14:57:25.633 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.629+0000 7f2b71ffb640 1 -- 172.21.15.164:0/857521454 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f2b7c147e00 con 0x7f2b7c0b56f0 2024-03-23T14:57:25.633 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 32 2024-03-23T14:57:25.636 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.629+0000 7f2b833f6640 1 -- 172.21.15.164:0/857521454 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f2b600672a0 msgr2=0x7f2b60069760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:25.636 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.629+0000 7f2b833f6640 1 --2- 172.21.15.164:0/857521454 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f2b600672a0 0x7f2b60069760 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f2b780ad000 tx=0x7f2b780a6000 comp rx=0 tx=0).stop 2024-03-23T14:57:25.636 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.629+0000 7f2b833f6640 1 -- 172.21.15.164:0/857521454 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2b7c0b56f0 msgr2=0x7f2b7c144750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:25.636 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.629+0000 7f2b833f6640 1 --2- 172.21.15.164:0/857521454 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2b7c0b56f0 0x7f2b7c144750 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f2b6c00d830 tx=0x7f2b6c00dcf0 comp rx=0 tx=0).stop 2024-03-23T14:57:25.636 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.629+0000 7f2b833f6640 1 -- 172.21.15.164:0/857521454 shutdown_connections 2024-03-23T14:57:25.636 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.629+0000 7f2b833f6640 1 --2- 172.21.15.164:0/857521454 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f2b600672a0 0x7f2b60069760 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:25.636 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.629+0000 7f2b833f6640 1 --2- 172.21.15.164:0/857521454 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2b7c141470 0x7f2b7c1dc960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:25.636 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.629+0000 7f2b833f6640 1 --2- 172.21.15.164:0/857521454 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2b7c0b56f0 0x7f2b7c144750 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:25.636 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.629+0000 7f2b833f6640 1 --2- 172.21.15.164:0/857521454 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2b7c0b4a70 0x7f2b7c1480e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:25.636 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.629+0000 7f2b833f6640 1 -- 172.21.15.164:0/857521454 >> 172.21.15.164:0/857521454 conn(0x7f2b7c02c3c0 msgr2=0x7f2b7c14b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:25.636 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.633+0000 7f2b833f6640 1 -- 172.21.15.164:0/857521454 shutdown_connections 2024-03-23T14:57:25.636 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.633+0000 7f2b833f6640 1 -- 172.21.15.164:0/857521454 wait complete. 2024-03-23T14:57:25.651 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-23T14:57:25.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.773+0000 7f077e58b640 1 Processor -- start 2024-03-23T14:57:25.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.773+0000 7f077e58b640 1 -- start start 2024-03-23T14:57:25.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.773+0000 7f077e58b640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f07780b1cd0 0x7f077814efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:25.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.773+0000 7f077e58b640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f077814f4e0 0x7f0778156070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:25.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.773+0000 7f077e58b640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f077814f970 con 0x7f07780bd3b0 2024-03-23T14:57:25.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.773+0000 7f077e58b640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f07780bd790 con 0x7f077814f4e0 2024-03-23T14:57:25.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.773+0000 7f077e58b640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f07780b2dc0 con 0x7f07780b1cd0 2024-03-23T14:57:25.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.773+0000 7f077cb28640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f077814f4e0 0x7f0778156070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:25.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.773+0000 7f0777fff640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f07780bd3b0 0x7f07781456d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:42470/0 (socket says 172.21.15.164:42470) 2024-03-23T14:57:25.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.773+0000 7f0777fff640 1 -- 172.21.15.164:0/2599946073 learned_addr learned my addr 172.21.15.164:0/2599946073 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-03-23T14:57:25.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.773+0000 7f07777fe640 1 --2- 172.21.15.164:0/2599946073 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f07780b1cd0 0x7f077814efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:25.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.773+0000 7f0776ffd640 1 -- 172.21.15.164:0/2599946073 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 107286745 0 0) 0x7f077814f970 con 0x7f07780bd3b0 2024-03-23T14:57:25.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.773+0000 7f0776ffd640 1 -- 172.21.15.164:0/2599946073 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0760003530 con 0x7f07780bd3b0 2024-03-23T14:57:25.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.773+0000 7f077cb28640 1 -- 172.21.15.164:0/2599946073 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f07780b1cd0 msgr2=0x7f077814efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:25.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.773+0000 7f077cb28640 1 --2- 172.21.15.164:0/2599946073 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f07780b1cd0 0x7f077814efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:25.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.773+0000 7f077cb28640 1 -- 172.21.15.164:0/2599946073 >> v1:172.21.15.164:6789/0 conn(0x7f07780bd3b0 legacy=0x7f07781456d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:25.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.773+0000 7f077cb28640 1 -- 172.21.15.164:0/2599946073 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07780b30a0 con 0x7f077814f4e0 2024-03-23T14:57:25.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.773+0000 7f077cb28640 1 --2- 172.21.15.164:0/2599946073 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f077814f4e0 0x7f0778156070 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f0770009d90 tx=0x7f077001c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2679b54ce6f57a06 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:25.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.777+0000 7f0776ffd640 1 -- 172.21.15.164:0/2599946073 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07700a5950 con 0x7f077814f4e0 2024-03-23T14:57:25.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.777+0000 7f0776ffd640 1 -- 172.21.15.164:0/2599946073 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07700a5ab0 con 0x7f077814f4e0 2024-03-23T14:57:25.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.777+0000 7f0776ffd640 1 -- 172.21.15.164:0/2599946073 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07700a5e40 con 0x7f077814f4e0 2024-03-23T14:57:25.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.777+0000 7f077e58b640 1 -- 172.21.15.164:0/2599946073 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f077814f4e0 msgr2=0x7f0778156070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:25.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.777+0000 7f077e58b640 1 --2- 172.21.15.164:0/2599946073 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f077814f4e0 0x7f0778156070 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f0770009d90 tx=0x7f077001c8b0 comp rx=0 tx=0).stop 2024-03-23T14:57:25.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.777+0000 7f077e58b640 1 -- 172.21.15.164:0/2599946073 shutdown_connections 2024-03-23T14:57:25.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.777+0000 7f077e58b640 1 --2- 172.21.15.164:0/2599946073 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f077814f4e0 0x7f0778156070 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:25.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.777+0000 7f077e58b640 1 --2- 172.21.15.164:0/2599946073 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f07780b1cd0 0x7f077814efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:25.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.777+0000 7f077e58b640 1 -- 172.21.15.164:0/2599946073 >> 172.21.15.164:0/2599946073 conn(0x7f077802c3c0 msgr2=0x7f077802c7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:25.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.777+0000 7f077e58b640 1 -- 172.21.15.164:0/2599946073 shutdown_connections 2024-03-23T14:57:25.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.777+0000 7f077e58b640 1 -- 172.21.15.164:0/2599946073 wait complete. 2024-03-23T14:57:25.783 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.777+0000 7f077e58b640 1 Processor -- start 2024-03-23T14:57:25.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.777+0000 7f077e58b640 1 -- start start 2024-03-23T14:57:25.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.777+0000 7f077e58b640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f07780b1cd0 0x7f0778146490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:25.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.777+0000 7f077e58b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f07780bd3b0 0x7f07781e7d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:25.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.777+0000 7f077e58b640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f077814f4e0 0x7f077814b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:25.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.777+0000 7f077e58b640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f07781e78b0 con 0x7f07780bd3b0 2024-03-23T14:57:25.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.777+0000 7f077e58b640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f07781448b0 con 0x7f077814f4e0 2024-03-23T14:57:25.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.777+0000 7f077e58b640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f0778147700 con 0x7f07780b1cd0 2024-03-23T14:57:25.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.777+0000 7f07777fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f07780bd3b0 0x7f07781e7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:25.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.777+0000 7f07777fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f07780bd3b0 0x7f07781e7d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:49238/0 (socket says 172.21.15.164:49238) 2024-03-23T14:57:25.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.777+0000 7f07777fe640 1 -- 172.21.15.164:0/1862336121 learned_addr learned my addr 172.21.15.164:0/1862336121 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:25.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.777+0000 7f0777fff640 1 --2- 172.21.15.164:0/1862336121 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f07780b1cd0 0x7f0778146490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:25.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.777+0000 7f077cb28640 1 --2- 172.21.15.164:0/1862336121 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f077814f4e0 0x7f077814b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:25.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.777+0000 7f07777fe640 1 -- 172.21.15.164:0/1862336121 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f07780b1cd0 msgr2=0x7f0778146490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:25.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.777+0000 7f07777fe640 1 --2- 172.21.15.164:0/1862336121 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f07780b1cd0 0x7f0778146490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:25.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.777+0000 7f07777fe640 1 -- 172.21.15.164:0/1862336121 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f077814f4e0 msgr2=0x7f077814b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:25.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.777+0000 7f07777fe640 1 --2- 172.21.15.164:0/1862336121 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f077814f4e0 0x7f077814b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:25.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.777+0000 7f07777fe640 1 -- 172.21.15.164:0/1862336121 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0770009a70 con 0x7f07780bd3b0 2024-03-23T14:57:25.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.777+0000 7f07777fe640 1 --2- 172.21.15.164:0/1862336121 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f07780bd3b0 0x7f07781e7d40 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f076c00d800 tx=0x7f076c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:25.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.777+0000 7f0774ff9640 1 -- 172.21.15.164:0/1862336121 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f076c00b6c0 con 0x7f07780bd3b0 2024-03-23T14:57:25.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.777+0000 7f0774ff9640 1 -- 172.21.15.164:0/1862336121 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f076c00b820 con 0x7f07780bd3b0 2024-03-23T14:57:25.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.777+0000 7f0774ff9640 1 -- 172.21.15.164:0/1862336121 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f076c00bb90 con 0x7f07780bd3b0 2024-03-23T14:57:25.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.777+0000 7f077e58b640 1 -- 172.21.15.164:0/1862336121 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f077814a720 con 0x7f07780bd3b0 2024-03-23T14:57:25.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.777+0000 7f077e58b640 1 -- 172.21.15.164:0/1862336121 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07781e7540 con 0x7f07780bd3b0 2024-03-23T14:57:25.786 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.781+0000 7f077e58b640 1 -- 172.21.15.164:0/1862336121 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07781e7a20 con 0x7f07780bd3b0 2024-03-23T14:57:25.786 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.781+0000 7f0774ff9640 1 -- 172.21.15.164:0/1862336121 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7f076c00bcf0 con 0x7f07780bd3b0 2024-03-23T14:57:25.786 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.781+0000 7f0774ff9640 1 --2- 172.21.15.164:0/1862336121 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f07640672a0 0x7f0764069760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:25.786 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.781+0000 7f0777fff640 1 --2- 172.21.15.164:0/1862336121 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f07640672a0 0x7f0764069760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:25.786 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.781+0000 7f0774ff9640 1 -- 172.21.15.164:0/1862336121 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f076c0854e0 con 0x7f07780bd3b0 2024-03-23T14:57:25.786 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.781+0000 7f0777fff640 1 --2- 172.21.15.164:0/1862336121 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f07640672a0 0x7f0764069760 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f0768005890 tx=0x7f0768009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:25.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.785+0000 7f0774ff9640 1 -- 172.21.15.164:0/1862336121 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7f076c052120 con 0x7f07780bd3b0 2024-03-23T14:57:25.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.949+0000 7f077e58b640 1 -- 172.21.15.164:0/1862336121 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f07780690a0 con 0x7f07780bd3b0 2024-03-23T14:57:25.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.949+0000 7f0774ff9640 1 -- 172.21.15.164:0/1862336121 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f076c0557e0 con 0x7f07780bd3b0 2024-03-23T14:57:25.955 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 64 2024-03-23T14:57:25.957 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.953+0000 7f077e58b640 1 -- 172.21.15.164:0/1862336121 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f07640672a0 msgr2=0x7f0764069760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:25.957 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.953+0000 7f077e58b640 1 --2- 172.21.15.164:0/1862336121 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f07640672a0 0x7f0764069760 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f0768005890 tx=0x7f0768009040 comp rx=0 tx=0).stop 2024-03-23T14:57:25.957 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.953+0000 7f077e58b640 1 -- 172.21.15.164:0/1862336121 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f07780bd3b0 msgr2=0x7f07781e7d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:25.957 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.953+0000 7f077e58b640 1 --2- 172.21.15.164:0/1862336121 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f07780bd3b0 0x7f07781e7d40 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f076c00d800 tx=0x7f076c00dcc0 comp rx=0 tx=0).stop 2024-03-23T14:57:25.957 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.953+0000 7f077e58b640 1 -- 172.21.15.164:0/1862336121 shutdown_connections 2024-03-23T14:57:25.957 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.953+0000 7f077e58b640 1 --2- 172.21.15.164:0/1862336121 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f07640672a0 0x7f0764069760 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:25.957 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.953+0000 7f077e58b640 1 --2- 172.21.15.164:0/1862336121 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f077814f4e0 0x7f077814b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:25.957 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.953+0000 7f077e58b640 1 --2- 172.21.15.164:0/1862336121 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f07780bd3b0 0x7f07781e7d40 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:25.957 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.953+0000 7f077e58b640 1 --2- 172.21.15.164:0/1862336121 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f07780b1cd0 0x7f0778146490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:25.958 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.953+0000 7f077e58b640 1 -- 172.21.15.164:0/1862336121 >> 172.21.15.164:0/1862336121 conn(0x7f077802c3c0 msgr2=0x7f0778140c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:25.958 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.953+0000 7f077e58b640 1 -- 172.21.15.164:0/1862336121 shutdown_connections 2024-03-23T14:57:25.958 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:25.953+0000 7f077e58b640 1 -- 172.21.15.164:0/1862336121 wait complete. 2024-03-23T14:57:25.973 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-23T14:57:26.092 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.089+0000 7f4477da7640 1 Processor -- start 2024-03-23T14:57:26.092 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.089+0000 7f4477da7640 1 -- start start 2024-03-23T14:57:26.094 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.089+0000 7f4477da7640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f44700adc90 0x7f44700ae070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:26.094 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.089+0000 7f4477da7640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f447006c9c0 con 0x7f447006bf70 2024-03-23T14:57:26.094 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.089+0000 7f4477da7640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44700af460 con 0x7f447014ad20 2024-03-23T14:57:26.095 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.089+0000 7f4477da7640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f44700ae810 con 0x7f44700adc90 2024-03-23T14:57:26.095 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.089+0000 7f4475342640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f447006bf70 0x7f44700b4a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:42484/0 (socket says 172.21.15.164:42484) 2024-03-23T14:57:26.095 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.089+0000 7f4475342640 1 -- 172.21.15.164:0/3210431980 learned_addr learned my addr 172.21.15.164:0/3210431980 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-03-23T14:57:26.095 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.089+0000 7f4475b43640 1 --2- 172.21.15.164:0/3210431980 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f44700adc90 0x7f44700ae070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:26.095 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.089+0000 7f4474b41640 1 -- 172.21.15.164:0/3210431980 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1654923936 0 0) 0x7f447006c9c0 con 0x7f447006bf70 2024-03-23T14:57:26.095 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.089+0000 7f4474b41640 1 -- 172.21.15.164:0/3210431980 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f445c003530 con 0x7f447006bf70 2024-03-23T14:57:26.095 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.089+0000 7f4474b41640 1 -- 172.21.15.164:0/3210431980 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2779081521 0 0) 0x7f445c003530 con 0x7f447006bf70 2024-03-23T14:57:26.095 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.089+0000 7f4474b41640 1 -- 172.21.15.164:0/3210431980 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f44700adc90 msgr2=0x7f44700ae070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:26.095 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.089+0000 7f4474b41640 1 --2- 172.21.15.164:0/3210431980 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f44700adc90 0x7f44700ae070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:26.095 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.089+0000 7f4474b41640 1 -- 172.21.15.164:0/3210431980 >> v1:172.21.15.173:6789/0 conn(0x7f447014ad20 legacy=0x7f447014b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:26.095 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.089+0000 7f4474b41640 1 -- 172.21.15.164:0/3210431980 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44700b5170 con 0x7f447006bf70 2024-03-23T14:57:26.095 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.089+0000 7f4474b41640 1 -- 172.21.15.164:0/3210431980 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2526390548 0 0) 0x7f446c001680 con 0x7f447006bf70 2024-03-23T14:57:26.095 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.089+0000 7f4474b41640 1 -- 172.21.15.164:0/3210431980 >> v1:172.21.15.164:6789/0 conn(0x7f447006bf70 legacy=0x7f44700b4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:26.096 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.089+0000 7f4474b41640 1 --2- 172.21.15.164:0/3210431980 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f44700adc90 0x7f445c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:26.096 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.089+0000 7f4474b41640 1 --2- 172.21.15.164:0/3210431980 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f445c003f30 0x7f445c0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:26.096 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.089+0000 7f4474b41640 1 --2- 172.21.15.164:0/3210431980 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f445c004d10 0x7f445c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:26.096 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.089+0000 7f4474b41640 1 -- 172.21.15.164:0/3210431980 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f44700ae810 con 0x7f445c003f30 2024-03-23T14:57:26.096 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.089+0000 7f4474b41640 1 -- 172.21.15.164:0/3210431980 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f445c00bbd0 con 0x7f445c004d10 2024-03-23T14:57:26.096 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.089+0000 7f4474b41640 1 -- 172.21.15.164:0/3210431980 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f445c00bd90 con 0x7f44700adc90 2024-03-23T14:57:26.096 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.089+0000 7f4475342640 1 --2- 172.21.15.164:0/3210431980 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f445c003f30 0x7f445c0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:26.096 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.089+0000 7f4475b43640 1 --2- 172.21.15.164:0/3210431980 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f44700adc90 0x7f445c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:26.096 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.089+0000 7f4476344640 1 --2- 172.21.15.164:0/3210431980 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f445c004d10 0x7f445c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:26.096 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.089+0000 7f4475342640 1 -- 172.21.15.164:0/3210431980 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f44700adc90 msgr2=0x7f445c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:26.096 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.089+0000 7f4475342640 1 --2- 172.21.15.164:0/3210431980 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f44700adc90 0x7f445c0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:26.096 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.089+0000 7f4475342640 1 -- 172.21.15.164:0/3210431980 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f445c004d10 msgr2=0x7f445c005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:26.096 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.089+0000 7f4475342640 1 --2- 172.21.15.164:0/3210431980 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f445c004d10 0x7f445c005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:26.096 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.089+0000 7f4475342640 1 -- 172.21.15.164:0/3210431980 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f445c003530 con 0x7f445c003f30 2024-03-23T14:57:26.098 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.093+0000 7f4475342640 1 --2- 172.21.15.164:0/3210431980 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f445c003f30 0x7f445c0047d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f446c00c240 tx=0x7f446c01cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3861f7eb6e2fe9b5 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:26.098 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.093+0000 7f4474b41640 1 -- 172.21.15.164:0/3210431980 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f446c005ae0 con 0x7f445c003f30 2024-03-23T14:57:26.098 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.093+0000 7f4474b41640 1 -- 172.21.15.164:0/3210431980 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f446c005c40 con 0x7f445c003f30 2024-03-23T14:57:26.098 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.093+0000 7f4474b41640 1 -- 172.21.15.164:0/3210431980 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f446c0a5a70 con 0x7f445c003f30 2024-03-23T14:57:26.098 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.093+0000 7f4477da7640 1 -- 172.21.15.164:0/3210431980 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f445c003f30 msgr2=0x7f445c0047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:26.098 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.093+0000 7f4477da7640 1 --2- 172.21.15.164:0/3210431980 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f445c003f30 0x7f445c0047d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f446c00c240 tx=0x7f446c01cc60 comp rx=0 tx=0).stop 2024-03-23T14:57:26.098 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.093+0000 7f4476344640 1 -- 172.21.15.164:0/3210431980 reap_dead start 2024-03-23T14:57:26.098 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.093+0000 7f4477da7640 1 -- 172.21.15.164:0/3210431980 shutdown_connections 2024-03-23T14:57:26.098 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.093+0000 7f4477da7640 1 -- 172.21.15.164:0/3210431980 >> 172.21.15.164:0/3210431980 conn(0x7f44700b8ca0 msgr2=0x7f44700b90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:26.099 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.093+0000 7f4477da7640 1 -- 172.21.15.164:0/3210431980 shutdown_connections 2024-03-23T14:57:26.099 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.093+0000 7f4477da7640 1 -- 172.21.15.164:0/3210431980 wait complete. 2024-03-23T14:57:26.099 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.093+0000 7f4477da7640 1 Processor -- start 2024-03-23T14:57:26.099 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.093+0000 7f4477da7640 1 -- start start 2024-03-23T14:57:26.101 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.093+0000 7f4477da7640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f445c003f30 0x7f44701515e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:26.101 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.093+0000 7f4477da7640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4470150090 0x7f4470150510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:26.101 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.093+0000 7f4477da7640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4470147580 0x7f4470147a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:26.101 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.093+0000 7f4477da7640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f447014f4c0 con 0x7f4470147580 2024-03-23T14:57:26.101 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.093+0000 7f4477da7640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f4470151c80 con 0x7f445c003f30 2024-03-23T14:57:26.101 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.093+0000 7f4477da7640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f4470072510 con 0x7f4470150090 2024-03-23T14:57:26.101 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.093+0000 7f4476344640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4470147580 0x7f4470147a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:26.101 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.093+0000 7f4476344640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4470147580 0x7f4470147a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:49256/0 (socket says 172.21.15.164:49256) 2024-03-23T14:57:26.102 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.093+0000 7f4476344640 1 -- 172.21.15.164:0/614134795 learned_addr learned my addr 172.21.15.164:0/614134795 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:26.102 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.093+0000 7f4475342640 1 --2- 172.21.15.164:0/614134795 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4470150090 0x7f4470150510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:26.102 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.093+0000 7f4475b43640 1 --2- 172.21.15.164:0/614134795 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f445c003f30 0x7f44701515e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:26.102 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.093+0000 7f4476344640 1 -- 172.21.15.164:0/614134795 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4470150090 msgr2=0x7f4470150510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:26.102 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.093+0000 7f4476344640 1 --2- 172.21.15.164:0/614134795 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4470150090 0x7f4470150510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:26.102 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.093+0000 7f4476344640 1 -- 172.21.15.164:0/614134795 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f445c003f30 msgr2=0x7f44701515e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:26.102 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.093+0000 7f4476344640 1 --2- 172.21.15.164:0/614134795 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f445c003f30 0x7f44701515e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:26.102 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.093+0000 7f4476344640 1 -- 172.21.15.164:0/614134795 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4470150b80 con 0x7f4470147580 2024-03-23T14:57:26.102 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.093+0000 7f4476344640 1 --2- 172.21.15.164:0/614134795 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4470147580 0x7f4470147a10 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f44700af340 tx=0x7f446800ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:26.102 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.093+0000 7f44667fc640 1 -- 172.21.15.164:0/614134795 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4468003980 con 0x7f4470147580 2024-03-23T14:57:26.102 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.093+0000 7f44667fc640 1 -- 172.21.15.164:0/614134795 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4468003ae0 con 0x7f4470147580 2024-03-23T14:57:26.102 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.093+0000 7f4477da7640 1 -- 172.21.15.164:0/614134795 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4470150e60 con 0x7f4470147580 2024-03-23T14:57:26.102 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.093+0000 7f44667fc640 1 -- 172.21.15.164:0/614134795 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4468003e90 con 0x7f4470147580 2024-03-23T14:57:26.103 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.093+0000 7f4477da7640 1 -- 172.21.15.164:0/614134795 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f447014e430 con 0x7f4470147580 2024-03-23T14:57:26.103 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.097+0000 7f4477da7640 1 -- 172.21.15.164:0/614134795 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f447014f600 con 0x7f4470147580 2024-03-23T14:57:26.103 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.097+0000 7f44667fc640 1 -- 172.21.15.164:0/614134795 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7f44680050d0 con 0x7f4470147580 2024-03-23T14:57:26.103 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.097+0000 7f44667fc640 1 --2- 172.21.15.164:0/614134795 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f44540672a0 0x7f4454069760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:26.103 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.097+0000 7f4475b43640 1 --2- 172.21.15.164:0/614134795 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f44540672a0 0x7f4454069760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:26.103 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.097+0000 7f44667fc640 1 -- 172.21.15.164:0/614134795 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f4468084d70 con 0x7f4470147580 2024-03-23T14:57:26.103 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.097+0000 7f4475b43640 1 --2- 172.21.15.164:0/614134795 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f44540672a0 0x7f4454069760 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f447014a750 tx=0x7f4460008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:26.105 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.101+0000 7f44667fc640 1 -- 172.21.15.164:0/614134795 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7f447014f600 con 0x7f4470147580 2024-03-23T14:57:26.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.273+0000 7f4477da7640 1 -- 172.21.15.164:0/614134795 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f447014f600 con 0x7f4470147580 2024-03-23T14:57:26.279 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.273+0000 7f44667fc640 1 -- 172.21.15.164:0/614134795 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) ==== 80+0+0 (secure 0 0 0) 0x7f447014f600 con 0x7f4470147580 2024-03-23T14:57:26.279 INFO:teuthology.orchestra.run.smithi164.stderr:listed 0 entries 2024-03-23T14:57:26.281 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.277+0000 7f4477da7640 1 -- 172.21.15.164:0/614134795 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f44540672a0 msgr2=0x7f4454069760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:26.281 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.277+0000 7f4477da7640 1 --2- 172.21.15.164:0/614134795 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f44540672a0 0x7f4454069760 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f447014a750 tx=0x7f4460008040 comp rx=0 tx=0).stop 2024-03-23T14:57:26.281 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.277+0000 7f4477da7640 1 -- 172.21.15.164:0/614134795 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4470147580 msgr2=0x7f4470147a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:26.281 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.277+0000 7f4477da7640 1 --2- 172.21.15.164:0/614134795 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4470147580 0x7f4470147a10 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f44700af340 tx=0x7f446800ec10 comp rx=0 tx=0).stop 2024-03-23T14:57:26.281 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.277+0000 7f4477da7640 1 -- 172.21.15.164:0/614134795 shutdown_connections 2024-03-23T14:57:26.281 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.277+0000 7f4477da7640 1 --2- 172.21.15.164:0/614134795 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4470147580 0x7f4470147a10 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:26.281 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.277+0000 7f4477da7640 1 --2- 172.21.15.164:0/614134795 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4470150090 0x7f4470150510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:26.282 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.277+0000 7f4477da7640 1 --2- 172.21.15.164:0/614134795 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f44540672a0 0x7f4454069760 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:26.282 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.277+0000 7f4477da7640 1 --2- 172.21.15.164:0/614134795 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f445c003f30 0x7f44701515e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:26.282 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.277+0000 7f4477da7640 1 -- 172.21.15.164:0/614134795 >> 172.21.15.164:0/614134795 conn(0x7f44700b8ca0 msgr2=0x7f447006cce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:26.282 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.277+0000 7f4477da7640 1 -- 172.21.15.164:0/614134795 shutdown_connections 2024-03-23T14:57:26.282 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.277+0000 7f4477da7640 1 -- 172.21.15.164:0/614134795 wait complete. 2024-03-23T14:57:26.297 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-23T14:57:26.297 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T14:57:26.303 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T14:57:26.468 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.461+0000 7fc784f53640 1 Processor -- start 2024-03-23T14:57:26.468 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.465+0000 7fc784f53640 1 -- start start 2024-03-23T14:57:26.469 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.465+0000 7fc784f53640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fc78009def0 0x7fc7800b45e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:26.469 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.465+0000 7fc784f53640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc7800add50 0x7fc78006bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:26.469 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.465+0000 7fc784f53640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc78006d650 con 0x7fc7800b4b20 2024-03-23T14:57:26.469 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.465+0000 7fc784f53640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 -- 0x7fc7800af4a0 con 0x7fc78009def0 2024-03-23T14:57:26.469 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.465+0000 7fc784f53640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fc78009e2d0 con 0x7fc7800add50 2024-03-23T14:57:26.469 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.465+0000 7fc77dd9b640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc7800add50 0x7fc78006bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:26.469 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.465+0000 7fc77dd9b640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc7800add50 0x7fc78006bf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.164:42662/0 (socket says 172.21.15.164:42662) 2024-03-23T14:57:26.470 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.465+0000 7fc77dd9b640 1 -- 172.21.15.164:0/852639668 learned_addr learned my addr 172.21.15.164:0/852639668 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:26.470 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.465+0000 7fc77e59c640 1 --2- 172.21.15.164:0/852639668 >> v2:172.21.15.173:3300/0 conn(0x7fc78009def0 0x7fc7800b45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:26.470 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.465+0000 7fc77dd9b640 1 -- 172.21.15.164:0/852639668 >> v2:172.21.15.173:3300/0 conn(0x7fc78009def0 msgr2=0x7fc7800b45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:26.470 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.465+0000 7fc77dd9b640 1 --2- 172.21.15.164:0/852639668 >> v2:172.21.15.173:3300/0 conn(0x7fc78009def0 0x7fc7800b45e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:26.470 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.465+0000 7fc77dd9b640 1 -- 172.21.15.164:0/852639668 >> v1:172.21.15.173:6789/0 conn(0x7fc7800b4b20 legacy=0x7fc78006c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:26.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.465+0000 7fc77dd9b640 1 -- 172.21.15.164:0/852639668 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7800aeca0 con 0x7fc7800add50 2024-03-23T14:57:26.472 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.469+0000 7fc77dd9b640 1 --2- 172.21.15.164:0/852639668 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc7800add50 0x7fc78006bf70 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc770009870 tx=0x7fc77001c440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=606a02209dbcea09 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:26.472 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.469+0000 7fc77d59a640 1 -- 172.21.15.164:0/852639668 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7700ac600 con 0x7fc7800add50 2024-03-23T14:57:26.472 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.469+0000 7fc77d59a640 1 -- 172.21.15.164:0/852639668 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7700a8050 con 0x7fc7800add50 2024-03-23T14:57:26.472 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.469+0000 7fc77d59a640 1 -- 172.21.15.164:0/852639668 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7700ac980 con 0x7fc7800add50 2024-03-23T14:57:26.473 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.469+0000 7fc784f53640 1 -- 172.21.15.164:0/852639668 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc7800add50 msgr2=0x7fc78006bf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:26.473 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.469+0000 7fc784f53640 1 --2- 172.21.15.164:0/852639668 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc7800add50 0x7fc78006bf70 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc770009870 tx=0x7fc77001c440 comp rx=0 tx=0).stop 2024-03-23T14:57:26.473 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.469+0000 7fc784f53640 1 -- 172.21.15.164:0/852639668 shutdown_connections 2024-03-23T14:57:26.473 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.469+0000 7fc784f53640 1 --2- 172.21.15.164:0/852639668 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc7800add50 0x7fc78006bf70 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:26.473 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.469+0000 7fc784f53640 1 --2- 172.21.15.164:0/852639668 >> v2:172.21.15.173:3300/0 conn(0x7fc78009def0 0x7fc7800b45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:26.473 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.469+0000 7fc784f53640 1 -- 172.21.15.164:0/852639668 >> 172.21.15.164:0/852639668 conn(0x7fc7800b8ca0 msgr2=0x7fc7800b90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:26.473 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.469+0000 7fc784f53640 1 -- 172.21.15.164:0/852639668 shutdown_connections 2024-03-23T14:57:26.473 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.469+0000 7fc784f53640 1 -- 172.21.15.164:0/852639668 wait complete. 2024-03-23T14:57:26.474 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.469+0000 7fc784f53640 1 Processor -- start 2024-03-23T14:57:26.474 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.469+0000 7fc784f53640 1 -- start start 2024-03-23T14:57:26.476 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.469+0000 7fc784f53640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc78009def0 0x7fc7800b7f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:26.476 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.469+0000 7fc784f53640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc7800add50 0x7fc780146070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:26.476 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.469+0000 7fc784f53640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc7800b4b20 0x7fc7801e09a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:26.477 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.469+0000 7fc784f53640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc780143010 con 0x7fc7800b4b20 2024-03-23T14:57:26.477 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.469+0000 7fc784f53640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fc780143a60 con 0x7fc7800add50 2024-03-23T14:57:26.477 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.469+0000 7fc784f53640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fc780143200 con 0x7fc78009def0 2024-03-23T14:57:26.477 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.469+0000 7fc77ed9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc7800b4b20 0x7fc7801e09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:26.477 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.469+0000 7fc77ed9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc7800b4b20 0x7fc7801e09a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:49262/0 (socket says 172.21.15.164:49262) 2024-03-23T14:57:26.477 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.469+0000 7fc77ed9d640 1 -- 172.21.15.164:0/3316134628 learned_addr learned my addr 172.21.15.164:0/3316134628 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:26.477 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.469+0000 7fc77e59c640 1 --2- 172.21.15.164:0/3316134628 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc78009def0 0x7fc7800b7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:26.477 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.473+0000 7fc77dd9b640 1 --2- 172.21.15.164:0/3316134628 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc7800add50 0x7fc780146070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:26.477 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.473+0000 7fc77ed9d640 1 -- 172.21.15.164:0/3316134628 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc78009def0 msgr2=0x7fc7800b7f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:26.477 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.473+0000 7fc77ed9d640 1 --2- 172.21.15.164:0/3316134628 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc78009def0 0x7fc7800b7f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:26.477 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.473+0000 7fc77ed9d640 1 -- 172.21.15.164:0/3316134628 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc7800add50 msgr2=0x7fc780146070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:26.477 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.473+0000 7fc77ed9d640 1 --2- 172.21.15.164:0/3316134628 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc7800add50 0x7fc780146070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:26.477 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.473+0000 7fc77ed9d640 1 -- 172.21.15.164:0/3316134628 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7700094d0 con 0x7fc7800b4b20 2024-03-23T14:57:26.477 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.473+0000 7fc77ed9d640 1 --2- 172.21.15.164:0/3316134628 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc7800b4b20 0x7fc7801e09a0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc77800c770 tx=0x7fc77800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:26.477 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.473+0000 7fc76effd640 1 -- 172.21.15.164:0/3316134628 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc778003b80 con 0x7fc7800b4b20 2024-03-23T14:57:26.478 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.473+0000 7fc76effd640 1 -- 172.21.15.164:0/3316134628 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc778003ce0 con 0x7fc7800b4b20 2024-03-23T14:57:26.478 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.473+0000 7fc784f53640 1 -- 172.21.15.164:0/3316134628 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc780143fa0 con 0x7fc7800b4b20 2024-03-23T14:57:26.478 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.473+0000 7fc76effd640 1 -- 172.21.15.164:0/3316134628 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc77800aa90 con 0x7fc7800b4b20 2024-03-23T14:57:26.478 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.473+0000 7fc784f53640 1 -- 172.21.15.164:0/3316134628 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc780146860 con 0x7fc7800b4b20 2024-03-23T14:57:26.478 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.473+0000 7fc784f53640 1 -- 172.21.15.164:0/3316134628 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc780144130 con 0x7fc7800b4b20 2024-03-23T14:57:26.479 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.473+0000 7fc76effd640 1 -- 172.21.15.164:0/3316134628 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7fc77800abf0 con 0x7fc7800b4b20 2024-03-23T14:57:26.479 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.473+0000 7fc76effd640 1 --2- 172.21.15.164:0/3316134628 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fc7600672a0 0x7fc760069760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:26.479 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.473+0000 7fc76effd640 1 -- 172.21.15.164:0/3316134628 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fc778084e10 con 0x7fc7800b4b20 2024-03-23T14:57:26.479 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.473+0000 7fc77e59c640 1 --2- 172.21.15.164:0/3316134628 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fc7600672a0 0x7fc760069760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:26.479 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.473+0000 7fc77e59c640 1 --2- 172.21.15.164:0/3316134628 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fc7600672a0 0x7fc760069760 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc774009ac0 tx=0x7fc774009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:26.482 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.477+0000 7fc76effd640 1 -- 172.21.15.164:0/3316134628 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7fc778088030 con 0x7fc7800b4b20 2024-03-23T14:57:26.638 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.633+0000 7fc784f53640 1 -- 172.21.15.164:0/3316134628 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc7800690a0 con 0x7fc7800b4b20 2024-03-23T14:57:26.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.633+0000 7fc76effd640 1 -- 172.21.15.164:0/3316134628 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16629 (secure 0 0 0) 0x7fc7780519b0 con 0x7fc7800b4b20 2024-03-23T14:57:26.639 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-23T14:57:26.640 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":20,"fsid":"45c7911b-b69d-4e68-907b-fbfeb780aa3b","created":"2024-03-23T14:56:34.367633+0000","modified":"2024-03-23T14:57:00.611265+0000","last_up_change":"2024-03-23T14:56:46.297403+0000","last_in_change":"2024-03-23T14:56:36.629884+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T14:56:47.186157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-23T14:56:56.994057+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-23T14:56:57.953862+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"80abdc62-bd0d-4e60-a554-7ed6fd67cf54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":984716690},{"type":"v1","addr":"172.21.15.164:6801","nonce":984716690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":984716690},{"type":"v1","addr":"172.21.15.164:6803","nonce":984716690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":984716690},{"type":"v1","addr":"172.21.15.164:6807","nonce":984716690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":984716690},{"type":"v1","addr":"172.21.15.164:6805","nonce":984716690}]},"public_addr":"172.21.15.164:6801/984716690","cluster_addr":"172.21.15.164:6803/984716690","heartbeat_back_addr":"172.21.15.164:6807/984716690","heartbeat_front_addr":"172.21.15.164:6805/984716690","state":["exists","up"]},{"osd":1,"uuid":"f9001b06-be4d-4f42-a0e6-bcef2afa0d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":1349657239},{"type":"v1","addr":"172.21.15.164:6817","nonce":1349657239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":1349657239},{"type":"v1","addr":"172.21.15.164:6819","nonce":1349657239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":1349657239},{"type":"v1","addr":"172.21.15.164:6823","nonce":1349657239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":1349657239},{"type":"v1","addr":"172.21.15.164:6821","nonce":1349657239}]},"public_addr":"172.21.15.164:6817/1349657239","cluster_addr":"172.21.15.164:6819/1349657239","heartbeat_back_addr":"172.21.15.164:6823/1349657239","heartbeat_front_addr":"172.21.15.164:6821/1349657239","state":["exists","up"]},{"osd":2,"uuid":"fa0aeb9f-8b54-4831-a188-3e361c7980d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":887861252},{"type":"v1","addr":"172.21.15.164:6809","nonce":887861252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":887861252},{"type":"v1","addr":"172.21.15.164:6811","nonce":887861252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":887861252},{"type":"v1","addr":"172.21.15.164:6815","nonce":887861252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":887861252},{"type":"v1","addr":"172.21.15.164:6813","nonce":887861252}]},"public_addr":"172.21.15.164:6809/887861252","cluster_addr":"172.21.15.164:6811/887861252","heartbeat_back_addr":"172.21.15.164:6815/887861252","heartbeat_front_addr":"172.21.15.164:6813/887861252","state":["exists","up"]},{"osd":3,"uuid":"7e18c957-3091-46d5-8e17-8f3354c1726c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":812540271},{"type":"v1","addr":"172.21.15.164:6825","nonce":812540271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":812540271},{"type":"v1","addr":"172.21.15.164:6827","nonce":812540271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":812540271},{"type":"v1","addr":"172.21.15.164:6831","nonce":812540271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":812540271},{"type":"v1","addr":"172.21.15.164:6829","nonce":812540271}]},"public_addr":"172.21.15.164:6825/812540271","cluster_addr":"172.21.15.164:6827/812540271","heartbeat_back_addr":"172.21.15.164:6831/812540271","heartbeat_front_addr":"172.21.15.164:6829/812540271","state":["exists","up"]},{"osd":4,"uuid":"8e83f315-42bf-4471-9ce8-d81948da0f9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":2656857421},{"type":"v1","addr":"172.21.15.173:6825","nonce":2656857421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2656857421},{"type":"v1","addr":"172.21.15.173:6827","nonce":2656857421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2656857421},{"type":"v1","addr":"172.21.15.173:6831","nonce":2656857421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":2656857421},{"type":"v1","addr":"172.21.15.173:6829","nonce":2656857421}]},"public_addr":"172.21.15.173:6825/2656857421","cluster_addr":"172.21.15.173:6827/2656857421","heartbeat_back_addr":"172.21.15.173:6831/2656857421","heartbeat_front_addr":"172.21.15.173:6829/2656857421","state":["exists","up"]},{"osd":5,"uuid":"a230c872-71f9-40ab-8999-defd0162d4ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":3676483185},{"type":"v1","addr":"172.21.15.173:6801","nonce":3676483185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":3676483185},{"type":"v1","addr":"172.21.15.173:6803","nonce":3676483185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":3676483185},{"type":"v1","addr":"172.21.15.173:6807","nonce":3676483185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":3676483185},{"type":"v1","addr":"172.21.15.173:6805","nonce":3676483185}]},"public_addr":"172.21.15.173:6801/3676483185","cluster_addr":"172.21.15.173:6803/3676483185","heartbeat_back_addr":"172.21.15.173:6807/3676483185","heartbeat_front_addr":"172.21.15.173:6805/3676483185","state":["exists","up"]},{"osd":6,"uuid":"6193bfd7-8432-4f7f-a117-a3fcee21af99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":3631684087},{"type":"v1","addr":"172.21.15.173:6809","nonce":3631684087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":3631684087},{"type":"v1","addr":"172.21.15.173:6811","nonce":3631684087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":3631684087},{"type":"v1","addr":"172.21.15.173:6815","nonce":3631684087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":3631684087},{"type":"v1","addr":"172.21.15.173:6813","nonce":3631684087}]},"public_addr":"172.21.15.173:6809/3631684087","cluster_addr":"172.21.15.173:6811/3631684087","heartbeat_back_addr":"172.21.15.173:6815/3631684087","heartbeat_front_addr":"172.21.15.173:6813/3631684087","state":["exists","up"]},{"osd":7,"uuid":"b0f1b80a-afc4-4538-83b9-f081d2dcb8cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":3261114948},{"type":"v1","addr":"172.21.15.173:6817","nonce":3261114948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":3261114948},{"type":"v1","addr":"172.21.15.173:6819","nonce":3261114948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":3261114948},{"type":"v1","addr":"172.21.15.173:6823","nonce":3261114948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":3261114948},{"type":"v1","addr":"172.21.15.173:6821","nonce":3261114948}]},"public_addr":"172.21.15.173:6817/3261114948","cluster_addr":"172.21.15.173:6819/3261114948","heartbeat_back_addr":"172.21.15.173:6823/3261114948","heartbeat_front_addr":"172.21.15.173:6821/3261114948","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.188390+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.159932+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.191101+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.181776+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.224365+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.177515+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.153631+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.139972+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T14:57:26.641 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.637+0000 7fc784f53640 1 -- 172.21.15.164:0/3316134628 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fc7600672a0 msgr2=0x7fc760069760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:26.641 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.637+0000 7fc784f53640 1 --2- 172.21.15.164:0/3316134628 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fc7600672a0 0x7fc760069760 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc774009ac0 tx=0x7fc774009190 comp rx=0 tx=0).stop 2024-03-23T14:57:26.641 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.637+0000 7fc784f53640 1 -- 172.21.15.164:0/3316134628 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc7800b4b20 msgr2=0x7fc7801e09a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:26.642 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.637+0000 7fc784f53640 1 --2- 172.21.15.164:0/3316134628 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc7800b4b20 0x7fc7801e09a0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc77800c770 tx=0x7fc77800cc30 comp rx=0 tx=0).stop 2024-03-23T14:57:26.642 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.637+0000 7fc784f53640 1 -- 172.21.15.164:0/3316134628 shutdown_connections 2024-03-23T14:57:26.642 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.637+0000 7fc784f53640 1 --2- 172.21.15.164:0/3316134628 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc7800b4b20 0x7fc7801e09a0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:26.642 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.637+0000 7fc784f53640 1 --2- 172.21.15.164:0/3316134628 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fc7600672a0 0x7fc760069760 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:26.642 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.637+0000 7fc784f53640 1 --2- 172.21.15.164:0/3316134628 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc7800add50 0x7fc780146070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:26.642 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.637+0000 7fc784f53640 1 --2- 172.21.15.164:0/3316134628 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc78009def0 0x7fc7800b7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:26.642 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.637+0000 7fc784f53640 1 -- 172.21.15.164:0/3316134628 >> 172.21.15.164:0/3316134628 conn(0x7fc7800b8ca0 msgr2=0x7fc78013e610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:26.642 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.637+0000 7fc784f53640 1 -- 172.21.15.164:0/3316134628 shutdown_connections 2024-03-23T14:57:26.642 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.637+0000 7fc784f53640 1 -- 172.21.15.164:0/3316134628 wait complete. 2024-03-23T14:57:26.658 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T14:56:47.186157+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-23T14:56:56.994057+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-23T14:56:57.953862+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-23T14:57:26.659 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-23T14:57:26.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.773+0000 7f37e714f640 1 Processor -- start 2024-03-23T14:57:26.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.773+0000 7f37e714f640 1 -- start start 2024-03-23T14:57:26.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.773+0000 7f37e714f640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f37e00b0230 0x7f37e00b34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:26.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.773+0000 7f37e714f640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f37e0148cf0 0x7f37e0149180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:26.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.773+0000 7f37e714f640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37e00b51d0 con 0x7f37e00b1c70 2024-03-23T14:57:26.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.773+0000 7f37e714f640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 -- 0x7f37e00b0610 con 0x7f37e00b0230 2024-03-23T14:57:26.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.773+0000 7f37e714f640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f37e00b1500 con 0x7f37e0148cf0 2024-03-23T14:57:26.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.773+0000 7f37e56ec640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f37e00b1c70 0x7f37e00b20a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:42486/0 (socket says 172.21.15.164:42486) 2024-03-23T14:57:26.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.773+0000 7f37e56ec640 1 -- 172.21.15.164:0/514895478 learned_addr learned my addr 172.21.15.164:0/514895478 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-03-23T14:57:26.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.773+0000 7f37dffff640 1 --2- 172.21.15.164:0/514895478 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f37e0148cf0 0x7f37e0149180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:26.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.773+0000 7f37e4eeb640 1 --2- 172.21.15.164:0/514895478 >> v2:172.21.15.173:3300/0 conn(0x7f37e00b0230 0x7f37e00b34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:26.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.773+0000 7f37df7fe640 1 -- 172.21.15.164:0/514895478 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2505497072 0 0) 0x7f37e00b51d0 con 0x7f37e00b1c70 2024-03-23T14:57:26.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.773+0000 7f37df7fe640 1 -- 172.21.15.164:0/514895478 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f37cc003530 con 0x7f37e00b1c70 2024-03-23T14:57:26.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.773+0000 7f37df7fe640 1 -- 172.21.15.164:0/514895478 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1668959490 0 0) 0x7f37cc003530 con 0x7f37e00b1c70 2024-03-23T14:57:26.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.773+0000 7f37df7fe640 1 -- 172.21.15.164:0/514895478 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f37e0148cf0 msgr2=0x7f37e0149180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:26.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.773+0000 7f37df7fe640 1 --2- 172.21.15.164:0/514895478 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f37e0148cf0 0x7f37e0149180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:26.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.773+0000 7f37df7fe640 1 -- 172.21.15.164:0/514895478 >> v2:172.21.15.173:3300/0 conn(0x7f37e00b0230 msgr2=0x7f37e00b34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:26.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.773+0000 7f37df7fe640 1 --2- 172.21.15.164:0/514895478 >> v2:172.21.15.173:3300/0 conn(0x7f37e00b0230 0x7f37e00b34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:26.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.773+0000 7f37df7fe640 1 -- 172.21.15.164:0/514895478 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37e006d250 con 0x7f37e00b1c70 2024-03-23T14:57:26.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.773+0000 7f37df7fe640 1 -- 172.21.15.164:0/514895478 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2526390548 0 0) 0x7f37d0001680 con 0x7f37e00b1c70 2024-03-23T14:57:26.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.773+0000 7f37df7fe640 1 -- 172.21.15.164:0/514895478 >> v1:172.21.15.164:6789/0 conn(0x7f37e00b1c70 legacy=0x7f37e00b20a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:26.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.773+0000 7f37df7fe640 1 --2- 172.21.15.164:0/514895478 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f37cc003980 0x7f37cc003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:26.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.773+0000 7f37df7fe640 1 --2- 172.21.15.164:0/514895478 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f37cc0047d0 0x7f37cc004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:26.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.773+0000 7f37df7fe640 1 --2- 172.21.15.164:0/514895478 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f37e0148cf0 0x7f37cc005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:26.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.773+0000 7f37df7fe640 1 -- 172.21.15.164:0/514895478 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f37e00b0610 con 0x7f37cc003980 2024-03-23T14:57:26.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.773+0000 7f37df7fe640 1 -- 172.21.15.164:0/514895478 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f37e00b1500 con 0x7f37cc0047d0 2024-03-23T14:57:26.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.773+0000 7f37df7fe640 1 -- 172.21.15.164:0/514895478 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f37cc00b9c0 con 0x7f37e0148cf0 2024-03-23T14:57:26.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.777+0000 7f37e4eeb640 1 --2- 172.21.15.164:0/514895478 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f37cc003980 0x7f37cc003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:26.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.777+0000 7f37dffff640 1 --2- 172.21.15.164:0/514895478 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f37cc0047d0 0x7f37cc004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:26.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.777+0000 7f37e56ec640 1 --2- 172.21.15.164:0/514895478 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f37e0148cf0 0x7f37cc005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:26.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.777+0000 7f37e4eeb640 1 -- 172.21.15.164:0/514895478 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f37e0148cf0 msgr2=0x7f37cc005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:26.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.777+0000 7f37e4eeb640 1 --2- 172.21.15.164:0/514895478 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f37e0148cf0 0x7f37cc005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:26.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.777+0000 7f37e4eeb640 1 -- 172.21.15.164:0/514895478 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f37cc0047d0 msgr2=0x7f37cc004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:26.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.777+0000 7f37e4eeb640 1 --2- 172.21.15.164:0/514895478 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f37cc0047d0 0x7f37cc004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:26.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.777+0000 7f37e4eeb640 1 -- 172.21.15.164:0/514895478 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37cc003530 con 0x7f37cc003980 2024-03-23T14:57:26.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.777+0000 7f37e4eeb640 1 --2- 172.21.15.164:0/514895478 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f37cc003980 0x7f37cc003da0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f37d8009fc0 tx=0x7f37d801cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f95de8ee235b325e server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:26.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.777+0000 7f37df7fe640 1 -- 172.21.15.164:0/514895478 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d80a5b70 con 0x7f37cc003980 2024-03-23T14:57:26.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.777+0000 7f37df7fe640 1 -- 172.21.15.164:0/514895478 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37d800f040 con 0x7f37cc003980 2024-03-23T14:57:26.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.777+0000 7f37df7fe640 1 -- 172.21.15.164:0/514895478 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d80ad440 con 0x7f37cc003980 2024-03-23T14:57:26.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.777+0000 7f37e714f640 1 -- 172.21.15.164:0/514895478 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f37cc003980 msgr2=0x7f37cc003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:26.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.777+0000 7f37e714f640 1 --2- 172.21.15.164:0/514895478 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f37cc003980 0x7f37cc003da0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f37d8009fc0 tx=0x7f37d801cc30 comp rx=0 tx=0).stop 2024-03-23T14:57:26.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.777+0000 7f37e56ec640 1 -- 172.21.15.164:0/514895478 reap_dead start 2024-03-23T14:57:26.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.777+0000 7f37e714f640 1 -- 172.21.15.164:0/514895478 shutdown_connections 2024-03-23T14:57:26.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.777+0000 7f37e714f640 1 -- 172.21.15.164:0/514895478 >> 172.21.15.164:0/514895478 conn(0x7f37e00b8ca0 msgr2=0x7f37e00b90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:26.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.777+0000 7f37e714f640 1 -- 172.21.15.164:0/514895478 shutdown_connections 2024-03-23T14:57:26.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.777+0000 7f37e714f640 1 -- 172.21.15.164:0/514895478 wait complete. 2024-03-23T14:57:26.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.781+0000 7f37e714f640 1 Processor -- start 2024-03-23T14:57:26.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.781+0000 7f37e714f640 1 -- start start 2024-03-23T14:57:26.787 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.781+0000 7f37e714f640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f37cc003980 0x7f37e0142780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:26.787 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.781+0000 7f37e714f640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f37e01e1ae0 0x7f37e01402b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:26.787 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.781+0000 7f37e714f640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f37e0145f30 0x7f37e01e0f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:26.787 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.781+0000 7f37e714f640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f37e0141b30 con 0x7f37cc003980 2024-03-23T14:57:26.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.781+0000 7f37e714f640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f37e0142060 con 0x7f37e0145f30 2024-03-23T14:57:26.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.781+0000 7f37e714f640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f37e013eeb0 con 0x7f37e01e1ae0 2024-03-23T14:57:26.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.781+0000 7f37e4eeb640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f37cc003980 0x7f37e0142780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:26.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.781+0000 7f37e4eeb640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f37cc003980 0x7f37e0142780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:49284/0 (socket says 172.21.15.164:49284) 2024-03-23T14:57:26.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.781+0000 7f37e4eeb640 1 -- 172.21.15.164:0/1563553760 learned_addr learned my addr 172.21.15.164:0/1563553760 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:26.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.781+0000 7f37dffff640 1 --2- 172.21.15.164:0/1563553760 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f37e01e1ae0 0x7f37e01402b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:26.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.781+0000 7f37e56ec640 1 --2- 172.21.15.164:0/1563553760 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f37e0145f30 0x7f37e01e0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:26.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.781+0000 7f37e4eeb640 1 -- 172.21.15.164:0/1563553760 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f37e01e1ae0 msgr2=0x7f37e01402b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:26.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.781+0000 7f37e4eeb640 1 --2- 172.21.15.164:0/1563553760 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f37e01e1ae0 0x7f37e01402b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:26.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.781+0000 7f37e4eeb640 1 -- 172.21.15.164:0/1563553760 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f37e0145f30 msgr2=0x7f37e01e0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:26.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.781+0000 7f37e4eeb640 1 --2- 172.21.15.164:0/1563553760 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f37e0145f30 0x7f37e01e0f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:26.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.781+0000 7f37e4eeb640 1 -- 172.21.15.164:0/1563553760 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37e00b0f70 con 0x7f37cc003980 2024-03-23T14:57:26.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.781+0000 7f37e4eeb640 1 --2- 172.21.15.164:0/1563553760 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f37cc003980 0x7f37e0142780 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f37d801c6a0 tx=0x7f37d8009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:26.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.781+0000 7f37dd7fa640 1 -- 172.21.15.164:0/1563553760 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d8003ba0 con 0x7f37cc003980 2024-03-23T14:57:26.788 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.781+0000 7f37dd7fa640 1 -- 172.21.15.164:0/1563553760 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37d8003d00 con 0x7f37cc003980 2024-03-23T14:57:26.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.781+0000 7f37e714f640 1 -- 172.21.15.164:0/1563553760 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37e01433a0 con 0x7f37cc003980 2024-03-23T14:57:26.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.781+0000 7f37dd7fa640 1 -- 172.21.15.164:0/1563553760 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d8004220 con 0x7f37cc003980 2024-03-23T14:57:26.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.781+0000 7f37e714f640 1 -- 172.21.15.164:0/1563553760 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37e0071fe0 con 0x7f37cc003980 2024-03-23T14:57:26.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.781+0000 7f37e714f640 1 -- 172.21.15.164:0/1563553760 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37e00690a0 con 0x7f37cc003980 2024-03-23T14:57:26.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.781+0000 7f37dd7fa640 1 -- 172.21.15.164:0/1563553760 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7f37d80b4040 con 0x7f37cc003980 2024-03-23T14:57:26.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.781+0000 7f37dd7fa640 1 --2- 172.21.15.164:0/1563553760 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f37c00672d0 0x7f37c0069790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:26.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.781+0000 7f37dd7fa640 1 -- 172.21.15.164:0/1563553760 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f37d811f600 con 0x7f37cc003980 2024-03-23T14:57:26.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.781+0000 7f37dffff640 1 --2- 172.21.15.164:0/1563553760 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f37c00672d0 0x7f37c0069790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:26.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.781+0000 7f37dffff640 1 --2- 172.21.15.164:0/1563553760 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f37c00672d0 0x7f37c0069790 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f37e0071a70 tx=0x7f37c8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:26.791 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.785+0000 7f37dd7fa640 1 -- 172.21.15.164:0/1563553760 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7f37d80ec2f0 con 0x7f37cc003980 2024-03-23T14:57:26.950 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.945+0000 7f37e714f640 1 -- 172.21.15.164:0/1563553760 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f37e00b2fa0 con 0x7f37cc003980 2024-03-23T14:57:26.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.945+0000 7f37dd7fa640 1 -- 172.21.15.164:0/1563553760 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f37d80ef9b0 con 0x7f37cc003980 2024-03-23T14:57:26.951 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 1 2024-03-23T14:57:26.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.949+0000 7f37e714f640 1 -- 172.21.15.164:0/1563553760 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f37c00672d0 msgr2=0x7f37c0069790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:26.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.949+0000 7f37e714f640 1 --2- 172.21.15.164:0/1563553760 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f37c00672d0 0x7f37c0069790 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f37e0071a70 tx=0x7f37c8008040 comp rx=0 tx=0).stop 2024-03-23T14:57:26.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.949+0000 7f37e714f640 1 -- 172.21.15.164:0/1563553760 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f37cc003980 msgr2=0x7f37e0142780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:26.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.949+0000 7f37e714f640 1 --2- 172.21.15.164:0/1563553760 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f37cc003980 0x7f37e0142780 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f37d801c6a0 tx=0x7f37d8009750 comp rx=0 tx=0).stop 2024-03-23T14:57:26.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.949+0000 7f37e714f640 1 -- 172.21.15.164:0/1563553760 shutdown_connections 2024-03-23T14:57:26.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.949+0000 7f37e714f640 1 --2- 172.21.15.164:0/1563553760 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f37c00672d0 0x7f37c0069790 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:26.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.949+0000 7f37e714f640 1 --2- 172.21.15.164:0/1563553760 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f37e0145f30 0x7f37e01e0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:26.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.949+0000 7f37e714f640 1 --2- 172.21.15.164:0/1563553760 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f37e01e1ae0 0x7f37e01402b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:26.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.949+0000 7f37e714f640 1 --2- 172.21.15.164:0/1563553760 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f37cc003980 0x7f37e0142780 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:26.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.949+0000 7f37e714f640 1 -- 172.21.15.164:0/1563553760 >> 172.21.15.164:0/1563553760 conn(0x7f37e00b8ca0 msgr2=0x7f37e00b90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:26.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.949+0000 7f37e714f640 1 -- 172.21.15.164:0/1563553760 shutdown_connections 2024-03-23T14:57:26.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:26.949+0000 7f37e714f640 1 -- 172.21.15.164:0/1563553760 wait complete. 2024-03-23T14:57:26.970 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-23T14:57:27.088 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.085+0000 7fe795afe640 1 Processor -- start 2024-03-23T14:57:27.088 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.085+0000 7fe795afe640 1 -- start start 2024-03-23T14:57:27.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.085+0000 7fe795afe640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fe7900add50 0x7fe790147770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:27.090 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.085+0000 7fe795afe640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe790147cb0 0x7fe7901480e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:27.090 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.085+0000 7fe795afe640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe79006d650 con 0x7fe79009def0 2024-03-23T14:57:27.090 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.085+0000 7fe795afe640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fe790148620 con 0x7fe7900add50 2024-03-23T14:57:27.090 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.085+0000 7fe795afe640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fe7900af4a0 con 0x7fe790147cb0 2024-03-23T14:57:27.090 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.085+0000 7fe78effd640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fe7900add50 0x7fe790147770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:27.090 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.085+0000 7fe78f7fe640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fe79009def0 0x7fe7900b45e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:42498/0 (socket says 172.21.15.164:42498) 2024-03-23T14:57:27.090 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.085+0000 7fe78f7fe640 1 -- 172.21.15.164:0/3983989247 learned_addr learned my addr 172.21.15.164:0/3983989247 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-03-23T14:57:27.090 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.085+0000 7fe78ffff640 1 --2- 172.21.15.164:0/3983989247 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe790147cb0 0x7fe7901480e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:27.090 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.085+0000 7fe78effd640 1 -- 172.21.15.164:0/3983989247 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe790147cb0 msgr2=0x7fe7901480e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:27.090 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.085+0000 7fe78effd640 1 --2- 172.21.15.164:0/3983989247 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe790147cb0 0x7fe7901480e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:27.090 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.085+0000 7fe78effd640 1 -- 172.21.15.164:0/3983989247 >> v1:172.21.15.164:6789/0 conn(0x7fe79009def0 legacy=0x7fe7900b45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:27.090 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.085+0000 7fe78effd640 1 -- 172.21.15.164:0/3983989247 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe79009e2d0 con 0x7fe7900add50 2024-03-23T14:57:27.092 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.089+0000 7fe78effd640 1 --2- 172.21.15.164:0/3983989247 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe7900add50 0x7fe790147770 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fe7840097c0 tx=0x7fe78401c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76143a567bb8790 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:27.092 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.089+0000 7fe78e7fc640 1 -- 172.21.15.164:0/3983989247 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7840ad680 con 0x7fe7900add50 2024-03-23T14:57:27.092 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.089+0000 7fe78e7fc640 1 -- 172.21.15.164:0/3983989247 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7840ad7e0 con 0x7fe7900add50 2024-03-23T14:57:27.092 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.089+0000 7fe78e7fc640 1 -- 172.21.15.164:0/3983989247 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7840adb70 con 0x7fe7900add50 2024-03-23T14:57:27.093 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.089+0000 7fe795afe640 1 -- 172.21.15.164:0/3983989247 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe7900add50 msgr2=0x7fe790147770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:27.093 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.089+0000 7fe795afe640 1 --2- 172.21.15.164:0/3983989247 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe7900add50 0x7fe790147770 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fe7840097c0 tx=0x7fe78401c380 comp rx=0 tx=0).stop 2024-03-23T14:57:27.093 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.089+0000 7fe795afe640 1 -- 172.21.15.164:0/3983989247 shutdown_connections 2024-03-23T14:57:27.093 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.089+0000 7fe795afe640 1 --2- 172.21.15.164:0/3983989247 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe790147cb0 0x7fe7901480e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:27.094 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.089+0000 7fe795afe640 1 --2- 172.21.15.164:0/3983989247 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe7900add50 0x7fe790147770 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:27.094 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.089+0000 7fe795afe640 1 -- 172.21.15.164:0/3983989247 >> 172.21.15.164:0/3983989247 conn(0x7fe7900b8ca0 msgr2=0x7fe7900b90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:27.094 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.089+0000 7fe795afe640 1 -- 172.21.15.164:0/3983989247 shutdown_connections 2024-03-23T14:57:27.094 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.089+0000 7fe795afe640 1 -- 172.21.15.164:0/3983989247 wait complete. 2024-03-23T14:57:27.096 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.089+0000 7fe795afe640 1 Processor -- start 2024-03-23T14:57:27.096 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.089+0000 7fe795afe640 1 -- start start 2024-03-23T14:57:27.096 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.089+0000 7fe795afe640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe79009def0 0x7fe7901410c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:27.096 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.089+0000 7fe795afe640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe7900add50 0x7fe7901e0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:27.096 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.089+0000 7fe795afe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe790147cb0 0x7fe790133d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:27.096 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.089+0000 7fe795afe640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe79013fe90 con 0x7fe790147cb0 2024-03-23T14:57:27.096 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.089+0000 7fe795afe640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fe790141600 con 0x7fe7900add50 2024-03-23T14:57:27.096 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.089+0000 7fe795afe640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fe790139e10 con 0x7fe79009def0 2024-03-23T14:57:27.096 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.089+0000 7fe78ffff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe790147cb0 0x7fe790133d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:27.096 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.089+0000 7fe78ffff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe790147cb0 0x7fe790133d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:49308/0 (socket says 172.21.15.164:49308) 2024-03-23T14:57:27.096 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.089+0000 7fe78ffff640 1 -- 172.21.15.164:0/3389935076 learned_addr learned my addr 172.21.15.164:0/3389935076 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:27.096 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.089+0000 7fe78f7fe640 1 --2- 172.21.15.164:0/3389935076 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe79009def0 0x7fe7901410c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:27.096 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.089+0000 7fe78effd640 1 --2- 172.21.15.164:0/3389935076 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe7900add50 0x7fe7901e0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:27.097 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.089+0000 7fe78ffff640 1 -- 172.21.15.164:0/3389935076 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe79009def0 msgr2=0x7fe7901410c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:27.097 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.089+0000 7fe78ffff640 1 --2- 172.21.15.164:0/3389935076 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe79009def0 0x7fe7901410c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:27.097 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.089+0000 7fe78ffff640 1 -- 172.21.15.164:0/3389935076 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe7900add50 msgr2=0x7fe7901e0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:27.097 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.089+0000 7fe78ffff640 1 --2- 172.21.15.164:0/3389935076 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe7900add50 0x7fe7901e0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:27.097 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.089+0000 7fe78ffff640 1 -- 172.21.15.164:0/3389935076 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe784009420 con 0x7fe790147cb0 2024-03-23T14:57:27.097 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.089+0000 7fe78effd640 1 --2- 172.21.15.164:0/3389935076 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe7900add50 0x7fe7901e0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T14:57:27.097 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.089+0000 7fe78ffff640 1 --2- 172.21.15.164:0/3389935076 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe790147cb0 0x7fe790133d30 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fe788007e70 tx=0x7fe78800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:27.097 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.089+0000 7fe777fff640 1 -- 172.21.15.164:0/3389935076 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe788014620 con 0x7fe790147cb0 2024-03-23T14:57:27.097 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.089+0000 7fe777fff640 1 -- 172.21.15.164:0/3389935076 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7880147a0 con 0x7fe790147cb0 2024-03-23T14:57:27.097 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.089+0000 7fe795afe640 1 -- 172.21.15.164:0/3389935076 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7901400e0 con 0x7fe790147cb0 2024-03-23T14:57:27.097 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.089+0000 7fe777fff640 1 -- 172.21.15.164:0/3389935076 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe788014b50 con 0x7fe790147cb0 2024-03-23T14:57:27.097 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.089+0000 7fe795afe640 1 -- 172.21.15.164:0/3389935076 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7901398f0 con 0x7fe790147cb0 2024-03-23T14:57:27.097 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.093+0000 7fe795afe640 1 -- 172.21.15.164:0/3389935076 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe790139ae0 con 0x7fe790147cb0 2024-03-23T14:57:27.098 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.093+0000 7fe777fff640 1 -- 172.21.15.164:0/3389935076 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7fe788014cb0 con 0x7fe790147cb0 2024-03-23T14:57:27.098 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.093+0000 7fe777fff640 1 --2- 172.21.15.164:0/3389935076 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fe77c0672d0 0x7fe77c069790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:27.098 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.093+0000 7fe777fff640 1 -- 172.21.15.164:0/3389935076 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fe788085920 con 0x7fe790147cb0 2024-03-23T14:57:27.098 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.093+0000 7fe78f7fe640 1 --2- 172.21.15.164:0/3389935076 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fe77c0672d0 0x7fe77c069790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:27.098 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.093+0000 7fe78f7fe640 1 --2- 172.21.15.164:0/3389935076 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fe77c0672d0 0x7fe77c069790 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe780000f70 tx=0x7fe780009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:27.101 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.097+0000 7fe777fff640 1 -- 172.21.15.164:0/3389935076 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7fe788052560 con 0x7fe790147cb0 2024-03-23T14:57:27.259 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.253+0000 7fe795afe640 1 -- 172.21.15.164:0/3389935076 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fe7900690a0 con 0x7fe790147cb0 2024-03-23T14:57:27.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.253+0000 7fe777fff640 1 -- 172.21.15.164:0/3389935076 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7fe788055c20 con 0x7fe790147cb0 2024-03-23T14:57:27.260 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 32 2024-03-23T14:57:27.262 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.257+0000 7fe795afe640 1 -- 172.21.15.164:0/3389935076 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fe77c0672d0 msgr2=0x7fe77c069790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:27.262 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.257+0000 7fe795afe640 1 --2- 172.21.15.164:0/3389935076 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fe77c0672d0 0x7fe77c069790 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe780000f70 tx=0x7fe780009040 comp rx=0 tx=0).stop 2024-03-23T14:57:27.262 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.257+0000 7fe795afe640 1 -- 172.21.15.164:0/3389935076 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe790147cb0 msgr2=0x7fe790133d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:27.262 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.257+0000 7fe795afe640 1 --2- 172.21.15.164:0/3389935076 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe790147cb0 0x7fe790133d30 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fe788007e70 tx=0x7fe78800e4b0 comp rx=0 tx=0).stop 2024-03-23T14:57:27.262 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.257+0000 7fe795afe640 1 -- 172.21.15.164:0/3389935076 shutdown_connections 2024-03-23T14:57:27.263 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.257+0000 7fe795afe640 1 --2- 172.21.15.164:0/3389935076 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe790147cb0 0x7fe790133d30 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:27.263 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.257+0000 7fe795afe640 1 --2- 172.21.15.164:0/3389935076 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fe77c0672d0 0x7fe77c069790 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:27.263 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.257+0000 7fe795afe640 1 --2- 172.21.15.164:0/3389935076 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe7900add50 0x7fe7901e0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:27.263 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.257+0000 7fe795afe640 1 --2- 172.21.15.164:0/3389935076 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe79009def0 0x7fe7901410c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:27.263 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.257+0000 7fe795afe640 1 -- 172.21.15.164:0/3389935076 >> 172.21.15.164:0/3389935076 conn(0x7fe7900b8ca0 msgr2=0x7fe79006c6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:27.263 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.257+0000 7fe795afe640 1 -- 172.21.15.164:0/3389935076 shutdown_connections 2024-03-23T14:57:27.263 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.257+0000 7fe795afe640 1 -- 172.21.15.164:0/3389935076 wait complete. 2024-03-23T14:57:27.278 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-23T14:57:27.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.397+0000 7faf54d91640 1 Processor -- start 2024-03-23T14:57:27.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.397+0000 7faf54d91640 1 -- start start 2024-03-23T14:57:27.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.397+0000 7faf54d91640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7faf500b82b0 0x7faf5014acf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:27.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.397+0000 7faf54d91640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf5006ca90 con 0x7faf500b5790 2024-03-23T14:57:27.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.397+0000 7faf54d91640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf5006c7b0 con 0x7faf500b5b70 2024-03-23T14:57:27.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.397+0000 7faf54d91640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7faf500b7110 con 0x7faf500b82b0 2024-03-23T14:57:27.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.397+0000 7faf4e59c640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7faf500b5790 0x7faf50141320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:42506/0 (socket says 172.21.15.164:42506) 2024-03-23T14:57:27.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.397+0000 7faf4e59c640 1 -- 172.21.15.164:0/411795165 learned_addr learned my addr 172.21.15.164:0/411795165 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-03-23T14:57:27.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.397+0000 7faf4dd9b640 1 --2- 172.21.15.164:0/411795165 >> v2:172.21.15.164:3300/0 conn(0x7faf500b82b0 0x7faf5014acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:27.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.397+0000 7faf4d59a640 1 -- 172.21.15.164:0/411795165 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2456140947 0 0) 0x7faf5006ca90 con 0x7faf500b5790 2024-03-23T14:57:27.402 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.397+0000 7faf4d59a640 1 -- 172.21.15.164:0/411795165 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faf38003530 con 0x7faf500b5790 2024-03-23T14:57:27.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.397+0000 7faf4dd9b640 1 -- 172.21.15.164:0/411795165 >> v1:172.21.15.173:6789/0 conn(0x7faf500b5b70 legacy=0x7faf50151cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:27.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.397+0000 7faf4dd9b640 1 -- 172.21.15.164:0/411795165 >> v1:172.21.15.164:6789/0 conn(0x7faf500b5790 legacy=0x7faf50141320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:27.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.397+0000 7faf4dd9b640 1 -- 172.21.15.164:0/411795165 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf5006ceb0 con 0x7faf500b82b0 2024-03-23T14:57:27.404 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.401+0000 7faf4dd9b640 1 --2- 172.21.15.164:0/411795165 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf500b82b0 0x7faf5014acf0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7faf440097c0 tx=0x7faf4401c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d118fc64ba63c985 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:27.404 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.401+0000 7faf4d59a640 1 -- 172.21.15.164:0/411795165 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf440ad660 con 0x7faf500b82b0 2024-03-23T14:57:27.404 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.401+0000 7faf4d59a640 1 -- 172.21.15.164:0/411795165 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf440ad7c0 con 0x7faf500b82b0 2024-03-23T14:57:27.404 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.401+0000 7faf4d59a640 1 -- 172.21.15.164:0/411795165 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf440adb50 con 0x7faf500b82b0 2024-03-23T14:57:27.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.401+0000 7faf54d91640 1 -- 172.21.15.164:0/411795165 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf500b82b0 msgr2=0x7faf5014acf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:27.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.401+0000 7faf54d91640 1 --2- 172.21.15.164:0/411795165 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf500b82b0 0x7faf5014acf0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7faf440097c0 tx=0x7faf4401c380 comp rx=0 tx=0).stop 2024-03-23T14:57:27.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.401+0000 7faf54d91640 1 -- 172.21.15.164:0/411795165 shutdown_connections 2024-03-23T14:57:27.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.401+0000 7faf54d91640 1 --2- 172.21.15.164:0/411795165 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf500b82b0 0x7faf5014acf0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:27.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.401+0000 7faf54d91640 1 -- 172.21.15.164:0/411795165 >> 172.21.15.164:0/411795165 conn(0x7faf500b7630 msgr2=0x7faf500b7a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:27.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.401+0000 7faf54d91640 1 -- 172.21.15.164:0/411795165 shutdown_connections 2024-03-23T14:57:27.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.401+0000 7faf54d91640 1 -- 172.21.15.164:0/411795165 wait complete. 2024-03-23T14:57:27.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.401+0000 7faf54d91640 1 Processor -- start 2024-03-23T14:57:27.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.401+0000 7faf54d91640 1 -- start start 2024-03-23T14:57:27.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.401+0000 7faf54d91640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7faf500b5790 0x7faf500ba080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:27.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.401+0000 7faf54d91640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faf500b5b70 0x7faf501e41d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:27.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.401+0000 7faf54d91640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf500b4be0 0x7faf501e0990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:27.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.401+0000 7faf54d91640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7faf5006d220 con 0x7faf500b4be0 2024-03-23T14:57:27.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.401+0000 7faf54d91640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7faf5006c5d0 con 0x7faf500b5b70 2024-03-23T14:57:27.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.401+0000 7faf54d91640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7faf501e4710 con 0x7faf500b5790 2024-03-23T14:57:27.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.401+0000 7faf4ed9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf500b4be0 0x7faf501e0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:27.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.401+0000 7faf4ed9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf500b4be0 0x7faf501e0990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:49332/0 (socket says 172.21.15.164:49332) 2024-03-23T14:57:27.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.401+0000 7faf4ed9d640 1 -- 172.21.15.164:0/3757467956 learned_addr learned my addr 172.21.15.164:0/3757467956 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:27.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.401+0000 7faf4e59c640 1 --2- 172.21.15.164:0/3757467956 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7faf500b5790 0x7faf500ba080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:27.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.401+0000 7faf4dd9b640 1 --2- 172.21.15.164:0/3757467956 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faf500b5b70 0x7faf501e41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:27.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.401+0000 7faf4ed9d640 1 -- 172.21.15.164:0/3757467956 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7faf500b5790 msgr2=0x7faf500ba080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:27.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.401+0000 7faf4ed9d640 1 --2- 172.21.15.164:0/3757467956 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7faf500b5790 0x7faf500ba080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:27.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.401+0000 7faf4ed9d640 1 -- 172.21.15.164:0/3757467956 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faf500b5b70 msgr2=0x7faf501e41d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:27.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.401+0000 7faf4ed9d640 1 --2- 172.21.15.164:0/3757467956 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faf500b5b70 0x7faf501e41d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:27.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.401+0000 7faf4ed9d640 1 -- 172.21.15.164:0/3757467956 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf44009420 con 0x7faf500b4be0 2024-03-23T14:57:27.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.401+0000 7faf4ed9d640 1 --2- 172.21.15.164:0/3757467956 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf500b4be0 0x7faf501e0990 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7faf4800c7a0 tx=0x7faf4800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:27.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.401+0000 7faf3effd640 1 -- 172.21.15.164:0/3757467956 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf48003b80 con 0x7faf500b4be0 2024-03-23T14:57:27.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.405+0000 7faf3effd640 1 -- 172.21.15.164:0/3757467956 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf48003ce0 con 0x7faf500b4be0 2024-03-23T14:57:27.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.405+0000 7faf3effd640 1 -- 172.21.15.164:0/3757467956 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf4800aa30 con 0x7faf500b4be0 2024-03-23T14:57:27.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.405+0000 7faf54d91640 1 -- 172.21.15.164:0/3757467956 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf501e39d0 con 0x7faf500b4be0 2024-03-23T14:57:27.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.405+0000 7faf54d91640 1 -- 172.21.15.164:0/3757467956 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf500b4750 con 0x7faf500b4be0 2024-03-23T14:57:27.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.405+0000 7faf54d91640 1 -- 172.21.15.164:0/3757467956 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf500b5060 con 0x7faf500b4be0 2024-03-23T14:57:27.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.405+0000 7faf3effd640 1 -- 172.21.15.164:0/3757467956 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7faf4800ab90 con 0x7faf500b4be0 2024-03-23T14:57:27.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.405+0000 7faf3effd640 1 --2- 172.21.15.164:0/3757467956 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7faf300672a0 0x7faf30069760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:27.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.405+0000 7faf4e59c640 1 --2- 172.21.15.164:0/3757467956 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7faf300672a0 0x7faf30069760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:27.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.405+0000 7faf3effd640 1 -- 172.21.15.164:0/3757467956 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7faf48084db0 con 0x7faf500b4be0 2024-03-23T14:57:27.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.405+0000 7faf4e59c640 1 --2- 172.21.15.164:0/3757467956 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7faf300672a0 0x7faf30069760 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7faf400011f0 tx=0x7faf40009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:27.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.409+0000 7faf3effd640 1 -- 172.21.15.164:0/3757467956 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7faf500b5060 con 0x7faf500b4be0 2024-03-23T14:57:27.570 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.565+0000 7faf54d91640 1 -- 172.21.15.164:0/3757467956 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7faf500b5060 con 0x7faf500b4be0 2024-03-23T14:57:27.570 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.565+0000 7faf3effd640 1 -- 172.21.15.164:0/3757467956 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7faf500b5060 con 0x7faf500b4be0 2024-03-23T14:57:27.571 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 64 2024-03-23T14:57:27.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.569+0000 7faf54d91640 1 -- 172.21.15.164:0/3757467956 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7faf300672a0 msgr2=0x7faf30069760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:27.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.569+0000 7faf54d91640 1 --2- 172.21.15.164:0/3757467956 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7faf300672a0 0x7faf30069760 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7faf400011f0 tx=0x7faf40009040 comp rx=0 tx=0).stop 2024-03-23T14:57:27.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.569+0000 7faf54d91640 1 -- 172.21.15.164:0/3757467956 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf500b4be0 msgr2=0x7faf501e0990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:27.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.569+0000 7faf54d91640 1 --2- 172.21.15.164:0/3757467956 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf500b4be0 0x7faf501e0990 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7faf4800c7a0 tx=0x7faf4800cc60 comp rx=0 tx=0).stop 2024-03-23T14:57:27.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.569+0000 7faf54d91640 1 -- 172.21.15.164:0/3757467956 shutdown_connections 2024-03-23T14:57:27.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.569+0000 7faf54d91640 1 --2- 172.21.15.164:0/3757467956 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf500b4be0 0x7faf501e0990 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:27.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.569+0000 7faf54d91640 1 --2- 172.21.15.164:0/3757467956 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7faf300672a0 0x7faf30069760 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:27.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.569+0000 7faf54d91640 1 --2- 172.21.15.164:0/3757467956 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faf500b5b70 0x7faf501e41d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:27.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.569+0000 7faf54d91640 1 --2- 172.21.15.164:0/3757467956 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7faf500b5790 0x7faf500ba080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:27.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.569+0000 7faf54d91640 1 -- 172.21.15.164:0/3757467956 >> 172.21.15.164:0/3757467956 conn(0x7faf500b7630 msgr2=0x7faf50147440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:27.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.569+0000 7faf54d91640 1 -- 172.21.15.164:0/3757467956 shutdown_connections 2024-03-23T14:57:27.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.569+0000 7faf54d91640 1 -- 172.21.15.164:0/3757467956 wait complete. 2024-03-23T14:57:27.589 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-23T14:57:27.700 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.697+0000 7fb8cd722640 1 Processor -- start 2024-03-23T14:57:27.700 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.697+0000 7fb8cd722640 1 -- start start 2024-03-23T14:57:27.703 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.697+0000 7fb8cd722640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fb8c80b56f0 0x7fb8c814a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:27.703 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.697+0000 7fb8cd722640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8c80b4300 con 0x7fb8c80b4a70 2024-03-23T14:57:27.703 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.697+0000 7fb8cd722640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8c80b45e0 con 0x7fb8c8141640 2024-03-23T14:57:27.703 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.697+0000 7fb8cd722640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 -- 0x7fb8c80b5fa0 con 0x7fb8c80b56f0 2024-03-23T14:57:27.703 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.697+0000 7fb8c67fc640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fb8c80b4a70 0x7fb8c8140f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:42516/0 (socket says 172.21.15.164:42516) 2024-03-23T14:57:27.703 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.697+0000 7fb8c67fc640 1 -- 172.21.15.164:0/1072659018 learned_addr learned my addr 172.21.15.164:0/1072659018 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-03-23T14:57:27.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.697+0000 7fb8c6ffd640 1 --2- 172.21.15.164:0/1072659018 >> v2:172.21.15.173:3300/0 conn(0x7fb8c80b56f0 0x7fb8c814a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:27.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.697+0000 7fb8c5ffb640 1 -- 172.21.15.164:0/1072659018 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1461014519 0 0) 0x7fb8c80b4300 con 0x7fb8c80b4a70 2024-03-23T14:57:27.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.697+0000 7fb8c5ffb640 1 -- 172.21.15.164:0/1072659018 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb8b0003530 con 0x7fb8c80b4a70 2024-03-23T14:57:27.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.697+0000 7fb8c5ffb640 1 -- 172.21.15.164:0/1072659018 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 86138440 0 0) 0x7fb8b0003530 con 0x7fb8c80b4a70 2024-03-23T14:57:27.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.697+0000 7fb8c5ffb640 1 -- 172.21.15.164:0/1072659018 >> v2:172.21.15.173:3300/0 conn(0x7fb8c80b56f0 msgr2=0x7fb8c814a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:27.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.697+0000 7fb8c5ffb640 1 --2- 172.21.15.164:0/1072659018 >> v2:172.21.15.173:3300/0 conn(0x7fb8c80b56f0 0x7fb8c814a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:27.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.697+0000 7fb8c5ffb640 1 -- 172.21.15.164:0/1072659018 >> v1:172.21.15.173:6789/0 conn(0x7fb8c8141640 legacy=0x7fb8c8151c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:27.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.697+0000 7fb8c5ffb640 1 -- 172.21.15.164:0/1072659018 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8c814ade0 con 0x7fb8c80b4a70 2024-03-23T14:57:27.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.697+0000 7fb8c5ffb640 1 -- 172.21.15.164:0/1072659018 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2526390548 0 0) 0x7fb8bc001680 con 0x7fb8c80b4a70 2024-03-23T14:57:27.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.697+0000 7fb8c5ffb640 1 -- 172.21.15.164:0/1072659018 >> v1:172.21.15.164:6789/0 conn(0x7fb8c80b4a70 legacy=0x7fb8c8140f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:27.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.697+0000 7fb8c5ffb640 1 --2- 172.21.15.164:0/1072659018 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb8b0003980 0x7fb8b0003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:27.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.697+0000 7fb8c5ffb640 1 --2- 172.21.15.164:0/1072659018 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb8b00047d0 0x7fb8b0004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:27.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.697+0000 7fb8c5ffb640 1 --2- 172.21.15.164:0/1072659018 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb8b0005190 0x7fb8b000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:27.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.697+0000 7fb8c5ffb640 1 -- 172.21.15.164:0/1072659018 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb8b000c000 con 0x7fb8b0003980 2024-03-23T14:57:27.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.697+0000 7fb8c5ffb640 1 -- 172.21.15.164:0/1072659018 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fb8b000c170 con 0x7fb8b00047d0 2024-03-23T14:57:27.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.697+0000 7fb8c5ffb640 1 -- 172.21.15.164:0/1072659018 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fb8b000c330 con 0x7fb8b0005190 2024-03-23T14:57:27.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.697+0000 7fb8c6ffd640 1 --2- 172.21.15.164:0/1072659018 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb8b0003980 0x7fb8b0003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:27.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.697+0000 7fb8c67fc640 1 --2- 172.21.15.164:0/1072659018 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb8b00047d0 0x7fb8b0004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:27.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.697+0000 7fb8c77fe640 1 --2- 172.21.15.164:0/1072659018 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb8b0005190 0x7fb8b000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:27.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.697+0000 7fb8c6ffd640 1 -- 172.21.15.164:0/1072659018 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb8b0005190 msgr2=0x7fb8b000b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:27.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.697+0000 7fb8c6ffd640 1 --2- 172.21.15.164:0/1072659018 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb8b0005190 0x7fb8b000b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:27.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.697+0000 7fb8c6ffd640 1 -- 172.21.15.164:0/1072659018 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb8b00047d0 msgr2=0x7fb8b0004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:27.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.697+0000 7fb8c6ffd640 1 --2- 172.21.15.164:0/1072659018 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb8b00047d0 0x7fb8b0004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:27.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.697+0000 7fb8c6ffd640 1 -- 172.21.15.164:0/1072659018 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8b0003530 con 0x7fb8b0003980 2024-03-23T14:57:27.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.697+0000 7fb8c77fe640 1 -- 172.21.15.164:0/1072659018 reap_dead start 2024-03-23T14:57:27.707 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.701+0000 7fb8c6ffd640 1 --2- 172.21.15.164:0/1072659018 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb8b0003980 0x7fb8b0003da0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fb8b8009ae0 tx=0x7fb8b801ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cad2ee3490e0545 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:27.707 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.701+0000 7fb8c5ffb640 1 -- 172.21.15.164:0/1072659018 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8b80a5520 con 0x7fb8b0003980 2024-03-23T14:57:27.707 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.701+0000 7fb8c5ffb640 1 -- 172.21.15.164:0/1072659018 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8b80a5680 con 0x7fb8b0003980 2024-03-23T14:57:27.707 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.701+0000 7fb8c5ffb640 1 -- 172.21.15.164:0/1072659018 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8b80a5a30 con 0x7fb8b0003980 2024-03-23T14:57:27.707 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.701+0000 7fb8cd722640 1 -- 172.21.15.164:0/1072659018 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb8b0003980 msgr2=0x7fb8b0003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:27.707 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.701+0000 7fb8cd722640 1 --2- 172.21.15.164:0/1072659018 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb8b0003980 0x7fb8b0003da0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fb8b8009ae0 tx=0x7fb8b801ca40 comp rx=0 tx=0).stop 2024-03-23T14:57:27.707 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.701+0000 7fb8cd722640 1 -- 172.21.15.164:0/1072659018 shutdown_connections 2024-03-23T14:57:27.707 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.701+0000 7fb8cd722640 1 --2- 172.21.15.164:0/1072659018 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb8b0003980 0x7fb8b0003da0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:27.707 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.701+0000 7fb8cd722640 1 -- 172.21.15.164:0/1072659018 >> 172.21.15.164:0/1072659018 conn(0x7fb8c802c3c0 msgr2=0x7fb8c802c7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:27.707 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.701+0000 7fb8cd722640 1 -- 172.21.15.164:0/1072659018 shutdown_connections 2024-03-23T14:57:27.707 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.701+0000 7fb8cd722640 1 -- 172.21.15.164:0/1072659018 wait complete. 2024-03-23T14:57:27.707 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.701+0000 7fb8cd722640 1 Processor -- start 2024-03-23T14:57:27.708 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.701+0000 7fb8cd722640 1 -- start start 2024-03-23T14:57:27.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.705+0000 7fb8cd722640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb8b0003980 0x7fb8c8145380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:27.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.705+0000 7fb8cd722640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb8c8146280 0x7fb8c81e3530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:27.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.705+0000 7fb8cd722640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb8c8146700 0x7fb8c81e3a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:27.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.705+0000 7fb8cd722640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb8c8145e60 con 0x7fb8c8146700 2024-03-23T14:57:27.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.705+0000 7fb8cd722640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fb8c8144a70 con 0x7fb8c8146280 2024-03-23T14:57:27.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.705+0000 7fb8cd722640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fb8c8147eb0 con 0x7fb8b0003980 2024-03-23T14:57:27.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.705+0000 7fb8c77fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb8c8146700 0x7fb8c81e3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:27.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.705+0000 7fb8c77fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb8c8146700 0x7fb8c81e3a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:49344/0 (socket says 172.21.15.164:49344) 2024-03-23T14:57:27.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.705+0000 7fb8c77fe640 1 -- 172.21.15.164:0/358904757 learned_addr learned my addr 172.21.15.164:0/358904757 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:27.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.705+0000 7fb8c67fc640 1 --2- 172.21.15.164:0/358904757 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb8c8146280 0x7fb8c81e3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:27.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.705+0000 7fb8c6ffd640 1 --2- 172.21.15.164:0/358904757 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb8b0003980 0x7fb8c8145380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:27.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.705+0000 7fb8c77fe640 1 -- 172.21.15.164:0/358904757 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb8b0003980 msgr2=0x7fb8c8145380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:27.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.705+0000 7fb8c77fe640 1 --2- 172.21.15.164:0/358904757 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb8b0003980 0x7fb8c8145380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:27.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.705+0000 7fb8c77fe640 1 -- 172.21.15.164:0/358904757 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb8c8146280 msgr2=0x7fb8c81e3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:27.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.705+0000 7fb8c77fe640 1 --2- 172.21.15.164:0/358904757 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb8c8146280 0x7fb8c81e3530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:27.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.705+0000 7fb8c77fe640 1 -- 172.21.15.164:0/358904757 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8c81447d0 con 0x7fb8c8146700 2024-03-23T14:57:27.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.705+0000 7fb8c6ffd640 1 --2- 172.21.15.164:0/358904757 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb8b0003980 0x7fb8c8145380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T14:57:27.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.705+0000 7fb8c77fe640 1 --2- 172.21.15.164:0/358904757 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb8c8146700 0x7fb8c81e3a70 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fb8c8073190 tx=0x7fb8c000d9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:27.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.705+0000 7fb8af7fe640 1 -- 172.21.15.164:0/358904757 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8c0003d90 con 0x7fb8c8146700 2024-03-23T14:57:27.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.705+0000 7fb8af7fe640 1 -- 172.21.15.164:0/358904757 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8c0005070 con 0x7fb8c8146700 2024-03-23T14:57:27.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.705+0000 7fb8cd722640 1 -- 172.21.15.164:0/358904757 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8c8143920 con 0x7fb8c8146700 2024-03-23T14:57:27.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.705+0000 7fb8af7fe640 1 -- 172.21.15.164:0/358904757 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8c0005400 con 0x7fb8c8146700 2024-03-23T14:57:27.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.705+0000 7fb8cd722640 1 -- 172.21.15.164:0/358904757 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8c8147a20 con 0x7fb8c8146700 2024-03-23T14:57:27.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.705+0000 7fb8cd722640 1 -- 172.21.15.164:0/358904757 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8c8145fa0 con 0x7fb8c8146700 2024-03-23T14:57:27.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.705+0000 7fb8af7fe640 1 -- 172.21.15.164:0/358904757 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7fb8c0005560 con 0x7fb8c8146700 2024-03-23T14:57:27.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.705+0000 7fb8af7fe640 1 --2- 172.21.15.164:0/358904757 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fb8b40672a0 0x7fb8b4069760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:27.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.705+0000 7fb8af7fe640 1 -- 172.21.15.164:0/358904757 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fb8c00847e0 con 0x7fb8c8146700 2024-03-23T14:57:27.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.705+0000 7fb8c6ffd640 1 --2- 172.21.15.164:0/358904757 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fb8b40672a0 0x7fb8b4069760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:27.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.705+0000 7fb8c6ffd640 1 --2- 172.21.15.164:0/358904757 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fb8b40672a0 0x7fb8b4069760 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb8b80af000 tx=0x7fb8b8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:27.714 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.709+0000 7fb8af7fe640 1 -- 172.21.15.164:0/358904757 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7fb8c8145fa0 con 0x7fb8c8146700 2024-03-23T14:57:27.879 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.873+0000 7fb8cd722640 1 -- 172.21.15.164:0/358904757 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fb8c8145fa0 con 0x7fb8c8146700 2024-03-23T14:57:27.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.877+0000 7fb8af7fe640 1 -- 172.21.15.164:0/358904757 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) ==== 80+0+0 (secure 0 0 0) 0x7fb8c8145fa0 con 0x7fb8c8146700 2024-03-23T14:57:27.984 INFO:teuthology.orchestra.run.smithi164.stderr:listed 0 entries 2024-03-23T14:57:27.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.877+0000 7fb8cd722640 1 -- 172.21.15.164:0/358904757 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fb8b40672a0 msgr2=0x7fb8b4069760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:27.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.877+0000 7fb8cd722640 1 --2- 172.21.15.164:0/358904757 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fb8b40672a0 0x7fb8b4069760 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb8b80af000 tx=0x7fb8b8008040 comp rx=0 tx=0).stop 2024-03-23T14:57:27.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.877+0000 7fb8cd722640 1 -- 172.21.15.164:0/358904757 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb8c8146700 msgr2=0x7fb8c81e3a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:27.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.877+0000 7fb8cd722640 1 --2- 172.21.15.164:0/358904757 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb8c8146700 0x7fb8c81e3a70 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fb8c8073190 tx=0x7fb8c000d9d0 comp rx=0 tx=0).stop 2024-03-23T14:57:27.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.877+0000 7fb8cd722640 1 -- 172.21.15.164:0/358904757 shutdown_connections 2024-03-23T14:57:27.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.877+0000 7fb8cd722640 1 --2- 172.21.15.164:0/358904757 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb8c8146700 0x7fb8c81e3a70 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:27.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.877+0000 7fb8cd722640 1 --2- 172.21.15.164:0/358904757 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fb8b40672a0 0x7fb8b4069760 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:27.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.877+0000 7fb8cd722640 1 --2- 172.21.15.164:0/358904757 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb8c8146280 0x7fb8c81e3530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:27.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.877+0000 7fb8cd722640 1 --2- 172.21.15.164:0/358904757 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb8b0003980 0x7fb8c8145380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:27.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.877+0000 7fb8cd722640 1 -- 172.21.15.164:0/358904757 >> 172.21.15.164:0/358904757 conn(0x7fb8c802c3c0 msgr2=0x7fb8c80bb910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:27.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.877+0000 7fb8cd722640 1 -- 172.21.15.164:0/358904757 shutdown_connections 2024-03-23T14:57:27.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:27.877+0000 7fb8cd722640 1 -- 172.21.15.164:0/358904757 wait complete. 2024-03-23T14:57:27.985 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi164.front.sepia.ceph.com... 2024-03-23T14:57:27.985 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T14:57:27.985 DEBUG:teuthology.orchestra.run.smithi164:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-03-23T14:57:28.009 INFO:teuthology.orchestra.run:waiting for 300 2024-03-23T14:57:28.009 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi164.front.sepia.ceph.com 2024-03-23T14:57:28.009 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T14:57:28.009 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-23T14:57:28.061 INFO:tasks.cephfs.cephfs_test_case:[] 2024-03-23T14:57:28.062 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T14:57:28.224 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.221+0000 7fcac8d1d640 1 Processor -- start 2024-03-23T14:57:28.224 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.221+0000 7fcac8d1d640 1 -- start start 2024-03-23T14:57:28.225 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.221+0000 7fcac8d1d640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcac409def0 0x7fcac40ae620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:28.225 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.221+0000 7fcac8d1d640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fcac40b4e60 0x7fcac414e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:28.225 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.221+0000 7fcac8d1d640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcac40b5270 con 0x7fcac40b4a00 2024-03-23T14:57:28.226 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.221+0000 7fcac8d1d640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fcac4146de0 con 0x7fcac40b4e60 2024-03-23T14:57:28.226 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.221+0000 7fcac8d1d640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fcac409e2d0 con 0x7fcac409def0 2024-03-23T14:57:28.226 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.221+0000 7fcac2d9d640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fcac40b4e60 0x7fcac414e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:28.226 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.221+0000 7fcac2d9d640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fcac40b4e60 0x7fcac414e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:49354/0 (socket says 172.21.15.164:49354) 2024-03-23T14:57:28.226 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.221+0000 7fcac2d9d640 1 -- 172.21.15.164:0/2368567007 learned_addr learned my addr 172.21.15.164:0/2368567007 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:28.226 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.221+0000 7fcac259c640 1 --2- 172.21.15.164:0/2368567007 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcac409def0 0x7fcac40ae620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:28.226 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.221+0000 7fcac2d9d640 1 -- 172.21.15.164:0/2368567007 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcac409def0 msgr2=0x7fcac40ae620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:28.226 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.221+0000 7fcac2d9d640 1 --2- 172.21.15.164:0/2368567007 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcac409def0 0x7fcac40ae620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:28.226 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.221+0000 7fcac2d9d640 1 -- 172.21.15.164:0/2368567007 >> v1:172.21.15.173:6789/0 conn(0x7fcac40b4a00 legacy=0x7fcac4147420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:28.226 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.221+0000 7fcac2d9d640 1 -- 172.21.15.164:0/2368567007 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcac406cb70 con 0x7fcac40b4e60 2024-03-23T14:57:28.226 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.221+0000 7fcac259c640 1 --2- 172.21.15.164:0/2368567007 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcac409def0 0x7fcac40ae620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T14:57:28.227 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.221+0000 7fcac2d9d640 1 --2- 172.21.15.164:0/2368567007 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcac40b4e60 0x7fcac414e110 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fcabc009e50 tx=0x7fcabc01da70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f91e3b9b7b419f1e server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:28.227 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.221+0000 7fcac1d9b640 1 -- 172.21.15.164:0/2368567007 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcabc0a67e0 con 0x7fcac40b4e60 2024-03-23T14:57:28.228 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.225+0000 7fcac1d9b640 1 -- 172.21.15.164:0/2368567007 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcabc0a6940 con 0x7fcac40b4e60 2024-03-23T14:57:28.228 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.225+0000 7fcac1d9b640 1 -- 172.21.15.164:0/2368567007 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcabc0a6cd0 con 0x7fcac40b4e60 2024-03-23T14:57:28.228 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.225+0000 7fcac8d1d640 1 -- 172.21.15.164:0/2368567007 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcac40b4e60 msgr2=0x7fcac414e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:28.229 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.225+0000 7fcac8d1d640 1 --2- 172.21.15.164:0/2368567007 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcac40b4e60 0x7fcac414e110 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fcabc009e50 tx=0x7fcabc01da70 comp rx=0 tx=0).stop 2024-03-23T14:57:28.229 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.225+0000 7fcac8d1d640 1 -- 172.21.15.164:0/2368567007 shutdown_connections 2024-03-23T14:57:28.229 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.225+0000 7fcac8d1d640 1 --2- 172.21.15.164:0/2368567007 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcac40b4e60 0x7fcac414e110 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:28.229 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.225+0000 7fcac8d1d640 1 --2- 172.21.15.164:0/2368567007 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcac409def0 0x7fcac40ae620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:28.229 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.225+0000 7fcac8d1d640 1 -- 172.21.15.164:0/2368567007 >> 172.21.15.164:0/2368567007 conn(0x7fcac40b8ca0 msgr2=0x7fcac40b90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:28.229 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.225+0000 7fcac8d1d640 1 -- 172.21.15.164:0/2368567007 shutdown_connections 2024-03-23T14:57:28.229 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.225+0000 7fcac8d1d640 1 -- 172.21.15.164:0/2368567007 wait complete. 2024-03-23T14:57:28.231 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.225+0000 7fcac8d1d640 1 Processor -- start 2024-03-23T14:57:28.231 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.225+0000 7fcac8d1d640 1 -- start start 2024-03-23T14:57:28.231 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.225+0000 7fcac8d1d640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcac409def0 0x7fcac413d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:28.231 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.225+0000 7fcac8d1d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcac40b4a00 0x7fcac4143780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:28.231 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.225+0000 7fcac8d1d640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcac40b4e60 0x7fcac4143f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:28.231 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.225+0000 7fcac8d1d640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fcac40b2300 con 0x7fcac40b4a00 2024-03-23T14:57:28.232 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.225+0000 7fcac8d1d640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fcac4143cc0 con 0x7fcac409def0 2024-03-23T14:57:28.232 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.225+0000 7fcac8d1d640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fcac4143260 con 0x7fcac40b4e60 2024-03-23T14:57:28.232 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.225+0000 7fcab9d9b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcac40b4a00 0x7fcac4143780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:28.232 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.225+0000 7fcab9d9b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcac40b4a00 0x7fcac4143780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:49356/0 (socket says 172.21.15.164:49356) 2024-03-23T14:57:28.232 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.225+0000 7fcab9d9b640 1 -- 172.21.15.164:0/376933352 learned_addr learned my addr 172.21.15.164:0/376933352 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:28.232 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.225+0000 7fcac259c640 1 --2- 172.21.15.164:0/376933352 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcac409def0 0x7fcac413d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:28.232 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.225+0000 7fcac2d9d640 1 --2- 172.21.15.164:0/376933352 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcac40b4e60 0x7fcac4143f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:28.232 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.225+0000 7fcab9d9b640 1 -- 172.21.15.164:0/376933352 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcac40b4e60 msgr2=0x7fcac4143f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:28.232 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.225+0000 7fcab9d9b640 1 --2- 172.21.15.164:0/376933352 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcac40b4e60 0x7fcac4143f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:28.232 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.225+0000 7fcab9d9b640 1 -- 172.21.15.164:0/376933352 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcac409def0 msgr2=0x7fcac413d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:28.232 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.225+0000 7fcab9d9b640 1 --2- 172.21.15.164:0/376933352 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcac409def0 0x7fcac413d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:28.232 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.225+0000 7fcab9d9b640 1 -- 172.21.15.164:0/376933352 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcabc009b30 con 0x7fcac40b4a00 2024-03-23T14:57:28.232 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.225+0000 7fcab9d9b640 1 --2- 172.21.15.164:0/376933352 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcac40b4a00 0x7fcac4143780 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fcaac00c450 tx=0x7fcaac00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:28.232 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.225+0000 7fcabb7fe640 1 -- 172.21.15.164:0/376933352 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaac012830 con 0x7fcac40b4a00 2024-03-23T14:57:28.232 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.225+0000 7fcabb7fe640 1 -- 172.21.15.164:0/376933352 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcaac0129b0 con 0x7fcac40b4a00 2024-03-23T14:57:28.232 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.225+0000 7fcac8d1d640 1 -- 172.21.15.164:0/376933352 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcac40addc0 con 0x7fcac40b4a00 2024-03-23T14:57:28.233 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.225+0000 7fcabb7fe640 1 -- 172.21.15.164:0/376933352 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaac012d60 con 0x7fcac40b4a00 2024-03-23T14:57:28.233 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.225+0000 7fcac8d1d640 1 -- 172.21.15.164:0/376933352 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcac40b1f60 con 0x7fcac40b4a00 2024-03-23T14:57:28.233 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.229+0000 7fcac8d1d640 1 -- 172.21.15.164:0/376933352 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcac40690a0 con 0x7fcac40b4a00 2024-03-23T14:57:28.234 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.229+0000 7fcabb7fe640 1 -- 172.21.15.164:0/376933352 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7fcaac01c460 con 0x7fcac40b4a00 2024-03-23T14:57:28.234 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.229+0000 7fcabb7fe640 1 --2- 172.21.15.164:0/376933352 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fcaa80672a0 0x7fcaa8069760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:28.234 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.229+0000 7fcac259c640 1 --2- 172.21.15.164:0/376933352 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fcaa80672a0 0x7fcaa8069760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:28.234 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.229+0000 7fcabb7fe640 1 -- 172.21.15.164:0/376933352 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fcaac084e10 con 0x7fcac40b4a00 2024-03-23T14:57:28.234 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.229+0000 7fcac259c640 1 --2- 172.21.15.164:0/376933352 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fcaa80672a0 0x7fcaa8069760 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fcab400a9a0 tx=0x7fcab4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:28.237 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.233+0000 7fcabb7fe640 1 -- 172.21.15.164:0/376933352 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7fcaac051a50 con 0x7fcac40b4a00 2024-03-23T14:57:28.425 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.421+0000 7fcac8d1d640 1 -- 172.21.15.164:0/376933352 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcac40b9390 con 0x7fcac40b4a00 2024-03-23T14:57:28.426 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.421+0000 7fcabb7fe640 1 -- 172.21.15.164:0/376933352 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+3533 (secure 0 0 0) 0x7fcaac055110 con 0x7fcac40b4a00 2024-03-23T14:57:28.427 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-23T14:57:28.427 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/3352372536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":3352372536},{"type":"v1","addr":"172.21.15.164:6835","nonce":3352372536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-23T14:56:58.953326+0000","modified":"2024-03-23T14:57:00.999623+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4279":{"gid":4279,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.173:6833/3506402771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":3506402771},{"type":"v1","addr":"172.21.15.173:6833","nonce":3506402771}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4279,"qdb_cluster":[4279]},"id":1}]} 2024-03-23T14:57:28.427 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 6 2024-03-23T14:57:28.429 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.425+0000 7fcac8d1d640 1 -- 172.21.15.164:0/376933352 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fcaa80672a0 msgr2=0x7fcaa8069760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:28.429 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.425+0000 7fcac8d1d640 1 --2- 172.21.15.164:0/376933352 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fcaa80672a0 0x7fcaa8069760 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fcab400a9a0 tx=0x7fcab4008040 comp rx=0 tx=0).stop 2024-03-23T14:57:28.429 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.425+0000 7fcac8d1d640 1 -- 172.21.15.164:0/376933352 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcac40b4a00 msgr2=0x7fcac4143780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:28.429 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.425+0000 7fcac8d1d640 1 --2- 172.21.15.164:0/376933352 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcac40b4a00 0x7fcac4143780 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fcaac00c450 tx=0x7fcaac00c910 comp rx=0 tx=0).stop 2024-03-23T14:57:28.429 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.425+0000 7fcac8d1d640 1 -- 172.21.15.164:0/376933352 shutdown_connections 2024-03-23T14:57:28.429 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.425+0000 7fcac8d1d640 1 --2- 172.21.15.164:0/376933352 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcac40b4e60 0x7fcac4143f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:28.429 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.425+0000 7fcac8d1d640 1 --2- 172.21.15.164:0/376933352 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcac40b4a00 0x7fcac4143780 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:28.429 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.425+0000 7fcac8d1d640 1 --2- 172.21.15.164:0/376933352 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fcaa80672a0 0x7fcaa8069760 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:28.429 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.425+0000 7fcac8d1d640 1 --2- 172.21.15.164:0/376933352 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcac409def0 0x7fcac413d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:28.429 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.425+0000 7fcac8d1d640 1 -- 172.21.15.164:0/376933352 >> 172.21.15.164:0/376933352 conn(0x7fcac40b8ca0 msgr2=0x7fcac40b33d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:28.429 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.425+0000 7fcac8d1d640 1 -- 172.21.15.164:0/376933352 shutdown_connections 2024-03-23T14:57:28.429 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.425+0000 7fcac8d1d640 1 -- 172.21.15.164:0/376933352 wait complete. 2024-03-23T14:57:28.444 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T14:57:28.552 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.545+0000 7f4e3c2e5640 1 Processor -- start 2024-03-23T14:57:28.552 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.549+0000 7f4e3c2e5640 1 -- start start 2024-03-23T14:57:28.552 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.549+0000 7f4e3c2e5640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4e3409def0 0x7f4e340b45e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:28.552 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.549+0000 7f4e3c2e5640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f4e340b4b20 0x7f4e3406c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:28.552 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.549+0000 7f4e3c2e5640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e3406d650 con 0x7f4e340add50 2024-03-23T14:57:28.552 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.549+0000 7f4e3c2e5640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 -- 0x7f4e340af4a0 con 0x7f4e340b4b20 2024-03-23T14:57:28.553 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.549+0000 7f4e3c2e5640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f4e3409e2d0 con 0x7f4e3409def0 2024-03-23T14:57:28.553 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.549+0000 7f4e3a882640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f4e340b4b20 0x7f4e3406c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:28.553 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.549+0000 7f4e39880640 1 --1- >> v1:172.21.15.173:6789/0 conn(0x7f4e340add50 0x7f4e3406bf70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.173:6789/0 says I am v1:172.21.15.164:52980/0 (socket says 172.21.15.164:52980) 2024-03-23T14:57:28.554 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.549+0000 7f4e39880640 1 -- 172.21.15.164:0/4280970252 learned_addr learned my addr 172.21.15.164:0/4280970252 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-03-23T14:57:28.554 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.549+0000 7f4e3a081640 1 --2- 172.21.15.164:0/4280970252 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4e3409def0 0x7f4e340b45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:28.554 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.549+0000 7f4e3907f640 1 -- 172.21.15.164:0/4280970252 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1976037962 0 0) 0x7f4e3406d650 con 0x7f4e340add50 2024-03-23T14:57:28.554 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.549+0000 7f4e3907f640 1 -- 172.21.15.164:0/4280970252 --> v1:172.21.15.173:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4e20003530 con 0x7f4e340add50 2024-03-23T14:57:28.554 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.549+0000 7f4e3a882640 1 -- 172.21.15.164:0/4280970252 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4e3409def0 msgr2=0x7f4e340b45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:28.554 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.549+0000 7f4e3a882640 1 --2- 172.21.15.164:0/4280970252 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4e3409def0 0x7f4e340b45e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:28.554 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.549+0000 7f4e3a882640 1 -- 172.21.15.164:0/4280970252 >> v1:172.21.15.173:6789/0 conn(0x7f4e340add50 legacy=0x7f4e3406bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:28.554 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.549+0000 7f4e3a882640 1 -- 172.21.15.164:0/4280970252 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e340aeca0 con 0x7f4e340b4b20 2024-03-23T14:57:28.555 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.553+0000 7f4e3a882640 1 --2- 172.21.15.164:0/4280970252 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4e340b4b20 0x7f4e3406c680 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f4e2c009d90 tx=0x7f4e2c01c850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=85051f6b92916b90 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:28.556 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.553+0000 7f4e3907f640 1 -- 172.21.15.164:0/4280970252 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e2c0a5870 con 0x7f4e340b4b20 2024-03-23T14:57:28.556 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.553+0000 7f4e3907f640 1 -- 172.21.15.164:0/4280970252 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e2c0a59d0 con 0x7f4e340b4b20 2024-03-23T14:57:28.556 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.553+0000 7f4e3907f640 1 -- 172.21.15.164:0/4280970252 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e2c0a5d40 con 0x7f4e340b4b20 2024-03-23T14:57:28.556 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.553+0000 7f4e3c2e5640 1 -- 172.21.15.164:0/4280970252 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4e340b4b20 msgr2=0x7f4e3406c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:28.557 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.553+0000 7f4e3c2e5640 1 --2- 172.21.15.164:0/4280970252 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4e340b4b20 0x7f4e3406c680 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f4e2c009d90 tx=0x7f4e2c01c850 comp rx=0 tx=0).stop 2024-03-23T14:57:28.557 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.553+0000 7f4e3c2e5640 1 -- 172.21.15.164:0/4280970252 shutdown_connections 2024-03-23T14:57:28.557 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.553+0000 7f4e3c2e5640 1 --2- 172.21.15.164:0/4280970252 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4e340b4b20 0x7f4e3406c680 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:28.557 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.553+0000 7f4e3c2e5640 1 --2- 172.21.15.164:0/4280970252 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4e3409def0 0x7f4e340b45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:28.557 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.553+0000 7f4e3c2e5640 1 -- 172.21.15.164:0/4280970252 >> 172.21.15.164:0/4280970252 conn(0x7f4e340b8ca0 msgr2=0x7f4e340b90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:28.557 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.553+0000 7f4e3c2e5640 1 -- 172.21.15.164:0/4280970252 shutdown_connections 2024-03-23T14:57:28.557 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.553+0000 7f4e3c2e5640 1 -- 172.21.15.164:0/4280970252 wait complete. 2024-03-23T14:57:28.557 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.553+0000 7f4e3c2e5640 1 Processor -- start 2024-03-23T14:57:28.557 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.553+0000 7f4e3c2e5640 1 -- start start 2024-03-23T14:57:28.559 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.553+0000 7f4e3c2e5640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4e3409def0 0x7f4e34140b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:28.559 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.553+0000 7f4e3c2e5640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4e340add50 0x7f4e340b3730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:28.559 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.553+0000 7f4e3c2e5640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4e340b4b20 0x7f4e3413a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:28.559 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.553+0000 7f4e3c2e5640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f4e34140180 con 0x7f4e340add50 2024-03-23T14:57:28.559 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.553+0000 7f4e3c2e5640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f4e3414e860 con 0x7f4e340b4b20 2024-03-23T14:57:28.559 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.553+0000 7f4e3c2e5640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f4e3414e610 con 0x7f4e3409def0 2024-03-23T14:57:28.560 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.553+0000 7f4e39880640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4e340add50 0x7f4e340b3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:28.560 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.553+0000 7f4e39880640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4e340add50 0x7f4e340b3730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:49368/0 (socket says 172.21.15.164:49368) 2024-03-23T14:57:28.560 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.553+0000 7f4e39880640 1 -- 172.21.15.164:0/729242491 learned_addr learned my addr 172.21.15.164:0/729242491 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:28.560 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.553+0000 7f4e3a882640 1 --2- 172.21.15.164:0/729242491 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4e340b4b20 0x7f4e3413a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:28.560 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.553+0000 7f4e3a081640 1 --2- 172.21.15.164:0/729242491 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4e3409def0 0x7f4e34140b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:28.560 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.553+0000 7f4e39880640 1 -- 172.21.15.164:0/729242491 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4e3409def0 msgr2=0x7f4e34140b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:28.560 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.553+0000 7f4e39880640 1 --2- 172.21.15.164:0/729242491 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4e3409def0 0x7f4e34140b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:28.560 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.553+0000 7f4e39880640 1 -- 172.21.15.164:0/729242491 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4e340b4b20 msgr2=0x7f4e3413a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:28.560 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.553+0000 7f4e39880640 1 --2- 172.21.15.164:0/729242491 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4e340b4b20 0x7f4e3413a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:28.560 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.553+0000 7f4e39880640 1 -- 172.21.15.164:0/729242491 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e2c009a70 con 0x7f4e340add50 2024-03-23T14:57:28.560 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.553+0000 7f4e39880640 1 --2- 172.21.15.164:0/729242491 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4e340add50 0x7f4e340b3730 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f4e3000c480 tx=0x7f4e3000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:28.560 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.553+0000 7f4e2affd640 1 -- 172.21.15.164:0/729242491 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e30005cf0 con 0x7f4e340add50 2024-03-23T14:57:28.560 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.553+0000 7f4e2affd640 1 -- 172.21.15.164:0/729242491 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e30005e50 con 0x7f4e340add50 2024-03-23T14:57:28.560 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.553+0000 7f4e3c2e5640 1 -- 172.21.15.164:0/729242491 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e34139f80 con 0x7f4e340add50 2024-03-23T14:57:28.560 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.553+0000 7f4e2affd640 1 -- 172.21.15.164:0/729242491 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e30007290 con 0x7f4e340add50 2024-03-23T14:57:28.560 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.553+0000 7f4e3c2e5640 1 -- 172.21.15.164:0/729242491 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e3413a220 con 0x7f4e340add50 2024-03-23T14:57:28.561 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.557+0000 7f4e3c2e5640 1 -- 172.21.15.164:0/729242491 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e3413a410 con 0x7f4e340add50 2024-03-23T14:57:28.561 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.557+0000 7f4e2affd640 1 -- 172.21.15.164:0/729242491 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7f4e300073f0 con 0x7f4e340add50 2024-03-23T14:57:28.561 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.557+0000 7f4e2affd640 1 --2- 172.21.15.164:0/729242491 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f4e180672a0 0x7f4e18069760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:28.561 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.557+0000 7f4e2affd640 1 -- 172.21.15.164:0/729242491 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f4e300845c0 con 0x7f4e340add50 2024-03-23T14:57:28.561 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.557+0000 7f4e3a081640 1 --2- 172.21.15.164:0/729242491 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f4e180672a0 0x7f4e18069760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:28.562 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.557+0000 7f4e3a081640 1 --2- 172.21.15.164:0/729242491 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f4e180672a0 0x7f4e18069760 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f4e24009a50 tx=0x7f4e24008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:28.565 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.561+0000 7f4e2affd640 1 -- 172.21.15.164:0/729242491 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7f4e30051200 con 0x7f4e340add50 2024-03-23T14:57:28.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.705+0000 7f4e3c2e5640 1 -- 172.21.15.164:0/729242491 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4e340690a0 con 0x7f4e340add50 2024-03-23T14:57:28.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.709+0000 7f4e2affd640 1 -- 172.21.15.164:0/729242491 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16629 (secure 0 0 0) 0x7f4e300548c0 con 0x7f4e340add50 2024-03-23T14:57:28.712 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-23T14:57:28.712 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":20,"fsid":"45c7911b-b69d-4e68-907b-fbfeb780aa3b","created":"2024-03-23T14:56:34.367633+0000","modified":"2024-03-23T14:57:00.611265+0000","last_up_change":"2024-03-23T14:56:46.297403+0000","last_in_change":"2024-03-23T14:56:36.629884+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T14:56:47.186157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-23T14:56:56.994057+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-23T14:56:57.953862+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"80abdc62-bd0d-4e60-a554-7ed6fd67cf54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":984716690},{"type":"v1","addr":"172.21.15.164:6801","nonce":984716690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":984716690},{"type":"v1","addr":"172.21.15.164:6803","nonce":984716690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":984716690},{"type":"v1","addr":"172.21.15.164:6807","nonce":984716690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":984716690},{"type":"v1","addr":"172.21.15.164:6805","nonce":984716690}]},"public_addr":"172.21.15.164:6801/984716690","cluster_addr":"172.21.15.164:6803/984716690","heartbeat_back_addr":"172.21.15.164:6807/984716690","heartbeat_front_addr":"172.21.15.164:6805/984716690","state":["exists","up"]},{"osd":1,"uuid":"f9001b06-be4d-4f42-a0e6-bcef2afa0d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":1349657239},{"type":"v1","addr":"172.21.15.164:6817","nonce":1349657239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":1349657239},{"type":"v1","addr":"172.21.15.164:6819","nonce":1349657239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":1349657239},{"type":"v1","addr":"172.21.15.164:6823","nonce":1349657239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":1349657239},{"type":"v1","addr":"172.21.15.164:6821","nonce":1349657239}]},"public_addr":"172.21.15.164:6817/1349657239","cluster_addr":"172.21.15.164:6819/1349657239","heartbeat_back_addr":"172.21.15.164:6823/1349657239","heartbeat_front_addr":"172.21.15.164:6821/1349657239","state":["exists","up"]},{"osd":2,"uuid":"fa0aeb9f-8b54-4831-a188-3e361c7980d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":887861252},{"type":"v1","addr":"172.21.15.164:6809","nonce":887861252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":887861252},{"type":"v1","addr":"172.21.15.164:6811","nonce":887861252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":887861252},{"type":"v1","addr":"172.21.15.164:6815","nonce":887861252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":887861252},{"type":"v1","addr":"172.21.15.164:6813","nonce":887861252}]},"public_addr":"172.21.15.164:6809/887861252","cluster_addr":"172.21.15.164:6811/887861252","heartbeat_back_addr":"172.21.15.164:6815/887861252","heartbeat_front_addr":"172.21.15.164:6813/887861252","state":["exists","up"]},{"osd":3,"uuid":"7e18c957-3091-46d5-8e17-8f3354c1726c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":812540271},{"type":"v1","addr":"172.21.15.164:6825","nonce":812540271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":812540271},{"type":"v1","addr":"172.21.15.164:6827","nonce":812540271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":812540271},{"type":"v1","addr":"172.21.15.164:6831","nonce":812540271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":812540271},{"type":"v1","addr":"172.21.15.164:6829","nonce":812540271}]},"public_addr":"172.21.15.164:6825/812540271","cluster_addr":"172.21.15.164:6827/812540271","heartbeat_back_addr":"172.21.15.164:6831/812540271","heartbeat_front_addr":"172.21.15.164:6829/812540271","state":["exists","up"]},{"osd":4,"uuid":"8e83f315-42bf-4471-9ce8-d81948da0f9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":2656857421},{"type":"v1","addr":"172.21.15.173:6825","nonce":2656857421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2656857421},{"type":"v1","addr":"172.21.15.173:6827","nonce":2656857421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2656857421},{"type":"v1","addr":"172.21.15.173:6831","nonce":2656857421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":2656857421},{"type":"v1","addr":"172.21.15.173:6829","nonce":2656857421}]},"public_addr":"172.21.15.173:6825/2656857421","cluster_addr":"172.21.15.173:6827/2656857421","heartbeat_back_addr":"172.21.15.173:6831/2656857421","heartbeat_front_addr":"172.21.15.173:6829/2656857421","state":["exists","up"]},{"osd":5,"uuid":"a230c872-71f9-40ab-8999-defd0162d4ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":3676483185},{"type":"v1","addr":"172.21.15.173:6801","nonce":3676483185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":3676483185},{"type":"v1","addr":"172.21.15.173:6803","nonce":3676483185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":3676483185},{"type":"v1","addr":"172.21.15.173:6807","nonce":3676483185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":3676483185},{"type":"v1","addr":"172.21.15.173:6805","nonce":3676483185}]},"public_addr":"172.21.15.173:6801/3676483185","cluster_addr":"172.21.15.173:6803/3676483185","heartbeat_back_addr":"172.21.15.173:6807/3676483185","heartbeat_front_addr":"172.21.15.173:6805/3676483185","state":["exists","up"]},{"osd":6,"uuid":"6193bfd7-8432-4f7f-a117-a3fcee21af99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":3631684087},{"type":"v1","addr":"172.21.15.173:6809","nonce":3631684087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":3631684087},{"type":"v1","addr":"172.21.15.173:6811","nonce":3631684087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":3631684087},{"type":"v1","addr":"172.21.15.173:6815","nonce":3631684087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":3631684087},{"type":"v1","addr":"172.21.15.173:6813","nonce":3631684087}]},"public_addr":"172.21.15.173:6809/3631684087","cluster_addr":"172.21.15.173:6811/3631684087","heartbeat_back_addr":"172.21.15.173:6815/3631684087","heartbeat_front_addr":"172.21.15.173:6813/3631684087","state":["exists","up"]},{"osd":7,"uuid":"b0f1b80a-afc4-4538-83b9-f081d2dcb8cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":3261114948},{"type":"v1","addr":"172.21.15.173:6817","nonce":3261114948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":3261114948},{"type":"v1","addr":"172.21.15.173:6819","nonce":3261114948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":3261114948},{"type":"v1","addr":"172.21.15.173:6823","nonce":3261114948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":3261114948},{"type":"v1","addr":"172.21.15.173:6821","nonce":3261114948}]},"public_addr":"172.21.15.173:6817/3261114948","cluster_addr":"172.21.15.173:6819/3261114948","heartbeat_back_addr":"172.21.15.173:6823/3261114948","heartbeat_front_addr":"172.21.15.173:6821/3261114948","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.188390+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.159932+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.191101+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.181776+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.224365+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.177515+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.153631+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.139972+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T14:57:28.714 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.709+0000 7f4e3c2e5640 1 -- 172.21.15.164:0/729242491 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f4e180672a0 msgr2=0x7f4e18069760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:28.714 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.709+0000 7f4e3c2e5640 1 --2- 172.21.15.164:0/729242491 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f4e180672a0 0x7f4e18069760 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f4e24009a50 tx=0x7f4e24008040 comp rx=0 tx=0).stop 2024-03-23T14:57:28.714 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.709+0000 7f4e3c2e5640 1 -- 172.21.15.164:0/729242491 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4e340add50 msgr2=0x7f4e340b3730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:28.714 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.709+0000 7f4e3c2e5640 1 --2- 172.21.15.164:0/729242491 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4e340add50 0x7f4e340b3730 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f4e3000c480 tx=0x7f4e3000c940 comp rx=0 tx=0).stop 2024-03-23T14:57:28.715 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.709+0000 7f4e3c2e5640 1 -- 172.21.15.164:0/729242491 shutdown_connections 2024-03-23T14:57:28.715 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.709+0000 7f4e3c2e5640 1 --2- 172.21.15.164:0/729242491 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f4e180672a0 0x7f4e18069760 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:28.715 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.709+0000 7f4e3c2e5640 1 --2- 172.21.15.164:0/729242491 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4e340b4b20 0x7f4e3413a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:28.715 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.709+0000 7f4e3c2e5640 1 --2- 172.21.15.164:0/729242491 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4e340add50 0x7f4e340b3730 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:28.715 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.709+0000 7f4e3c2e5640 1 --2- 172.21.15.164:0/729242491 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4e3409def0 0x7f4e34140b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:28.715 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.709+0000 7f4e3c2e5640 1 -- 172.21.15.164:0/729242491 >> 172.21.15.164:0/729242491 conn(0x7f4e340b8ca0 msgr2=0x7f4e341428d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:28.715 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.709+0000 7f4e3c2e5640 1 -- 172.21.15.164:0/729242491 shutdown_connections 2024-03-23T14:57:28.715 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.709+0000 7f4e3c2e5640 1 -- 172.21.15.164:0/729242491 wait complete. 2024-03-23T14:57:28.730 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T14:56:47.186157+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-23T14:56:56.994057+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-23T14:56:57.953862+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-23T14:57:28.730 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-23T14:57:28.808 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-23T14:57:28.810 DEBUG:teuthology.orchestra.run.smithi173:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-23T14:57:28.843 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.837+0000 7fb3686d1640 1 Processor -- start 2024-03-23T14:57:28.843 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.837+0000 7fb3686d1640 1 -- start start 2024-03-23T14:57:28.845 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.841+0000 7fb3686d1640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fb3600b56f0 0x7fb36014a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:28.845 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.841+0000 7fb3686d1640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb3600b4a70 0x7fb360140f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:28.845 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.841+0000 7fb3686d1640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fb360141470 0x7fb3600b75f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:28.845 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.841+0000 7fb3686d1640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fb3600b5fa0 con 0x7fb3600b56f0 2024-03-23T14:57:28.845 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.841+0000 7fb3686d1640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 -- 0x7fb3600b4300 con 0x7fb360141470 2024-03-23T14:57:28.845 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.841+0000 7fb3686d1640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fb3600b45e0 con 0x7fb3600b4a70 2024-03-23T14:57:28.845 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.841+0000 7fb36646d640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fb3600b56f0 0x7fb36014a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:28.845 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.841+0000 7fb36646d640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fb3600b56f0 0x7fb36014a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:49372/0 (socket says 172.21.15.164:49372) 2024-03-23T14:57:28.845 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.841+0000 7fb36646d640 1 -- 172.21.15.164:0/2064257882 learned_addr learned my addr 172.21.15.164:0/2064257882 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:28.845 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.841+0000 7fb365c6c640 1 --2- 172.21.15.164:0/2064257882 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb3600b4a70 0x7fb360140f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:28.845 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.841+0000 7fb366c6e640 1 --2- 172.21.15.164:0/2064257882 >> v2:172.21.15.173:3300/0 conn(0x7fb360141470 0x7fb3600b75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:28.845 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.841+0000 7fb36646d640 1 -- 172.21.15.164:0/2064257882 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb3600b4a70 msgr2=0x7fb360140f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:28.845 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.841+0000 7fb36646d640 1 --2- 172.21.15.164:0/2064257882 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb3600b4a70 0x7fb360140f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:28.846 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.841+0000 7fb36646d640 1 -- 172.21.15.164:0/2064257882 >> v2:172.21.15.173:3300/0 conn(0x7fb360141470 msgr2=0x7fb3600b75f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:28.846 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.841+0000 7fb36646d640 1 --2- 172.21.15.164:0/2064257882 >> v2:172.21.15.173:3300/0 conn(0x7fb360141470 0x7fb3600b75f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:28.846 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.841+0000 7fb36646d640 1 -- 172.21.15.164:0/2064257882 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb36014ade0 con 0x7fb3600b56f0 2024-03-23T14:57:28.846 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.841+0000 7fb365c6c640 1 --2- 172.21.15.164:0/2064257882 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb3600b4a70 0x7fb360140f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T14:57:28.846 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.841+0000 7fb366c6e640 1 --2- 172.21.15.164:0/2064257882 >> v2:172.21.15.173:3300/0 conn(0x7fb360141470 0x7fb3600b75f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T14:57:28.846 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.841+0000 7fb36646d640 1 --2- 172.21.15.164:0/2064257882 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb3600b56f0 0x7fb36014a8a0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fb3600bab00 tx=0x7fb35001c240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77d08307555479d0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:28.847 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.841+0000 7fb36546b640 1 -- 172.21.15.164:0/2064257882 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3500ab600 con 0x7fb3600b56f0 2024-03-23T14:57:28.847 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.841+0000 7fb36546b640 1 -- 172.21.15.164:0/2064257882 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3500a7050 con 0x7fb3600b56f0 2024-03-23T14:57:28.847 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.841+0000 7fb36546b640 1 -- 172.21.15.164:0/2064257882 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3500ab980 con 0x7fb3600b56f0 2024-03-23T14:57:28.848 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.841+0000 7fb3686d1640 1 -- 172.21.15.164:0/2064257882 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb3600b56f0 msgr2=0x7fb36014a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:28.848 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.841+0000 7fb3686d1640 1 --2- 172.21.15.164:0/2064257882 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb3600b56f0 0x7fb36014a8a0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fb3600bab00 tx=0x7fb35001c240 comp rx=0 tx=0).stop 2024-03-23T14:57:28.848 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.841+0000 7fb3686d1640 1 -- 172.21.15.164:0/2064257882 shutdown_connections 2024-03-23T14:57:28.848 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.841+0000 7fb3686d1640 1 --2- 172.21.15.164:0/2064257882 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb3600b4a70 0x7fb360140f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:28.848 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.841+0000 7fb3686d1640 1 --2- 172.21.15.164:0/2064257882 >> v2:172.21.15.173:3300/0 conn(0x7fb360141470 0x7fb3600b75f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:28.848 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.841+0000 7fb3686d1640 1 --2- 172.21.15.164:0/2064257882 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb3600b56f0 0x7fb36014a8a0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:28.848 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.841+0000 7fb3686d1640 1 -- 172.21.15.164:0/2064257882 >> 172.21.15.164:0/2064257882 conn(0x7fb36002c3c0 msgr2=0x7fb36002c7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:28.848 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.841+0000 7fb3686d1640 1 -- 172.21.15.164:0/2064257882 shutdown_connections 2024-03-23T14:57:28.848 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.841+0000 7fb3686d1640 1 -- 172.21.15.164:0/2064257882 wait complete. 2024-03-23T14:57:28.850 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.845+0000 7fb3686d1640 1 Processor -- start 2024-03-23T14:57:28.850 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.845+0000 7fb3686d1640 1 -- start start 2024-03-23T14:57:28.850 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.845+0000 7fb3686d1640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb3600b4a70 0x7fb360143320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:28.850 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.845+0000 7fb3686d1640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb3600b56f0 0x7fb360148bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:28.850 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.845+0000 7fb3686d1640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb360141470 0x7fb360149100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:28.851 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.845+0000 7fb3686d1640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb360148470 con 0x7fb360141470 2024-03-23T14:57:28.851 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.845+0000 7fb3686d1640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fb36014a430 con 0x7fb3600b56f0 2024-03-23T14:57:28.851 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.845+0000 7fb3686d1640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fb360145840 con 0x7fb3600b4a70 2024-03-23T14:57:28.851 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.845+0000 7fb366c6e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb360141470 0x7fb360149100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:28.851 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.845+0000 7fb366c6e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb360141470 0x7fb360149100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:49386/0 (socket says 172.21.15.164:49386) 2024-03-23T14:57:28.851 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.845+0000 7fb366c6e640 1 -- 172.21.15.164:0/3625311131 learned_addr learned my addr 172.21.15.164:0/3625311131 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:28.851 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.845+0000 7fb36646d640 1 --2- 172.21.15.164:0/3625311131 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb3600b4a70 0x7fb360143320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:28.851 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.845+0000 7fb365c6c640 1 --2- 172.21.15.164:0/3625311131 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb3600b56f0 0x7fb360148bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:28.851 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.845+0000 7fb366c6e640 1 -- 172.21.15.164:0/3625311131 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb3600b4a70 msgr2=0x7fb360143320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:28.851 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.845+0000 7fb366c6e640 1 --2- 172.21.15.164:0/3625311131 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb3600b4a70 0x7fb360143320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:28.851 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.845+0000 7fb366c6e640 1 -- 172.21.15.164:0/3625311131 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb3600b56f0 msgr2=0x7fb360148bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:28.851 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.845+0000 7fb366c6e640 1 --2- 172.21.15.164:0/3625311131 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb3600b56f0 0x7fb360148bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:28.851 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.845+0000 7fb366c6e640 1 -- 172.21.15.164:0/3625311131 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3500094d0 con 0x7fb360141470 2024-03-23T14:57:28.851 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.845+0000 7fb366c6e640 1 --2- 172.21.15.164:0/3625311131 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb360141470 0x7fb360149100 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fb358007df0 tx=0x7fb35800d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:28.851 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.845+0000 7fb356ffd640 1 -- 172.21.15.164:0/3625311131 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3580039a0 con 0x7fb360141470 2024-03-23T14:57:28.852 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.845+0000 7fb356ffd640 1 -- 172.21.15.164:0/3625311131 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb35800f040 con 0x7fb360141470 2024-03-23T14:57:28.852 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.845+0000 7fb3686d1640 1 -- 172.21.15.164:0/3625311131 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3601424c0 con 0x7fb360141470 2024-03-23T14:57:28.852 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.845+0000 7fb356ffd640 1 -- 172.21.15.164:0/3625311131 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb358003d40 con 0x7fb360141470 2024-03-23T14:57:28.852 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.845+0000 7fb3686d1640 1 -- 172.21.15.164:0/3625311131 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb360144090 con 0x7fb360141470 2024-03-23T14:57:28.852 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.845+0000 7fb3686d1640 1 -- 172.21.15.164:0/3625311131 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3600690a0 con 0x7fb360141470 2024-03-23T14:57:28.853 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.845+0000 7fb356ffd640 1 -- 172.21.15.164:0/3625311131 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7fb358004030 con 0x7fb360141470 2024-03-23T14:57:28.853 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.849+0000 7fb356ffd640 1 --2- 172.21.15.164:0/3625311131 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fb3440672a0 0x7fb344069760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:28.853 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.849+0000 7fb36646d640 1 --2- 172.21.15.164:0/3625311131 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fb3440672a0 0x7fb344069760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:28.853 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.849+0000 7fb356ffd640 1 -- 172.21.15.164:0/3625311131 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fb358085190 con 0x7fb360141470 2024-03-23T14:57:28.853 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.849+0000 7fb36646d640 1 --2- 172.21.15.164:0/3625311131 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fb3440672a0 0x7fb344069760 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb3500ad000 tx=0x7fb350008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:28.856 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.853+0000 7fb356ffd640 1 -- 172.21.15.164:0/3625311131 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7fb358051e80 con 0x7fb360141470 2024-03-23T14:57:29.003 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.997+0000 7fb3686d1640 1 -- 172.21.15.164:0/3625311131 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb36006b7b0 con 0x7fb360141470 2024-03-23T14:57:29.003 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:28.997+0000 7fb356ffd640 1 -- 172.21.15.164:0/3625311131 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7fb358055540 con 0x7fb360141470 2024-03-23T14:57:29.003 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 1 2024-03-23T14:57:29.006 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.001+0000 7fb3686d1640 1 -- 172.21.15.164:0/3625311131 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fb3440672a0 msgr2=0x7fb344069760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:29.006 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.001+0000 7fb3686d1640 1 --2- 172.21.15.164:0/3625311131 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fb3440672a0 0x7fb344069760 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb3500ad000 tx=0x7fb350008040 comp rx=0 tx=0).stop 2024-03-23T14:57:29.006 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.001+0000 7fb3686d1640 1 -- 172.21.15.164:0/3625311131 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb360141470 msgr2=0x7fb360149100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:29.006 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.001+0000 7fb3686d1640 1 --2- 172.21.15.164:0/3625311131 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb360141470 0x7fb360149100 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fb358007df0 tx=0x7fb35800d4a0 comp rx=0 tx=0).stop 2024-03-23T14:57:29.006 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.001+0000 7fb3686d1640 1 -- 172.21.15.164:0/3625311131 shutdown_connections 2024-03-23T14:57:29.006 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.001+0000 7fb3686d1640 1 --2- 172.21.15.164:0/3625311131 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb360141470 0x7fb360149100 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:29.006 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.001+0000 7fb3686d1640 1 --2- 172.21.15.164:0/3625311131 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fb3440672a0 0x7fb344069760 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:29.006 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.001+0000 7fb3686d1640 1 --2- 172.21.15.164:0/3625311131 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb3600b56f0 0x7fb360148bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:29.006 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.001+0000 7fb3686d1640 1 --2- 172.21.15.164:0/3625311131 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb3600b4a70 0x7fb360143320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:29.007 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.001+0000 7fb3686d1640 1 -- 172.21.15.164:0/3625311131 >> 172.21.15.164:0/3625311131 conn(0x7fb36002c3c0 msgr2=0x7fb36014b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:29.007 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.001+0000 7fb3686d1640 1 -- 172.21.15.164:0/3625311131 shutdown_connections 2024-03-23T14:57:29.007 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.001+0000 7fb3686d1640 1 -- 172.21.15.164:0/3625311131 wait complete. 2024-03-23T14:57:29.021 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-23T14:57:29.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.141+0000 7f640519f640 1 Processor -- start 2024-03-23T14:57:29.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.141+0000 7f640519f640 1 -- start start 2024-03-23T14:57:29.146 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.141+0000 7f640519f640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f6400147890 0x7f64000b0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:29.146 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.141+0000 7f640519f640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f64000707c0 0x7f6400070c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:29.146 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.141+0000 7f640519f640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64001480c0 con 0x7f640013df90 2024-03-23T14:57:29.146 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.141+0000 7f640519f640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 -- 0x7f64000705c0 con 0x7f6400147890 2024-03-23T14:57:29.146 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.141+0000 7f640519f640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f6400148550 con 0x7f64000707c0 2024-03-23T14:57:29.146 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.141+0000 7f63ff59e640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f640013df90 0x7f64001440c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:42528/0 (socket says 172.21.15.164:42528) 2024-03-23T14:57:29.146 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.141+0000 7f63ff59e640 1 -- 172.21.15.164:0/2742130010 learned_addr learned my addr 172.21.15.164:0/2742130010 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-03-23T14:57:29.146 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.141+0000 7f63fe59c640 1 --2- 172.21.15.164:0/2742130010 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f64000707c0 0x7f6400070c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:29.146 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.141+0000 7f63fed9d640 1 --2- 172.21.15.164:0/2742130010 >> v2:172.21.15.173:3300/0 conn(0x7f6400147890 0x7f64000b0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:29.146 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.141+0000 7f63fdd9b640 1 -- 172.21.15.164:0/2742130010 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3143394464 0 0) 0x7f64001480c0 con 0x7f640013df90 2024-03-23T14:57:29.147 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.141+0000 7f63fdd9b640 1 -- 172.21.15.164:0/2742130010 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f63ec003530 con 0x7f640013df90 2024-03-23T14:57:29.147 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.141+0000 7f63fdd9b640 1 -- 172.21.15.164:0/2742130010 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1814482398 0 0) 0x7f63ec003530 con 0x7f640013df90 2024-03-23T14:57:29.147 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.141+0000 7f63fe59c640 1 -- 172.21.15.164:0/2742130010 >> v2:172.21.15.173:3300/0 conn(0x7f6400147890 msgr2=0x7f64000b0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:29.147 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.141+0000 7f63fe59c640 1 --2- 172.21.15.164:0/2742130010 >> v2:172.21.15.173:3300/0 conn(0x7f6400147890 0x7f64000b0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:29.147 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.141+0000 7f63fe59c640 1 -- 172.21.15.164:0/2742130010 >> v1:172.21.15.164:6789/0 conn(0x7f640013df90 legacy=0x7f64001440c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:29.147 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.141+0000 7f63fe59c640 1 -- 172.21.15.164:0/2742130010 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64000725c0 con 0x7f64000707c0 2024-03-23T14:57:29.148 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.145+0000 7f63fe59c640 1 --2- 172.21.15.164:0/2742130010 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f64000707c0 0x7f6400070c00 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f63e80097c0 tx=0x7f63e801c330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d2229c72741ec2d server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:29.149 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.145+0000 7f63fdd9b640 1 -- 172.21.15.164:0/2742130010 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6400144fe0 con 0x7f64000707c0 2024-03-23T14:57:29.149 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.145+0000 7f63fdd9b640 1 -- 172.21.15.164:0/2742130010 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63e80a7050 con 0x7f64000707c0 2024-03-23T14:57:29.149 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.145+0000 7f63fdd9b640 1 -- 172.21.15.164:0/2742130010 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63f8001a00 con 0x7f64000707c0 2024-03-23T14:57:29.149 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.145+0000 7f640519f640 1 -- 172.21.15.164:0/2742130010 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f64000707c0 msgr2=0x7f6400070c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:29.149 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.145+0000 7f640519f640 1 --2- 172.21.15.164:0/2742130010 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f64000707c0 0x7f6400070c00 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f63e80097c0 tx=0x7f63e801c330 comp rx=0 tx=0).stop 2024-03-23T14:57:29.149 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.145+0000 7f640519f640 1 -- 172.21.15.164:0/2742130010 shutdown_connections 2024-03-23T14:57:29.150 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.145+0000 7f640519f640 1 --2- 172.21.15.164:0/2742130010 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f64000707c0 0x7f6400070c00 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:29.150 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.145+0000 7f640519f640 1 --2- 172.21.15.164:0/2742130010 >> v2:172.21.15.173:3300/0 conn(0x7f6400147890 0x7f64000b0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:29.150 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.145+0000 7f640519f640 1 -- 172.21.15.164:0/2742130010 >> 172.21.15.164:0/2742130010 conn(0x7f64000b7970 msgr2=0x7f64000b7d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:29.150 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.145+0000 7f640519f640 1 -- 172.21.15.164:0/2742130010 shutdown_connections 2024-03-23T14:57:29.150 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.145+0000 7f640519f640 1 -- 172.21.15.164:0/2742130010 wait complete. 2024-03-23T14:57:29.152 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.145+0000 7f640519f640 1 Processor -- start 2024-03-23T14:57:29.152 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.145+0000 7f640519f640 1 -- start start 2024-03-23T14:57:29.152 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.145+0000 7f640519f640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f64000707c0 0x7f64001d70c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:29.152 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.145+0000 7f640519f640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f640013df90 0x7f640013cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:29.152 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.145+0000 7f640519f640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6400147890 0x7f640013f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:29.153 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.145+0000 7f640519f640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f640013bd10 con 0x7f6400147890 2024-03-23T14:57:29.153 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.145+0000 7f640519f640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f64001401d0 con 0x7f64000707c0 2024-03-23T14:57:29.153 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.145+0000 7f640519f640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f64001404b0 con 0x7f640013df90 2024-03-23T14:57:29.153 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.145+0000 7f63ff59e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6400147890 0x7f640013f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:29.153 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.145+0000 7f63ff59e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6400147890 0x7f640013f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:49388/0 (socket says 172.21.15.164:49388) 2024-03-23T14:57:29.153 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.145+0000 7f63ff59e640 1 -- 172.21.15.164:0/848387350 learned_addr learned my addr 172.21.15.164:0/848387350 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:29.153 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.145+0000 7f63fed9d640 1 --2- 172.21.15.164:0/848387350 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f64000707c0 0x7f64001d70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:29.153 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.145+0000 7f63fe59c640 1 --2- 172.21.15.164:0/848387350 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f640013df90 0x7f640013cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:29.153 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.145+0000 7f63ff59e640 1 -- 172.21.15.164:0/848387350 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f640013df90 msgr2=0x7f640013cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:29.153 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.145+0000 7f63ff59e640 1 --2- 172.21.15.164:0/848387350 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f640013df90 0x7f640013cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:29.153 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.145+0000 7f63ff59e640 1 -- 172.21.15.164:0/848387350 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f64000707c0 msgr2=0x7f64001d70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:29.153 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.145+0000 7f63ff59e640 1 --2- 172.21.15.164:0/848387350 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f64000707c0 0x7f64001d70c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:29.153 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.145+0000 7f63ff59e640 1 -- 172.21.15.164:0/848387350 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63e8009420 con 0x7f6400147890 2024-03-23T14:57:29.153 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.145+0000 7f63ff59e640 1 --2- 172.21.15.164:0/848387350 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6400147890 0x7f640013f730 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f63f8001840 tx=0x7f63f800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:29.154 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.145+0000 7f63f77fe640 1 -- 172.21.15.164:0/848387350 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63f8012400 con 0x7f6400147890 2024-03-23T14:57:29.154 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.145+0000 7f63f77fe640 1 -- 172.21.15.164:0/848387350 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63f8012560 con 0x7f6400147890 2024-03-23T14:57:29.154 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.145+0000 7f640519f640 1 -- 172.21.15.164:0/848387350 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f640013d770 con 0x7f6400147890 2024-03-23T14:57:29.154 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.145+0000 7f63f77fe640 1 -- 172.21.15.164:0/848387350 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63f8012910 con 0x7f6400147890 2024-03-23T14:57:29.154 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.145+0000 7f640519f640 1 -- 172.21.15.164:0/848387350 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64001e2d80 con 0x7f6400147890 2024-03-23T14:57:29.154 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.149+0000 7f640519f640 1 -- 172.21.15.164:0/848387350 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64000b5f10 con 0x7f6400147890 2024-03-23T14:57:29.155 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.149+0000 7f63f77fe640 1 -- 172.21.15.164:0/848387350 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7f63f8012a70 con 0x7f6400147890 2024-03-23T14:57:29.155 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.149+0000 7f63f77fe640 1 --2- 172.21.15.164:0/848387350 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f63e00672a0 0x7f63e0069760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:29.155 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.149+0000 7f63fed9d640 1 --2- 172.21.15.164:0/848387350 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f63e00672a0 0x7f63e0069760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:29.155 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.149+0000 7f63f77fe640 1 -- 172.21.15.164:0/848387350 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f63f80857e0 con 0x7f6400147890 2024-03-23T14:57:29.155 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.149+0000 7f63fed9d640 1 --2- 172.21.15.164:0/848387350 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f63e00672a0 0x7f63e0069760 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f640013b660 tx=0x7f63f0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:29.157 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.153+0000 7f63f77fe640 1 -- 172.21.15.164:0/848387350 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7f63f8052380 con 0x7f6400147890 2024-03-23T14:57:29.315 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.309+0000 7f640519f640 1 -- 172.21.15.164:0/848387350 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f64000690a0 con 0x7f6400147890 2024-03-23T14:57:29.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.313+0000 7f63f77fe640 1 -- 172.21.15.164:0/848387350 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f63f8055a40 con 0x7f6400147890 2024-03-23T14:57:29.316 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 32 2024-03-23T14:57:29.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.313+0000 7f640519f640 1 -- 172.21.15.164:0/848387350 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f63e00672a0 msgr2=0x7f63e0069760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:29.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.313+0000 7f640519f640 1 --2- 172.21.15.164:0/848387350 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f63e00672a0 0x7f63e0069760 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f640013b660 tx=0x7f63f0008040 comp rx=0 tx=0).stop 2024-03-23T14:57:29.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.313+0000 7f640519f640 1 -- 172.21.15.164:0/848387350 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6400147890 msgr2=0x7f640013f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:29.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.313+0000 7f640519f640 1 --2- 172.21.15.164:0/848387350 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6400147890 0x7f640013f730 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f63f8001840 tx=0x7f63f800a430 comp rx=0 tx=0).stop 2024-03-23T14:57:29.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.313+0000 7f640519f640 1 -- 172.21.15.164:0/848387350 shutdown_connections 2024-03-23T14:57:29.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.313+0000 7f640519f640 1 --2- 172.21.15.164:0/848387350 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6400147890 0x7f640013f730 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:29.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.313+0000 7f640519f640 1 --2- 172.21.15.164:0/848387350 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f640013df90 0x7f640013cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:29.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.313+0000 7f640519f640 1 --2- 172.21.15.164:0/848387350 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f63e00672a0 0x7f63e0069760 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:29.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.313+0000 7f640519f640 1 --2- 172.21.15.164:0/848387350 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f64000707c0 0x7f64001d70c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:29.320 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.313+0000 7f640519f640 1 -- 172.21.15.164:0/848387350 >> 172.21.15.164:0/848387350 conn(0x7f64000b7970 msgr2=0x7f64000b8be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:29.320 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.313+0000 7f640519f640 1 -- 172.21.15.164:0/848387350 shutdown_connections 2024-03-23T14:57:29.320 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.313+0000 7f640519f640 1 -- 172.21.15.164:0/848387350 wait complete. 2024-03-23T14:57:29.336 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-23T14:57:29.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.441+0000 7f651804e640 1 Processor -- start 2024-03-23T14:57:29.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.441+0000 7f651804e640 1 -- start start 2024-03-23T14:57:29.450 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.445+0000 7f651804e640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f651014eb80 0x7f6510150f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:29.450 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.445+0000 7f651804e640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65100b51d0 con 0x7f6510148cf0 2024-03-23T14:57:29.450 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.445+0000 7f651804e640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65100b0610 con 0x7f65100b0230 2024-03-23T14:57:29.450 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.445+0000 7f651804e640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 -- 0x7f65100b1500 con 0x7f651014eb80 2024-03-23T14:57:29.450 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.445+0000 7f65155e9640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f6510148cf0 0x7f6510149180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:42538/0 (socket says 172.21.15.164:42538) 2024-03-23T14:57:29.450 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.445+0000 7f65155e9640 1 -- 172.21.15.164:0/719623325 learned_addr learned my addr 172.21.15.164:0/719623325 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-03-23T14:57:29.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.445+0000 7f65165eb640 1 --2- 172.21.15.164:0/719623325 >> v2:172.21.15.173:3300/0 conn(0x7f651014eb80 0x7f6510150f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:29.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.445+0000 7f6514de8640 1 -- 172.21.15.164:0/719623325 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3999995892 0 0) 0x7f65100b51d0 con 0x7f6510148cf0 2024-03-23T14:57:29.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.445+0000 7f6514de8640 1 -- 172.21.15.164:0/719623325 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f64fc003530 con 0x7f6510148cf0 2024-03-23T14:57:29.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.445+0000 7f6514de8640 1 -- 172.21.15.164:0/719623325 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2644713845 0 0) 0x7f64fc003530 con 0x7f6510148cf0 2024-03-23T14:57:29.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.445+0000 7f6514de8640 1 -- 172.21.15.164:0/719623325 >> v2:172.21.15.173:3300/0 conn(0x7f651014eb80 msgr2=0x7f6510150f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:29.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.445+0000 7f6514de8640 1 --2- 172.21.15.164:0/719623325 >> v2:172.21.15.173:3300/0 conn(0x7f651014eb80 0x7f6510150f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:29.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.445+0000 7f6514de8640 1 -- 172.21.15.164:0/719623325 >> v1:172.21.15.173:6789/0 conn(0x7f65100b0230 legacy=0x7f65100b34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:29.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.445+0000 7f6514de8640 1 -- 172.21.15.164:0/719623325 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f651006d250 con 0x7f6510148cf0 2024-03-23T14:57:29.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.445+0000 7f6514de8640 1 -- 172.21.15.164:0/719623325 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2526390548 0 0) 0x7f650c001680 con 0x7f6510148cf0 2024-03-23T14:57:29.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.445+0000 7f6514de8640 1 -- 172.21.15.164:0/719623325 >> v1:172.21.15.164:6789/0 conn(0x7f6510148cf0 legacy=0x7f6510149180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:29.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.445+0000 7f6514de8640 1 --2- 172.21.15.164:0/719623325 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f64fc003980 0x7f64fc003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:29.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.445+0000 7f6514de8640 1 --2- 172.21.15.164:0/719623325 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f64fc0047d0 0x7f64fc004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:29.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.445+0000 7f6514de8640 1 --2- 172.21.15.164:0/719623325 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f64fc0052b0 0x7f64fc00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:29.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.445+0000 7f6514de8640 1 -- 172.21.15.164:0/719623325 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f65100b1500 con 0x7f64fc0052b0 2024-03-23T14:57:29.452 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.445+0000 7f6514de8640 1 -- 172.21.15.164:0/719623325 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f64fc00c180 con 0x7f64fc0047d0 2024-03-23T14:57:29.452 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.445+0000 7f6514de8640 1 -- 172.21.15.164:0/719623325 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f64fc00c340 con 0x7f64fc003980 2024-03-23T14:57:29.452 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.445+0000 7f65165eb640 1 --2- 172.21.15.164:0/719623325 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f64fc0052b0 0x7f64fc00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:29.452 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.445+0000 7f65155e9640 1 --2- 172.21.15.164:0/719623325 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f64fc0047d0 0x7f64fc004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:29.452 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.445+0000 7f6515dea640 1 --2- 172.21.15.164:0/719623325 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f64fc003980 0x7f64fc003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:29.452 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.445+0000 7f65165eb640 1 -- 172.21.15.164:0/719623325 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f64fc003980 msgr2=0x7f64fc003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:29.452 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.445+0000 7f65165eb640 1 --2- 172.21.15.164:0/719623325 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f64fc003980 0x7f64fc003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:29.452 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.445+0000 7f65165eb640 1 -- 172.21.15.164:0/719623325 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f64fc0047d0 msgr2=0x7f64fc004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:29.452 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.445+0000 7f65165eb640 1 --2- 172.21.15.164:0/719623325 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f64fc0047d0 0x7f64fc004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:29.452 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.445+0000 7f65165eb640 1 -- 172.21.15.164:0/719623325 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64fc003530 con 0x7f64fc0052b0 2024-03-23T14:57:29.452 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.445+0000 7f6515dea640 1 --2- 172.21.15.164:0/719623325 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f64fc003980 0x7f64fc003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T14:57:29.456 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.445+0000 7f65165eb640 1 -- 172.21.15.164:0/719623325 reap_dead start 2024-03-23T14:57:29.456 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.445+0000 7f65165eb640 1 --2- 172.21.15.164:0/719623325 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f64fc0052b0 0x7f64fc00b990 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f6508009fc0 tx=0x7f650801cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7de608b71af634ee server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:29.456 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.449+0000 7f6514de8640 1 -- 172.21.15.164:0/719623325 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65080a5c80 con 0x7f64fc0052b0 2024-03-23T14:57:29.456 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.449+0000 7f6514de8640 1 -- 172.21.15.164:0/719623325 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f650800f040 con 0x7f64fc0052b0 2024-03-23T14:57:29.456 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.449+0000 7f6514de8640 1 -- 172.21.15.164:0/719623325 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65080039a0 con 0x7f64fc0052b0 2024-03-23T14:57:29.456 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.449+0000 7f651804e640 1 -- 172.21.15.164:0/719623325 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f64fc0052b0 msgr2=0x7f64fc00b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:29.456 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.449+0000 7f651804e640 1 --2- 172.21.15.164:0/719623325 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f64fc0052b0 0x7f64fc00b990 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f6508009fc0 tx=0x7f650801cc90 comp rx=0 tx=0).stop 2024-03-23T14:57:29.456 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.449+0000 7f651804e640 1 -- 172.21.15.164:0/719623325 shutdown_connections 2024-03-23T14:57:29.456 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.449+0000 7f651804e640 1 --2- 172.21.15.164:0/719623325 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f64fc0052b0 0x7f64fc00b990 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:29.456 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.449+0000 7f651804e640 1 -- 172.21.15.164:0/719623325 >> 172.21.15.164:0/719623325 conn(0x7f65100b8ca0 msgr2=0x7f65100b90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:29.456 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.449+0000 7f651804e640 1 -- 172.21.15.164:0/719623325 shutdown_connections 2024-03-23T14:57:29.456 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.449+0000 7f651804e640 1 -- 172.21.15.164:0/719623325 wait complete. 2024-03-23T14:57:29.456 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.449+0000 7f651804e640 1 Processor -- start 2024-03-23T14:57:29.457 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.449+0000 7f651804e640 1 -- start start 2024-03-23T14:57:29.457 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.449+0000 7f651804e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f64fc0052b0 0x7f651006f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:29.457 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.449+0000 7f651804e640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f651013a5c0 0x7f65101327d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:29.457 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.449+0000 7f651804e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6510132d10 0x7f6510071280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:29.457 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.449+0000 7f651804e640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6510070df0 con 0x7f64fc0052b0 2024-03-23T14:57:29.457 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.449+0000 7f651804e640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f651006fbf0 con 0x7f6510132d10 2024-03-23T14:57:29.457 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.449+0000 7f651804e640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f6510070a80 con 0x7f651013a5c0 2024-03-23T14:57:29.457 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.449+0000 7f6515dea640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f64fc0052b0 0x7f651006f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:29.457 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.449+0000 7f6515dea640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f64fc0052b0 0x7f651006f2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:49398/0 (socket says 172.21.15.164:49398) 2024-03-23T14:57:29.457 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.449+0000 7f6515dea640 1 -- 172.21.15.164:0/3891356718 learned_addr learned my addr 172.21.15.164:0/3891356718 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:29.457 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.449+0000 7f65155e9640 1 --2- 172.21.15.164:0/3891356718 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f651013a5c0 0x7f65101327d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:29.457 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.449+0000 7f65165eb640 1 --2- 172.21.15.164:0/3891356718 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6510132d10 0x7f6510071280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:29.457 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.449+0000 7f6515dea640 1 -- 172.21.15.164:0/3891356718 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f651013a5c0 msgr2=0x7f65101327d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:29.457 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.449+0000 7f6515dea640 1 --2- 172.21.15.164:0/3891356718 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f651013a5c0 0x7f65101327d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:29.457 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.449+0000 7f6515dea640 1 -- 172.21.15.164:0/3891356718 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6510132d10 msgr2=0x7f6510071280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:29.458 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.449+0000 7f6515dea640 1 --2- 172.21.15.164:0/3891356718 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6510132d10 0x7f6510071280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:29.458 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.449+0000 7f6515dea640 1 -- 172.21.15.164:0/3891356718 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6510137dc0 con 0x7f64fc0052b0 2024-03-23T14:57:29.458 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.449+0000 7f65165eb640 1 --2- 172.21.15.164:0/3891356718 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6510132d10 0x7f6510071280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T14:57:29.458 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.449+0000 7f6515dea640 1 --2- 172.21.15.164:0/3891356718 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f64fc0052b0 0x7f651006f2b0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f650000d770 tx=0x7f650000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:29.458 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.449+0000 7f65067fc640 1 -- 172.21.15.164:0/3891356718 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6500005270 con 0x7f64fc0052b0 2024-03-23T14:57:29.458 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.449+0000 7f65067fc640 1 -- 172.21.15.164:0/3891356718 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65000053f0 con 0x7f64fc0052b0 2024-03-23T14:57:29.458 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.449+0000 7f651804e640 1 -- 172.21.15.164:0/3891356718 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f651006f7f0 con 0x7f64fc0052b0 2024-03-23T14:57:29.458 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.449+0000 7f65067fc640 1 -- 172.21.15.164:0/3891356718 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f650000b8f0 con 0x7f64fc0052b0 2024-03-23T14:57:29.458 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.449+0000 7f651804e640 1 -- 172.21.15.164:0/3891356718 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6510071990 con 0x7f64fc0052b0 2024-03-23T14:57:29.458 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.449+0000 7f651804e640 1 -- 172.21.15.164:0/3891356718 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6510070c40 con 0x7f64fc0052b0 2024-03-23T14:57:29.458 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.453+0000 7f65067fc640 1 -- 172.21.15.164:0/3891356718 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7f650000ba50 con 0x7f64fc0052b0 2024-03-23T14:57:29.459 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.453+0000 7f65067fc640 1 --2- 172.21.15.164:0/3891356718 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f64f40672a0 0x7f64f4069760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:29.459 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.453+0000 7f65067fc640 1 -- 172.21.15.164:0/3891356718 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f6500084f00 con 0x7f64fc0052b0 2024-03-23T14:57:29.459 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.453+0000 7f65155e9640 1 --2- 172.21.15.164:0/3891356718 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f64f40672a0 0x7f64f4069760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:29.459 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.453+0000 7f65155e9640 1 --2- 172.21.15.164:0/3891356718 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f64f40672a0 0x7f64f4069760 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f650c000fc0 tx=0x7f650c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:29.460 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.457+0000 7f65067fc640 1 -- 172.21.15.164:0/3891356718 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7f6500051b40 con 0x7f64fc0052b0 2024-03-23T14:57:29.606 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.601+0000 7f651804e640 1 -- 172.21.15.164:0/3891356718 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f651002d370 con 0x7f64fc0052b0 2024-03-23T14:57:29.606 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.601+0000 7f65067fc640 1 -- 172.21.15.164:0/3891356718 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f6500055200 con 0x7f64fc0052b0 2024-03-23T14:57:29.606 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 64 2024-03-23T14:57:29.608 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.605+0000 7f651804e640 1 -- 172.21.15.164:0/3891356718 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f64f40672a0 msgr2=0x7f64f4069760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:29.608 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.605+0000 7f651804e640 1 --2- 172.21.15.164:0/3891356718 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f64f40672a0 0x7f64f4069760 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f650c000fc0 tx=0x7f650c009040 comp rx=0 tx=0).stop 2024-03-23T14:57:29.608 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.605+0000 7f651804e640 1 -- 172.21.15.164:0/3891356718 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f64fc0052b0 msgr2=0x7f651006f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:29.608 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.605+0000 7f651804e640 1 --2- 172.21.15.164:0/3891356718 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f64fc0052b0 0x7f651006f2b0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f650000d770 tx=0x7f650000dc30 comp rx=0 tx=0).stop 2024-03-23T14:57:29.608 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.605+0000 7f651804e640 1 -- 172.21.15.164:0/3891356718 shutdown_connections 2024-03-23T14:57:29.608 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.605+0000 7f651804e640 1 --2- 172.21.15.164:0/3891356718 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f64f40672a0 0x7f64f4069760 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:29.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.605+0000 7f651804e640 1 --2- 172.21.15.164:0/3891356718 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6510132d10 0x7f6510071280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:29.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.605+0000 7f651804e640 1 --2- 172.21.15.164:0/3891356718 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f651013a5c0 0x7f65101327d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:29.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.605+0000 7f651804e640 1 --2- 172.21.15.164:0/3891356718 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f64fc0052b0 0x7f651006f2b0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:29.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.605+0000 7f651804e640 1 -- 172.21.15.164:0/3891356718 >> 172.21.15.164:0/3891356718 conn(0x7f65100b8ca0 msgr2=0x7f65100b9080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:29.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.605+0000 7f651804e640 1 -- 172.21.15.164:0/3891356718 shutdown_connections 2024-03-23T14:57:29.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.605+0000 7f651804e640 1 -- 172.21.15.164:0/3891356718 wait complete. 2024-03-23T14:57:29.624 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T14:57:29.730 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.725+0000 7f89b707e640 1 Processor -- start 2024-03-23T14:57:29.731 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.725+0000 7f89b707e640 1 -- start start 2024-03-23T14:57:29.732 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.725+0000 7f89b707e640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f89b00b0230 0x7f89b00b34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:29.732 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.725+0000 7f89b707e640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f89b0148cf0 0x7f89b0149180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:29.732 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.725+0000 7f89b707e640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89b00b51d0 con 0x7f89b00b1c70 2024-03-23T14:57:29.732 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.725+0000 7f89b707e640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f89b00b0610 con 0x7f89b00b0230 2024-03-23T14:57:29.732 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.725+0000 7f89b707e640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f89b00b1500 con 0x7f89b0148cf0 2024-03-23T14:57:29.732 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.725+0000 7f89b4e1a640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f89b00b0230 0x7f89b00b34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:29.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.725+0000 7f89b4e1a640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f89b00b0230 0x7f89b00b34d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:49412/0 (socket says 172.21.15.164:49412) 2024-03-23T14:57:29.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.725+0000 7f89b4e1a640 1 -- 172.21.15.164:0/1976543825 learned_addr learned my addr 172.21.15.164:0/1976543825 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:29.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.729+0000 7f89affff640 1 --2- 172.21.15.164:0/1976543825 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f89b0148cf0 0x7f89b0149180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:29.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.729+0000 7f89b4e1a640 1 -- 172.21.15.164:0/1976543825 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f89b0148cf0 msgr2=0x7f89b0149180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:29.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.729+0000 7f89b4e1a640 1 --2- 172.21.15.164:0/1976543825 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f89b0148cf0 0x7f89b0149180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:29.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.729+0000 7f89b4e1a640 1 -- 172.21.15.164:0/1976543825 >> v1:172.21.15.173:6789/0 conn(0x7f89b00b1c70 legacy=0x7f89b00b20a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:29.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.729+0000 7f89b4e1a640 1 -- 172.21.15.164:0/1976543825 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89b006d250 con 0x7f89b00b0230 2024-03-23T14:57:29.734 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.729+0000 7f89b4e1a640 1 --2- 172.21.15.164:0/1976543825 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f89b00b0230 0x7f89b00b34d0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f89a0009870 tx=0x7f89a001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a087a65151543844 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:29.734 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.729+0000 7f89af7fe640 1 -- 172.21.15.164:0/1976543825 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89a00a6040 con 0x7f89b00b0230 2024-03-23T14:57:29.734 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.729+0000 7f89af7fe640 1 -- 172.21.15.164:0/1976543825 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89a00aa600 con 0x7f89b00b0230 2024-03-23T14:57:29.734 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.729+0000 7f89af7fe640 1 -- 172.21.15.164:0/1976543825 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89a00aa990 con 0x7f89b00b0230 2024-03-23T14:57:29.735 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.729+0000 7f89b707e640 1 -- 172.21.15.164:0/1976543825 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f89b00b0230 msgr2=0x7f89b00b34d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:29.735 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.729+0000 7f89b707e640 1 --2- 172.21.15.164:0/1976543825 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f89b00b0230 0x7f89b00b34d0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f89a0009870 tx=0x7f89a001c4a0 comp rx=0 tx=0).stop 2024-03-23T14:57:29.735 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.729+0000 7f89b707e640 1 -- 172.21.15.164:0/1976543825 shutdown_connections 2024-03-23T14:57:29.735 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.729+0000 7f89b707e640 1 --2- 172.21.15.164:0/1976543825 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f89b0148cf0 0x7f89b0149180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:29.735 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.729+0000 7f89b707e640 1 --2- 172.21.15.164:0/1976543825 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f89b00b0230 0x7f89b00b34d0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:29.735 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.729+0000 7f89b707e640 1 -- 172.21.15.164:0/1976543825 >> 172.21.15.164:0/1976543825 conn(0x7f89b00b8ca0 msgr2=0x7f89b00b90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:29.735 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.729+0000 7f89b707e640 1 -- 172.21.15.164:0/1976543825 shutdown_connections 2024-03-23T14:57:29.736 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.729+0000 7f89b707e640 1 -- 172.21.15.164:0/1976543825 wait complete. 2024-03-23T14:57:29.736 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.733+0000 7f89b707e640 1 Processor -- start 2024-03-23T14:57:29.736 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.733+0000 7f89b707e640 1 -- start start 2024-03-23T14:57:29.738 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.733+0000 7f89b707e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f89b00b0230 0x7f89b01504a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:29.738 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.733+0000 7f89b707e640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f89b00b1c70 0x7f89b014f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:29.739 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.733+0000 7f89b707e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f89b0148cf0 0x7f89b00af3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:29.739 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.733+0000 7f89b707e640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f89b00aef60 con 0x7f89b00b0230 2024-03-23T14:57:29.739 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.733+0000 7f89b707e640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f89b014f7e0 con 0x7f89b0148cf0 2024-03-23T14:57:29.739 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.733+0000 7f89b707e640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f89b009c9c0 con 0x7f89b00b1c70 2024-03-23T14:57:29.739 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.733+0000 7f89b4e1a640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f89b00b0230 0x7f89b01504a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:29.739 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.733+0000 7f89b4e1a640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f89b00b0230 0x7f89b01504a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:49420/0 (socket says 172.21.15.164:49420) 2024-03-23T14:57:29.739 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.733+0000 7f89b4e1a640 1 -- 172.21.15.164:0/403934025 learned_addr learned my addr 172.21.15.164:0/403934025 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:29.739 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.733+0000 7f89affff640 1 --2- 172.21.15.164:0/403934025 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f89b00b1c70 0x7f89b014f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:29.739 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.733+0000 7f89b561b640 1 --2- 172.21.15.164:0/403934025 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f89b0148cf0 0x7f89b00af3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:29.739 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.733+0000 7f89b4e1a640 1 -- 172.21.15.164:0/403934025 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f89b00b1c70 msgr2=0x7f89b014f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:29.739 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.733+0000 7f89b4e1a640 1 --2- 172.21.15.164:0/403934025 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f89b00b1c70 0x7f89b014f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:29.739 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.733+0000 7f89b4e1a640 1 -- 172.21.15.164:0/403934025 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f89b0148cf0 msgr2=0x7f89b00af3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:29.739 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.733+0000 7f89b4e1a640 1 --2- 172.21.15.164:0/403934025 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f89b0148cf0 0x7f89b00af3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:29.739 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.733+0000 7f89b4e1a640 1 -- 172.21.15.164:0/403934025 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89a00094d0 con 0x7f89b00b0230 2024-03-23T14:57:29.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.733+0000 7f89b561b640 1 --2- 172.21.15.164:0/403934025 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f89b0148cf0 0x7f89b00af3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T14:57:29.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.733+0000 7f89b4e1a640 1 --2- 172.21.15.164:0/403934025 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f89b00b0230 0x7f89b01504a0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f89a001ab50 tx=0x7f89a001ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:29.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.733+0000 7f89ad7fa640 1 -- 172.21.15.164:0/403934025 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89a00b3800 con 0x7f89b00b0230 2024-03-23T14:57:29.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.733+0000 7f89ad7fa640 1 -- 172.21.15.164:0/403934025 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89a00b3960 con 0x7f89b00b0230 2024-03-23T14:57:29.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.733+0000 7f89b707e640 1 -- 172.21.15.164:0/403934025 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89b014fa40 con 0x7f89b00b0230 2024-03-23T14:57:29.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.733+0000 7f89ad7fa640 1 -- 172.21.15.164:0/403934025 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89a00b3cd0 con 0x7f89b00b0230 2024-03-23T14:57:29.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.733+0000 7f89b707e640 1 -- 172.21.15.164:0/403934025 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89b0142f50 con 0x7f89b00b0230 2024-03-23T14:57:29.740 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.733+0000 7f89b707e640 1 -- 172.21.15.164:0/403934025 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89b009cb50 con 0x7f89b00b0230 2024-03-23T14:57:29.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.733+0000 7f89ad7fa640 1 -- 172.21.15.164:0/403934025 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7f89a00c6430 con 0x7f89b00b0230 2024-03-23T14:57:29.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.737+0000 7f89ad7fa640 1 --2- 172.21.15.164:0/403934025 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f899c0672d0 0x7f899c069790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:29.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.737+0000 7f89ad7fa640 1 -- 172.21.15.164:0/403934025 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f89a011fad0 con 0x7f89b00b0230 2024-03-23T14:57:29.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.737+0000 7f89affff640 1 --2- 172.21.15.164:0/403934025 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f899c0672d0 0x7f899c069790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:29.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.737+0000 7f89affff640 1 --2- 172.21.15.164:0/403934025 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f899c0672d0 0x7f899c069790 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f89b00afc10 tx=0x7f89a4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:29.744 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.741+0000 7f89ad7fa640 1 -- 172.21.15.164:0/403934025 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7f89a00ec710 con 0x7f89b00b0230 2024-03-23T14:57:29.921 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.917+0000 7f89b707e640 1 -- 172.21.15.164:0/403934025 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f89b00690a0 con 0x7f89b00b0230 2024-03-23T14:57:29.922 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.917+0000 7f89ad7fa640 1 -- 172.21.15.164:0/403934025 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+3533 (secure 0 0 0) 0x7f89a00efdd0 con 0x7f89b00b0230 2024-03-23T14:57:29.922 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-23T14:57:29.922 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/3352372536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":3352372536},{"type":"v1","addr":"172.21.15.164:6835","nonce":3352372536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-23T14:56:58.953326+0000","modified":"2024-03-23T14:57:00.999623+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4279":{"gid":4279,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.173:6833/3506402771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":3506402771},{"type":"v1","addr":"172.21.15.173:6833","nonce":3506402771}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4279,"qdb_cluster":[4279]},"id":1}]} 2024-03-23T14:57:29.923 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 6 2024-03-23T14:57:29.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.921+0000 7f89b707e640 1 -- 172.21.15.164:0/403934025 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f899c0672d0 msgr2=0x7f899c069790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:29.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.921+0000 7f89b707e640 1 --2- 172.21.15.164:0/403934025 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f899c0672d0 0x7f899c069790 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f89b00afc10 tx=0x7f89a4008040 comp rx=0 tx=0).stop 2024-03-23T14:57:29.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.921+0000 7f89b707e640 1 -- 172.21.15.164:0/403934025 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f89b00b0230 msgr2=0x7f89b01504a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:29.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.921+0000 7f89b707e640 1 --2- 172.21.15.164:0/403934025 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f89b00b0230 0x7f89b01504a0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f89a001ab50 tx=0x7f89a001ace0 comp rx=0 tx=0).stop 2024-03-23T14:57:29.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.921+0000 7f89b707e640 1 -- 172.21.15.164:0/403934025 shutdown_connections 2024-03-23T14:57:29.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.921+0000 7f89b707e640 1 --2- 172.21.15.164:0/403934025 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f899c0672d0 0x7f899c069790 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:29.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.921+0000 7f89b707e640 1 --2- 172.21.15.164:0/403934025 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f89b0148cf0 0x7f89b00af3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:29.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.921+0000 7f89b707e640 1 --2- 172.21.15.164:0/403934025 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f89b00b1c70 0x7f89b014f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:29.925 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.921+0000 7f89b707e640 1 --2- 172.21.15.164:0/403934025 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f89b00b0230 0x7f89b01504a0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:29.925 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.921+0000 7f89b707e640 1 -- 172.21.15.164:0/403934025 >> 172.21.15.164:0/403934025 conn(0x7f89b00b8ca0 msgr2=0x7f89b00b5cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:29.925 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.921+0000 7f89b707e640 1 -- 172.21.15.164:0/403934025 shutdown_connections 2024-03-23T14:57:29.925 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:29.921+0000 7f89b707e640 1 -- 172.21.15.164:0/403934025 wait complete. 2024-03-23T14:57:29.940 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T14:57:30.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.049+0000 7fa40d61d640 1 Processor -- start 2024-03-23T14:57:30.055 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.049+0000 7fa40d61d640 1 -- start start 2024-03-23T14:57:30.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.053+0000 7fa40d61d640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa4080b56f0 0x7fa40814a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:30.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.053+0000 7fa40d61d640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4080b4300 con 0x7fa4080b4a70 2024-03-23T14:57:30.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.053+0000 7fa40d61d640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4080b45e0 con 0x7fa408141640 2024-03-23T14:57:30.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.053+0000 7fa40d61d640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fa4080b5fa0 con 0x7fa4080b56f0 2024-03-23T14:57:30.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.053+0000 7fa4067fc640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fa4080b4a70 0x7fa408140f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:42550/0 (socket says 172.21.15.164:42550) 2024-03-23T14:57:30.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.053+0000 7fa4067fc640 1 -- 172.21.15.164:0/2513262118 learned_addr learned my addr 172.21.15.164:0/2513262118 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-03-23T14:57:30.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.053+0000 7fa406ffd640 1 --2- 172.21.15.164:0/2513262118 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa4080b56f0 0x7fa40814a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:30.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.053+0000 7fa405ffb640 1 -- 172.21.15.164:0/2513262118 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3522431371 0 0) 0x7fa4080b4300 con 0x7fa4080b4a70 2024-03-23T14:57:30.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.053+0000 7fa405ffb640 1 -- 172.21.15.164:0/2513262118 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa3f0003530 con 0x7fa4080b4a70 2024-03-23T14:57:30.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.053+0000 7fa405ffb640 1 -- 172.21.15.164:0/2513262118 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1953247260 0 0) 0x7fa3f0003530 con 0x7fa4080b4a70 2024-03-23T14:57:30.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.053+0000 7fa405ffb640 1 -- 172.21.15.164:0/2513262118 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa4080b56f0 msgr2=0x7fa40814a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:30.058 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.053+0000 7fa405ffb640 1 --2- 172.21.15.164:0/2513262118 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa4080b56f0 0x7fa40814a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:30.059 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.053+0000 7fa405ffb640 1 -- 172.21.15.164:0/2513262118 >> v1:172.21.15.173:6789/0 conn(0x7fa408141640 legacy=0x7fa408151c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:30.059 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.053+0000 7fa405ffb640 1 -- 172.21.15.164:0/2513262118 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa40814ade0 con 0x7fa4080b4a70 2024-03-23T14:57:30.059 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.053+0000 7fa405ffb640 1 -- 172.21.15.164:0/2513262118 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2526390548 0 0) 0x7fa3fc001680 con 0x7fa4080b4a70 2024-03-23T14:57:30.059 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.053+0000 7fa405ffb640 1 -- 172.21.15.164:0/2513262118 >> v1:172.21.15.164:6789/0 conn(0x7fa4080b4a70 legacy=0x7fa408140f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:30.059 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.053+0000 7fa405ffb640 1 --2- 172.21.15.164:0/2513262118 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa4080b56f0 0x7fa3f00039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:30.059 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.053+0000 7fa405ffb640 1 --2- 172.21.15.164:0/2513262118 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa3f0003ef0 0x7fa3f00047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:30.059 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.053+0000 7fa405ffb640 1 --2- 172.21.15.164:0/2513262118 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa3f0004d10 0x7fa3f0005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:30.059 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.053+0000 7fa405ffb640 1 -- 172.21.15.164:0/2513262118 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa3f000ba40 con 0x7fa3f0004d10 2024-03-23T14:57:30.059 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.053+0000 7fa405ffb640 1 -- 172.21.15.164:0/2513262118 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa3f000bb80 con 0x7fa3f0003ef0 2024-03-23T14:57:30.059 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.053+0000 7fa405ffb640 1 -- 172.21.15.164:0/2513262118 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fa3f000bd40 con 0x7fa4080b56f0 2024-03-23T14:57:30.059 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.053+0000 7fa4077fe640 1 --2- 172.21.15.164:0/2513262118 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa3f0004d10 0x7fa3f0005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:30.059 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.053+0000 7fa4067fc640 1 --2- 172.21.15.164:0/2513262118 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa3f0003ef0 0x7fa3f00047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:30.059 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.053+0000 7fa406ffd640 1 --2- 172.21.15.164:0/2513262118 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa4080b56f0 0x7fa3f00039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:30.059 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.053+0000 7fa4077fe640 1 -- 172.21.15.164:0/2513262118 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa4080b56f0 msgr2=0x7fa3f00039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:30.060 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.053+0000 7fa4077fe640 1 --2- 172.21.15.164:0/2513262118 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa4080b56f0 0x7fa3f00039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:30.060 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.053+0000 7fa4077fe640 1 -- 172.21.15.164:0/2513262118 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa3f0003ef0 msgr2=0x7fa3f00047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:30.060 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.053+0000 7fa4077fe640 1 --2- 172.21.15.164:0/2513262118 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa3f0003ef0 0x7fa3f00047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:30.060 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.053+0000 7fa4077fe640 1 -- 172.21.15.164:0/2513262118 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3f0003530 con 0x7fa3f0004d10 2024-03-23T14:57:30.060 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.053+0000 7fa406ffd640 1 --2- 172.21.15.164:0/2513262118 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa4080b56f0 0x7fa3f00039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T14:57:30.061 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.053+0000 7fa4077fe640 1 --2- 172.21.15.164:0/2513262118 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa3f0004d10 0x7fa3f0005190 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fa400001f60 tx=0x7fa40001c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ed6fe1a6e2d6472 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:30.061 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.053+0000 7fa405ffb640 1 -- 172.21.15.164:0/2513262118 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4000a5770 con 0x7fa3f0004d10 2024-03-23T14:57:30.061 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.053+0000 7fa405ffb640 1 -- 172.21.15.164:0/2513262118 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4000a58d0 con 0x7fa3f0004d10 2024-03-23T14:57:30.061 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.053+0000 7fa405ffb640 1 -- 172.21.15.164:0/2513262118 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4000a5c80 con 0x7fa3f0004d10 2024-03-23T14:57:30.061 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.057+0000 7fa40d61d640 1 -- 172.21.15.164:0/2513262118 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa3f0004d10 msgr2=0x7fa3f0005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:30.061 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.057+0000 7fa40d61d640 1 --2- 172.21.15.164:0/2513262118 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa3f0004d10 0x7fa3f0005190 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fa400001f60 tx=0x7fa40001c840 comp rx=0 tx=0).stop 2024-03-23T14:57:30.062 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.057+0000 7fa4077fe640 1 -- 172.21.15.164:0/2513262118 reap_dead start 2024-03-23T14:57:30.062 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.057+0000 7fa40d61d640 1 -- 172.21.15.164:0/2513262118 shutdown_connections 2024-03-23T14:57:30.062 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.057+0000 7fa40d61d640 1 -- 172.21.15.164:0/2513262118 >> 172.21.15.164:0/2513262118 conn(0x7fa40802c3c0 msgr2=0x7fa40802c7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:30.062 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.057+0000 7fa40d61d640 1 -- 172.21.15.164:0/2513262118 shutdown_connections 2024-03-23T14:57:30.062 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.057+0000 7fa40d61d640 1 -- 172.21.15.164:0/2513262118 wait complete. 2024-03-23T14:57:30.062 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.057+0000 7fa40d61d640 1 Processor -- start 2024-03-23T14:57:30.062 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.057+0000 7fa40d61d640 1 -- start start 2024-03-23T14:57:30.064 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.057+0000 7fa40d61d640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa3f0004d10 0x7fa408142f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:30.064 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.057+0000 7fa40d61d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa4081dcac0 0x7fa4081dcf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:30.064 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.057+0000 7fa40d61d640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa4080b27c0 0x7fa4080b2c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:30.064 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.057+0000 7fa40d61d640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa408143ad0 con 0x7fa4081dcac0 2024-03-23T14:57:30.064 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.057+0000 7fa40d61d640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa4081dc670 con 0x7fa3f0004d10 2024-03-23T14:57:30.064 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.057+0000 7fa40d61d640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fa4080a9fb0 con 0x7fa4080b27c0 2024-03-23T14:57:30.064 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.057+0000 7fa4067fc640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa4081dcac0 0x7fa4081dcf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:30.065 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.057+0000 7fa4067fc640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa4081dcac0 0x7fa4081dcf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:49434/0 (socket says 172.21.15.164:49434) 2024-03-23T14:57:30.065 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.057+0000 7fa4067fc640 1 -- 172.21.15.164:0/2453981893 learned_addr learned my addr 172.21.15.164:0/2453981893 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:30.065 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.057+0000 7fa406ffd640 1 --2- 172.21.15.164:0/2453981893 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa3f0004d10 0x7fa408142f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:30.065 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.057+0000 7fa4077fe640 1 --2- 172.21.15.164:0/2453981893 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa4080b27c0 0x7fa4080b2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:30.065 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.057+0000 7fa4067fc640 1 -- 172.21.15.164:0/2453981893 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa4080b27c0 msgr2=0x7fa4080b2c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:30.065 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.057+0000 7fa4067fc640 1 --2- 172.21.15.164:0/2453981893 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa4080b27c0 0x7fa4080b2c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:30.065 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.057+0000 7fa4067fc640 1 -- 172.21.15.164:0/2453981893 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa3f0004d10 msgr2=0x7fa408142f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:30.065 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.057+0000 7fa4067fc640 1 --2- 172.21.15.164:0/2453981893 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa3f0004d10 0x7fa408142f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:30.065 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.057+0000 7fa4067fc640 1 -- 172.21.15.164:0/2453981893 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4081dd460 con 0x7fa4081dcac0 2024-03-23T14:57:30.065 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.057+0000 7fa4067fc640 1 --2- 172.21.15.164:0/2453981893 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa4081dcac0 0x7fa4081dcf20 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa3fc001060 tx=0x7fa3fc011d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:30.065 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.057+0000 7fa3ef7fe640 1 -- 172.21.15.164:0/2453981893 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3fc00fb60 con 0x7fa4081dcac0 2024-03-23T14:57:30.065 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.057+0000 7fa3ef7fe640 1 -- 172.21.15.164:0/2453981893 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3fc00fcc0 con 0x7fa4081dcac0 2024-03-23T14:57:30.065 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.057+0000 7fa40d61d640 1 -- 172.21.15.164:0/2453981893 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa408142110 con 0x7fa4081dcac0 2024-03-23T14:57:30.065 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.057+0000 7fa3ef7fe640 1 -- 172.21.15.164:0/2453981893 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3fc015610 con 0x7fa4081dcac0 2024-03-23T14:57:30.065 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.057+0000 7fa40d61d640 1 -- 172.21.15.164:0/2453981893 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa408143c90 con 0x7fa4081dcac0 2024-03-23T14:57:30.065 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.061+0000 7fa40d61d640 1 -- 172.21.15.164:0/2453981893 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4081dc7e0 con 0x7fa4081dcac0 2024-03-23T14:57:30.066 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.061+0000 7fa3ef7fe640 1 -- 172.21.15.164:0/2453981893 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7fa3fc015770 con 0x7fa4081dcac0 2024-03-23T14:57:30.067 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.061+0000 7fa3ef7fe640 1 --2- 172.21.15.164:0/2453981893 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fa3f40672a0 0x7fa3f4069760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:30.067 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.061+0000 7fa3ef7fe640 1 -- 172.21.15.164:0/2453981893 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa3fc088390 con 0x7fa4081dcac0 2024-03-23T14:57:30.067 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.061+0000 7fa406ffd640 1 --2- 172.21.15.164:0/2453981893 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fa3f40672a0 0x7fa3f4069760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:30.067 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.061+0000 7fa406ffd640 1 --2- 172.21.15.164:0/2453981893 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fa3f40672a0 0x7fa3f4069760 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa3f8009ab0 tx=0x7fa3f8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:30.069 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.065+0000 7fa3ef7fe640 1 -- 172.21.15.164:0/2453981893 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7fa4081dc7e0 con 0x7fa4081dcac0 2024-03-23T14:57:30.216 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.213+0000 7fa40d61d640 1 -- 172.21.15.164:0/2453981893 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa4081dc7e0 con 0x7fa4081dcac0 2024-03-23T14:57:30.218 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.213+0000 7fa3ef7fe640 1 -- 172.21.15.164:0/2453981893 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16629 (secure 0 0 0) 0x7fa4081dc7e0 con 0x7fa4081dcac0 2024-03-23T14:57:30.219 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-23T14:57:30.219 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":20,"fsid":"45c7911b-b69d-4e68-907b-fbfeb780aa3b","created":"2024-03-23T14:56:34.367633+0000","modified":"2024-03-23T14:57:00.611265+0000","last_up_change":"2024-03-23T14:56:46.297403+0000","last_in_change":"2024-03-23T14:56:36.629884+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T14:56:47.186157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-23T14:56:56.994057+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-23T14:56:57.953862+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"80abdc62-bd0d-4e60-a554-7ed6fd67cf54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":984716690},{"type":"v1","addr":"172.21.15.164:6801","nonce":984716690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":984716690},{"type":"v1","addr":"172.21.15.164:6803","nonce":984716690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":984716690},{"type":"v1","addr":"172.21.15.164:6807","nonce":984716690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":984716690},{"type":"v1","addr":"172.21.15.164:6805","nonce":984716690}]},"public_addr":"172.21.15.164:6801/984716690","cluster_addr":"172.21.15.164:6803/984716690","heartbeat_back_addr":"172.21.15.164:6807/984716690","heartbeat_front_addr":"172.21.15.164:6805/984716690","state":["exists","up"]},{"osd":1,"uuid":"f9001b06-be4d-4f42-a0e6-bcef2afa0d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":1349657239},{"type":"v1","addr":"172.21.15.164:6817","nonce":1349657239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":1349657239},{"type":"v1","addr":"172.21.15.164:6819","nonce":1349657239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":1349657239},{"type":"v1","addr":"172.21.15.164:6823","nonce":1349657239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":1349657239},{"type":"v1","addr":"172.21.15.164:6821","nonce":1349657239}]},"public_addr":"172.21.15.164:6817/1349657239","cluster_addr":"172.21.15.164:6819/1349657239","heartbeat_back_addr":"172.21.15.164:6823/1349657239","heartbeat_front_addr":"172.21.15.164:6821/1349657239","state":["exists","up"]},{"osd":2,"uuid":"fa0aeb9f-8b54-4831-a188-3e361c7980d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":887861252},{"type":"v1","addr":"172.21.15.164:6809","nonce":887861252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":887861252},{"type":"v1","addr":"172.21.15.164:6811","nonce":887861252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":887861252},{"type":"v1","addr":"172.21.15.164:6815","nonce":887861252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":887861252},{"type":"v1","addr":"172.21.15.164:6813","nonce":887861252}]},"public_addr":"172.21.15.164:6809/887861252","cluster_addr":"172.21.15.164:6811/887861252","heartbeat_back_addr":"172.21.15.164:6815/887861252","heartbeat_front_addr":"172.21.15.164:6813/887861252","state":["exists","up"]},{"osd":3,"uuid":"7e18c957-3091-46d5-8e17-8f3354c1726c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":812540271},{"type":"v1","addr":"172.21.15.164:6825","nonce":812540271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":812540271},{"type":"v1","addr":"172.21.15.164:6827","nonce":812540271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":812540271},{"type":"v1","addr":"172.21.15.164:6831","nonce":812540271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":812540271},{"type":"v1","addr":"172.21.15.164:6829","nonce":812540271}]},"public_addr":"172.21.15.164:6825/812540271","cluster_addr":"172.21.15.164:6827/812540271","heartbeat_back_addr":"172.21.15.164:6831/812540271","heartbeat_front_addr":"172.21.15.164:6829/812540271","state":["exists","up"]},{"osd":4,"uuid":"8e83f315-42bf-4471-9ce8-d81948da0f9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":2656857421},{"type":"v1","addr":"172.21.15.173:6825","nonce":2656857421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2656857421},{"type":"v1","addr":"172.21.15.173:6827","nonce":2656857421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2656857421},{"type":"v1","addr":"172.21.15.173:6831","nonce":2656857421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":2656857421},{"type":"v1","addr":"172.21.15.173:6829","nonce":2656857421}]},"public_addr":"172.21.15.173:6825/2656857421","cluster_addr":"172.21.15.173:6827/2656857421","heartbeat_back_addr":"172.21.15.173:6831/2656857421","heartbeat_front_addr":"172.21.15.173:6829/2656857421","state":["exists","up"]},{"osd":5,"uuid":"a230c872-71f9-40ab-8999-defd0162d4ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":3676483185},{"type":"v1","addr":"172.21.15.173:6801","nonce":3676483185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":3676483185},{"type":"v1","addr":"172.21.15.173:6803","nonce":3676483185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":3676483185},{"type":"v1","addr":"172.21.15.173:6807","nonce":3676483185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":3676483185},{"type":"v1","addr":"172.21.15.173:6805","nonce":3676483185}]},"public_addr":"172.21.15.173:6801/3676483185","cluster_addr":"172.21.15.173:6803/3676483185","heartbeat_back_addr":"172.21.15.173:6807/3676483185","heartbeat_front_addr":"172.21.15.173:6805/3676483185","state":["exists","up"]},{"osd":6,"uuid":"6193bfd7-8432-4f7f-a117-a3fcee21af99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":3631684087},{"type":"v1","addr":"172.21.15.173:6809","nonce":3631684087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":3631684087},{"type":"v1","addr":"172.21.15.173:6811","nonce":3631684087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":3631684087},{"type":"v1","addr":"172.21.15.173:6815","nonce":3631684087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":3631684087},{"type":"v1","addr":"172.21.15.173:6813","nonce":3631684087}]},"public_addr":"172.21.15.173:6809/3631684087","cluster_addr":"172.21.15.173:6811/3631684087","heartbeat_back_addr":"172.21.15.173:6815/3631684087","heartbeat_front_addr":"172.21.15.173:6813/3631684087","state":["exists","up"]},{"osd":7,"uuid":"b0f1b80a-afc4-4538-83b9-f081d2dcb8cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":3261114948},{"type":"v1","addr":"172.21.15.173:6817","nonce":3261114948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":3261114948},{"type":"v1","addr":"172.21.15.173:6819","nonce":3261114948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":3261114948},{"type":"v1","addr":"172.21.15.173:6823","nonce":3261114948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":3261114948},{"type":"v1","addr":"172.21.15.173:6821","nonce":3261114948}]},"public_addr":"172.21.15.173:6817/3261114948","cluster_addr":"172.21.15.173:6819/3261114948","heartbeat_back_addr":"172.21.15.173:6823/3261114948","heartbeat_front_addr":"172.21.15.173:6821/3261114948","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.188390+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.159932+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.191101+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.181776+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.224365+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.177515+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.153631+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.139972+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T14:57:30.221 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.217+0000 7fa40d61d640 1 -- 172.21.15.164:0/2453981893 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fa3f40672a0 msgr2=0x7fa3f4069760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:30.221 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.217+0000 7fa40d61d640 1 --2- 172.21.15.164:0/2453981893 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fa3f40672a0 0x7fa3f4069760 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa3f8009ab0 tx=0x7fa3f8008040 comp rx=0 tx=0).stop 2024-03-23T14:57:30.221 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.217+0000 7fa40d61d640 1 -- 172.21.15.164:0/2453981893 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa4081dcac0 msgr2=0x7fa4081dcf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:30.221 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.217+0000 7fa40d61d640 1 --2- 172.21.15.164:0/2453981893 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa4081dcac0 0x7fa4081dcf20 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa3fc001060 tx=0x7fa3fc011d30 comp rx=0 tx=0).stop 2024-03-23T14:57:30.221 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.217+0000 7fa40d61d640 1 -- 172.21.15.164:0/2453981893 shutdown_connections 2024-03-23T14:57:30.221 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.217+0000 7fa40d61d640 1 --2- 172.21.15.164:0/2453981893 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa4080b27c0 0x7fa4080b2c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:30.221 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.217+0000 7fa40d61d640 1 --2- 172.21.15.164:0/2453981893 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa4081dcac0 0x7fa4081dcf20 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:30.221 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.217+0000 7fa40d61d640 1 --2- 172.21.15.164:0/2453981893 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fa3f40672a0 0x7fa3f4069760 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:30.221 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.217+0000 7fa40d61d640 1 --2- 172.21.15.164:0/2453981893 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa3f0004d10 0x7fa408142f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:30.221 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.217+0000 7fa40d61d640 1 -- 172.21.15.164:0/2453981893 >> 172.21.15.164:0/2453981893 conn(0x7fa40802c3c0 msgr2=0x7fa40813f6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:30.221 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.217+0000 7fa40d61d640 1 -- 172.21.15.164:0/2453981893 shutdown_connections 2024-03-23T14:57:30.222 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.217+0000 7fa40d61d640 1 -- 172.21.15.164:0/2453981893 wait complete. 2024-03-23T14:57:30.237 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-23T14:57:30.237 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T14:57:30.358 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.353+0000 7fa951dc2640 1 Processor -- start 2024-03-23T14:57:30.359 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.353+0000 7fa951dc2640 1 -- start start 2024-03-23T14:57:30.360 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.353+0000 7fa951dc2640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fa94c0b6740 0x7fa94c0b41d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:30.360 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.353+0000 7fa951dc2640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa94c0b4710 0x7fa94c14fb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:30.360 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.353+0000 7fa951dc2640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa94c0b5530 con 0x7fa94c0b6360 2024-03-23T14:57:30.360 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.353+0000 7fa951dc2640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fa94c0b5a60 con 0x7fa94c0b6740 2024-03-23T14:57:30.360 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.353+0000 7fa951dc2640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fa94c071260 con 0x7fa94c0b4710 2024-03-23T14:57:30.361 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.353+0000 7fa94b7fe640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fa94c0b6360 0x7fa94c145a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:42556/0 (socket says 172.21.15.164:42556) 2024-03-23T14:57:30.361 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.357+0000 7fa94b7fe640 1 -- 172.21.15.164:0/532502040 learned_addr learned my addr 172.21.15.164:0/532502040 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-03-23T14:57:30.361 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.357+0000 7fa94affd640 1 --2- 172.21.15.164:0/532502040 >> v2:172.21.15.164:3300/0 conn(0x7fa94c0b6740 0x7fa94c0b41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:30.361 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.357+0000 7fa94bfff640 1 --2- 172.21.15.164:0/532502040 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa94c0b4710 0x7fa94c14fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:30.361 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.357+0000 7fa94a7fc640 1 -- 172.21.15.164:0/532502040 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 785694955 0 0) 0x7fa94c0b5530 con 0x7fa94c0b6360 2024-03-23T14:57:30.361 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.357+0000 7fa94a7fc640 1 -- 172.21.15.164:0/532502040 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa934003530 con 0x7fa94c0b6360 2024-03-23T14:57:30.361 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.357+0000 7fa94affd640 1 -- 172.21.15.164:0/532502040 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa94c0b4710 msgr2=0x7fa94c14fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:30.361 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.357+0000 7fa94affd640 1 --2- 172.21.15.164:0/532502040 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa94c0b4710 0x7fa94c14fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:30.361 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.357+0000 7fa94affd640 1 -- 172.21.15.164:0/532502040 >> v1:172.21.15.164:6789/0 conn(0x7fa94c0b6360 legacy=0x7fa94c145a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:30.361 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.357+0000 7fa94affd640 1 -- 172.21.15.164:0/532502040 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa94c0b2a70 con 0x7fa94c0b6740 2024-03-23T14:57:30.362 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.357+0000 7fa94affd640 1 --2- 172.21.15.164:0/532502040 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa94c0b6740 0x7fa94c0b41d0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fa940009d90 tx=0x7fa94001c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c63192e5a88342a0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:30.362 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.357+0000 7fa94a7fc640 1 -- 172.21.15.164:0/532502040 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9400a57a0 con 0x7fa94c0b6740 2024-03-23T14:57:30.362 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.357+0000 7fa94a7fc640 1 -- 172.21.15.164:0/532502040 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9400a5900 con 0x7fa94c0b6740 2024-03-23T14:57:30.362 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.357+0000 7fa94a7fc640 1 -- 172.21.15.164:0/532502040 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9400a5c90 con 0x7fa94c0b6740 2024-03-23T14:57:30.363 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.357+0000 7fa951dc2640 1 -- 172.21.15.164:0/532502040 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa94c0b6740 msgr2=0x7fa94c0b41d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:30.363 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.357+0000 7fa951dc2640 1 --2- 172.21.15.164:0/532502040 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa94c0b6740 0x7fa94c0b41d0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fa940009d90 tx=0x7fa94001c8e0 comp rx=0 tx=0).stop 2024-03-23T14:57:30.363 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.357+0000 7fa951dc2640 1 -- 172.21.15.164:0/532502040 shutdown_connections 2024-03-23T14:57:30.363 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.357+0000 7fa951dc2640 1 --2- 172.21.15.164:0/532502040 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa94c0b4710 0x7fa94c14fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:30.363 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.357+0000 7fa951dc2640 1 --2- 172.21.15.164:0/532502040 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa94c0b6740 0x7fa94c0b41d0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:30.364 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.357+0000 7fa951dc2640 1 -- 172.21.15.164:0/532502040 >> 172.21.15.164:0/532502040 conn(0x7fa94c0b7630 msgr2=0x7fa94c0b7a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:30.364 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.357+0000 7fa951dc2640 1 -- 172.21.15.164:0/532502040 shutdown_connections 2024-03-23T14:57:30.364 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.357+0000 7fa951dc2640 1 -- 172.21.15.164:0/532502040 wait complete. 2024-03-23T14:57:30.366 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.361+0000 7fa951dc2640 1 Processor -- start 2024-03-23T14:57:30.366 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.361+0000 7fa951dc2640 1 -- start start 2024-03-23T14:57:30.366 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.361+0000 7fa951dc2640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa94c0b4710 0x7fa94c1e3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:30.366 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.361+0000 7fa951dc2640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa94c0b6360 0x7fa94c06b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:30.366 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.361+0000 7fa951dc2640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa94c0b6740 0x7fa94c06b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:30.366 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.361+0000 7fa951dc2640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa94c06c890 con 0x7fa94c0b4710 2024-03-23T14:57:30.366 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.361+0000 7fa951dc2640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa94c1e3e80 con 0x7fa94c0b6360 2024-03-23T14:57:30.367 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.361+0000 7fa951dc2640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fa94c1e3b70 con 0x7fa94c0b6740 2024-03-23T14:57:30.367 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.361+0000 7fa94b7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa94c0b4710 0x7fa94c1e3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:30.367 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.361+0000 7fa94b7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa94c0b4710 0x7fa94c1e3630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:49464/0 (socket says 172.21.15.164:49464) 2024-03-23T14:57:30.367 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.361+0000 7fa94b7fe640 1 -- 172.21.15.164:0/3754031532 learned_addr learned my addr 172.21.15.164:0/3754031532 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:30.367 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.361+0000 7fa94affd640 1 --2- 172.21.15.164:0/3754031532 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa94c0b6360 0x7fa94c06b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:30.367 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.361+0000 7fa94bfff640 1 --2- 172.21.15.164:0/3754031532 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa94c0b6740 0x7fa94c06b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:30.367 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.361+0000 7fa94b7fe640 1 -- 172.21.15.164:0/3754031532 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa94c0b6740 msgr2=0x7fa94c06b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:30.367 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.361+0000 7fa94b7fe640 1 --2- 172.21.15.164:0/3754031532 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa94c0b6740 0x7fa94c06b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:30.367 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.361+0000 7fa94b7fe640 1 -- 172.21.15.164:0/3754031532 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa94c0b6360 msgr2=0x7fa94c06b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:30.367 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.361+0000 7fa94b7fe640 1 --2- 172.21.15.164:0/3754031532 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa94c0b6360 0x7fa94c06b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:30.367 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.361+0000 7fa94b7fe640 1 -- 172.21.15.164:0/3754031532 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa940009a70 con 0x7fa94c0b4710 2024-03-23T14:57:30.367 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.361+0000 7fa94bfff640 1 --2- 172.21.15.164:0/3754031532 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa94c0b6740 0x7fa94c06b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T14:57:30.367 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.361+0000 7fa94b7fe640 1 --2- 172.21.15.164:0/3754031532 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa94c0b4710 0x7fa94c1e3630 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa93c00c450 tx=0x7fa93c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:30.367 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.361+0000 7fa933fff640 1 -- 172.21.15.164:0/3754031532 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa93c005cf0 con 0x7fa94c0b4710 2024-03-23T14:57:30.367 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.361+0000 7fa933fff640 1 -- 172.21.15.164:0/3754031532 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa93c005e50 con 0x7fa94c0b4710 2024-03-23T14:57:30.368 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.361+0000 7fa951dc2640 1 -- 172.21.15.164:0/3754031532 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa94c133940 con 0x7fa94c0b4710 2024-03-23T14:57:30.368 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.361+0000 7fa933fff640 1 -- 172.21.15.164:0/3754031532 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa93c007290 con 0x7fa94c0b4710 2024-03-23T14:57:30.368 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.361+0000 7fa951dc2640 1 -- 172.21.15.164:0/3754031532 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa94c14fad0 con 0x7fa94c0b4710 2024-03-23T14:57:30.369 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.361+0000 7fa951dc2640 1 -- 172.21.15.164:0/3754031532 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa94c06d370 con 0x7fa94c0b4710 2024-03-23T14:57:30.369 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.361+0000 7fa933fff640 1 -- 172.21.15.164:0/3754031532 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7fa93c0073f0 con 0x7fa94c0b4710 2024-03-23T14:57:30.369 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.361+0000 7fa933fff640 1 --2- 172.21.15.164:0/3754031532 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fa9380672a0 0x7fa938069760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:30.369 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.361+0000 7fa933fff640 1 -- 172.21.15.164:0/3754031532 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa93c0844f0 con 0x7fa94c0b4710 2024-03-23T14:57:30.369 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.365+0000 7fa94affd640 1 --2- 172.21.15.164:0/3754031532 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fa9380672a0 0x7fa938069760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:30.369 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.365+0000 7fa94affd640 1 --2- 172.21.15.164:0/3754031532 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fa9380672a0 0x7fa938069760 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa9400af000 tx=0x7fa9400a7070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:30.372 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.369+0000 7fa933fff640 1 -- 172.21.15.164:0/3754031532 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7fa93c051130 con 0x7fa94c0b4710 2024-03-23T14:57:30.555 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.549+0000 7fa951dc2640 1 -- 172.21.15.164:0/3754031532 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa94c06b1a0 con 0x7fa94c0b4710 2024-03-23T14:57:30.555 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.553+0000 7fa933fff640 1 -- 172.21.15.164:0/3754031532 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+3533 (secure 0 0 0) 0x7fa93c017de0 con 0x7fa94c0b4710 2024-03-23T14:57:30.556 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-23T14:57:30.556 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/3352372536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":3352372536},{"type":"v1","addr":"172.21.15.164:6835","nonce":3352372536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-23T14:56:58.953326+0000","modified":"2024-03-23T14:57:00.999623+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4279":{"gid":4279,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.173:6833/3506402771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":3506402771},{"type":"v1","addr":"172.21.15.173:6833","nonce":3506402771}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4279,"qdb_cluster":[4279]},"id":1}]} 2024-03-23T14:57:30.556 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 6 2024-03-23T14:57:30.558 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.553+0000 7fa951dc2640 1 -- 172.21.15.164:0/3754031532 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fa9380672a0 msgr2=0x7fa938069760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:30.559 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.553+0000 7fa951dc2640 1 --2- 172.21.15.164:0/3754031532 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fa9380672a0 0x7fa938069760 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa9400af000 tx=0x7fa9400a7070 comp rx=0 tx=0).stop 2024-03-23T14:57:30.559 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.553+0000 7fa951dc2640 1 -- 172.21.15.164:0/3754031532 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa94c0b4710 msgr2=0x7fa94c1e3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:30.559 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.553+0000 7fa951dc2640 1 --2- 172.21.15.164:0/3754031532 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa94c0b4710 0x7fa94c1e3630 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa93c00c450 tx=0x7fa93c00c910 comp rx=0 tx=0).stop 2024-03-23T14:57:30.559 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.553+0000 7fa951dc2640 1 -- 172.21.15.164:0/3754031532 shutdown_connections 2024-03-23T14:57:30.559 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.553+0000 7fa951dc2640 1 --2- 172.21.15.164:0/3754031532 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa94c0b6740 0x7fa94c06b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:30.559 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.553+0000 7fa951dc2640 1 --2- 172.21.15.164:0/3754031532 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fa9380672a0 0x7fa938069760 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:30.559 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.553+0000 7fa951dc2640 1 --2- 172.21.15.164:0/3754031532 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa94c0b6360 0x7fa94c06b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:30.559 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.553+0000 7fa951dc2640 1 --2- 172.21.15.164:0/3754031532 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa94c0b4710 0x7fa94c1e3630 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:30.559 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.553+0000 7fa951dc2640 1 -- 172.21.15.164:0/3754031532 >> 172.21.15.164:0/3754031532 conn(0x7fa94c0b7630 msgr2=0x7fa94c0b7a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:30.559 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.553+0000 7fa951dc2640 1 -- 172.21.15.164:0/3754031532 shutdown_connections 2024-03-23T14:57:30.559 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.553+0000 7fa951dc2640 1 -- 172.21.15.164:0/3754031532 wait complete. 2024-03-23T14:57:30.574 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T14:57:30.692 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.681+0000 7f4d7f7fb640 1 Processor -- start 2024-03-23T14:57:30.692 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.685+0000 7f4d7f7fb640 1 -- start start 2024-03-23T14:57:30.692 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.685+0000 7f4d7f7fb640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4d78148cf0 0x7f4d78149180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:30.692 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.685+0000 7f4d7f7fb640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f4d7814db70 0x7f4d7814ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:30.692 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.685+0000 7f4d7f7fb640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d780b51d0 con 0x7f4d780b0230 2024-03-23T14:57:30.692 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.685+0000 7f4d7f7fb640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 -- 0x7f4d780b0610 con 0x7f4d7814db70 2024-03-23T14:57:30.692 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.685+0000 7f4d7f7fb640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f4d780b1500 con 0x7f4d78148cf0 2024-03-23T14:57:30.692 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.685+0000 7f4d7d597640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f4d780b0230 0x7f4d780b34d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:42562/0 (socket says 172.21.15.164:42562) 2024-03-23T14:57:30.692 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.685+0000 7f4d7d597640 1 -- 172.21.15.164:0/1189118755 learned_addr learned my addr 172.21.15.164:0/1189118755 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-03-23T14:57:30.692 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.685+0000 7f4d7cd96640 1 --2- 172.21.15.164:0/1189118755 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4d78148cf0 0x7f4d78149180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:30.692 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.685+0000 7f4d7dd98640 1 --2- 172.21.15.164:0/1189118755 >> v2:172.21.15.173:3300/0 conn(0x7f4d7814db70 0x7f4d7814ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:30.692 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.685+0000 7f4d6ffff640 1 -- 172.21.15.164:0/1189118755 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1752646759 0 0) 0x7f4d780b51d0 con 0x7f4d780b0230 2024-03-23T14:57:30.692 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.685+0000 7f4d6ffff640 1 -- 172.21.15.164:0/1189118755 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4d64003530 con 0x7f4d780b0230 2024-03-23T14:57:30.693 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.685+0000 7f4d6ffff640 1 -- 172.21.15.164:0/1189118755 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1785189568 0 0) 0x7f4d64003530 con 0x7f4d780b0230 2024-03-23T14:57:30.693 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.685+0000 7f4d6ffff640 1 -- 172.21.15.164:0/1189118755 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4d78148cf0 msgr2=0x7f4d78149180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:30.693 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.685+0000 7f4d6ffff640 1 --2- 172.21.15.164:0/1189118755 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4d78148cf0 0x7f4d78149180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:30.693 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.685+0000 7f4d6ffff640 1 -- 172.21.15.164:0/1189118755 >> v2:172.21.15.173:3300/0 conn(0x7f4d7814db70 msgr2=0x7f4d7814ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:30.693 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.685+0000 7f4d6ffff640 1 --2- 172.21.15.164:0/1189118755 >> v2:172.21.15.173:3300/0 conn(0x7f4d7814db70 0x7f4d7814ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:30.693 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.685+0000 7f4d6ffff640 1 -- 172.21.15.164:0/1189118755 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d7806d250 con 0x7f4d780b0230 2024-03-23T14:57:30.693 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.685+0000 7f4d6ffff640 1 -- 172.21.15.164:0/1189118755 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2526390548 0 0) 0x7f4d74001680 con 0x7f4d780b0230 2024-03-23T14:57:30.693 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.685+0000 7f4d6ffff640 1 -- 172.21.15.164:0/1189118755 >> v1:172.21.15.164:6789/0 conn(0x7f4d780b0230 legacy=0x7f4d780b34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:30.693 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.685+0000 7f4d6ffff640 1 --2- 172.21.15.164:0/1189118755 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4d64003980 0x7f4d64003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:30.693 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.685+0000 7f4d6ffff640 1 --2- 172.21.15.164:0/1189118755 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4d640047d0 0x7f4d64004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:30.693 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.685+0000 7f4d6ffff640 1 --2- 172.21.15.164:0/1189118755 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4d78148cf0 0x7f4d64005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:30.693 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.685+0000 7f4d6ffff640 1 -- 172.21.15.164:0/1189118755 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f4d780b0610 con 0x7f4d64003980 2024-03-23T14:57:30.693 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.685+0000 7f4d6ffff640 1 -- 172.21.15.164:0/1189118755 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f4d780b1500 con 0x7f4d640047d0 2024-03-23T14:57:30.693 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.685+0000 7f4d6ffff640 1 -- 172.21.15.164:0/1189118755 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f4d6400b9c0 con 0x7f4d78148cf0 2024-03-23T14:57:30.693 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.685+0000 7f4d7d597640 1 --2- 172.21.15.164:0/1189118755 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4d64003980 0x7f4d64003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:30.693 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.685+0000 7f4d7d597640 1 -- 172.21.15.164:0/1189118755 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4d78148cf0 msgr2=0x7f4d64005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:30.694 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.685+0000 7f4d7d597640 1 --2- 172.21.15.164:0/1189118755 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4d78148cf0 0x7f4d64005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:30.694 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.685+0000 7f4d7d597640 1 -- 172.21.15.164:0/1189118755 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4d640047d0 msgr2=0x7f4d64004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:30.694 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.685+0000 7f4d7d597640 1 --2- 172.21.15.164:0/1189118755 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4d640047d0 0x7f4d64004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:30.694 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.685+0000 7f4d7d597640 1 -- 172.21.15.164:0/1189118755 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d64003530 con 0x7f4d64003980 2024-03-23T14:57:30.694 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.689+0000 7f4d7d597640 1 --2- 172.21.15.164:0/1189118755 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4d64003980 0x7f4d64003da0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f4d7400c240 tx=0x7f4d7401cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=667d3c1eb5f8adb0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:30.694 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.689+0000 7f4d6ffff640 1 -- 172.21.15.164:0/1189118755 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d740058e0 con 0x7f4d64003980 2024-03-23T14:57:30.694 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.689+0000 7f4d6ffff640 1 -- 172.21.15.164:0/1189118755 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d74005a40 con 0x7f4d64003980 2024-03-23T14:57:30.694 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.689+0000 7f4d6ffff640 1 -- 172.21.15.164:0/1189118755 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d74005dd0 con 0x7f4d64003980 2024-03-23T14:57:30.697 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.689+0000 7f4d7f7fb640 1 -- 172.21.15.164:0/1189118755 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4d64003980 msgr2=0x7f4d64003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:30.697 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.689+0000 7f4d7f7fb640 1 --2- 172.21.15.164:0/1189118755 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4d64003980 0x7f4d64003da0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f4d7400c240 tx=0x7f4d7401cbc0 comp rx=0 tx=0).stop 2024-03-23T14:57:30.697 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.689+0000 7f4d7dd98640 1 -- 172.21.15.164:0/1189118755 reap_dead start 2024-03-23T14:57:30.697 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.689+0000 7f4d7f7fb640 1 -- 172.21.15.164:0/1189118755 shutdown_connections 2024-03-23T14:57:30.697 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.689+0000 7f4d7f7fb640 1 -- 172.21.15.164:0/1189118755 >> 172.21.15.164:0/1189118755 conn(0x7f4d780b8ca0 msgr2=0x7f4d780b90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:30.698 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.689+0000 7f4d7f7fb640 1 -- 172.21.15.164:0/1189118755 shutdown_connections 2024-03-23T14:57:30.698 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.689+0000 7f4d7f7fb640 1 -- 172.21.15.164:0/1189118755 wait complete. 2024-03-23T14:57:30.698 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.689+0000 7f4d7f7fb640 1 Processor -- start 2024-03-23T14:57:30.698 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.689+0000 7f4d7f7fb640 1 -- start start 2024-03-23T14:57:30.698 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.689+0000 7f4d7f7fb640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4d64003980 0x7f4d78144f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:30.698 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.689+0000 7f4d7f7fb640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4d78071c80 0x7f4d7814ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:30.698 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.689+0000 7f4d7f7fb640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4d7806f4e0 0x7f4d7813ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:30.698 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.689+0000 7f4d7f7fb640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f4d78072230 con 0x7f4d64003980 2024-03-23T14:57:30.698 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.689+0000 7f4d7f7fb640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f4d780712c0 con 0x7f4d78071c80 2024-03-23T14:57:30.698 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.689+0000 7f4d7f7fb640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f4d7806f990 con 0x7f4d7806f4e0 2024-03-23T14:57:30.698 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.689+0000 7f4d7d597640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4d64003980 0x7f4d78144f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:30.698 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.689+0000 7f4d7d597640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4d64003980 0x7f4d78144f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:49474/0 (socket says 172.21.15.164:49474) 2024-03-23T14:57:30.698 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.689+0000 7f4d7d597640 1 -- 172.21.15.164:0/487907161 learned_addr learned my addr 172.21.15.164:0/487907161 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:30.698 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.689+0000 7f4d7cd96640 1 --2- 172.21.15.164:0/487907161 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4d78071c80 0x7f4d7814ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:30.698 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.689+0000 7f4d7dd98640 1 --2- 172.21.15.164:0/487907161 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4d7806f4e0 0x7f4d7813ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:30.699 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.689+0000 7f4d7d597640 1 -- 172.21.15.164:0/487907161 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4d7806f4e0 msgr2=0x7f4d7813ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:30.699 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.689+0000 7f4d7d597640 1 --2- 172.21.15.164:0/487907161 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4d7806f4e0 0x7f4d7813ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:30.699 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.689+0000 7f4d7d597640 1 -- 172.21.15.164:0/487907161 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4d78071c80 msgr2=0x7f4d7814ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:30.699 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.689+0000 7f4d7d597640 1 --2- 172.21.15.164:0/487907161 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4d78071c80 0x7f4d7814ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:30.699 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.689+0000 7f4d7d597640 1 -- 172.21.15.164:0/487907161 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d780b0f70 con 0x7f4d64003980 2024-03-23T14:57:30.699 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.689+0000 7f4d7d597640 1 --2- 172.21.15.164:0/487907161 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4d64003980 0x7f4d78144f20 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f4d7400c240 tx=0x7f4d64005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:30.699 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.689+0000 7f4d6dffb640 1 -- 172.21.15.164:0/487907161 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d740047f0 con 0x7f4d64003980 2024-03-23T14:57:30.699 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.689+0000 7f4d6dffb640 1 -- 172.21.15.164:0/487907161 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d74004950 con 0x7f4d64003980 2024-03-23T14:57:30.699 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.689+0000 7f4d7f7fb640 1 -- 172.21.15.164:0/487907161 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d7809dc30 con 0x7f4d64003980 2024-03-23T14:57:30.699 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.689+0000 7f4d6dffb640 1 -- 172.21.15.164:0/487907161 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d74004cc0 con 0x7f4d64003980 2024-03-23T14:57:30.699 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.689+0000 7f4d7f7fb640 1 -- 172.21.15.164:0/487907161 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d780703a0 con 0x7f4d64003980 2024-03-23T14:57:30.699 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.693+0000 7f4d7f7fb640 1 -- 172.21.15.164:0/487907161 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d78070200 con 0x7f4d64003980 2024-03-23T14:57:30.699 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.693+0000 7f4d6dffb640 1 -- 172.21.15.164:0/487907161 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7f4d740a57d0 con 0x7f4d64003980 2024-03-23T14:57:30.699 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.693+0000 7f4d6dffb640 1 --2- 172.21.15.164:0/487907161 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f4d5c0672d0 0x7f4d5c069790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:30.699 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.693+0000 7f4d7cd96640 1 --2- 172.21.15.164:0/487907161 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f4d5c0672d0 0x7f4d5c069790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:30.699 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.693+0000 7f4d6dffb640 1 -- 172.21.15.164:0/487907161 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f4d7411e960 con 0x7f4d64003980 2024-03-23T14:57:30.700 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.693+0000 7f4d7cd96640 1 --2- 172.21.15.164:0/487907161 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f4d5c0672d0 0x7f4d5c069790 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f4d780727e0 tx=0x7f4d68008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:30.701 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.697+0000 7f4d6dffb640 1 -- 172.21.15.164:0/487907161 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7f4d78070200 con 0x7f4d64003980 2024-03-23T14:57:30.892 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.889+0000 7f4d7f7fb640 1 -- 172.21.15.164:0/487907161 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4d78070200 con 0x7f4d64003980 2024-03-23T14:57:30.893 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.889+0000 7f4d6dffb640 1 -- 172.21.15.164:0/487907161 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+3533 (secure 0 0 0) 0x7f4d78070200 con 0x7f4d64003980 2024-03-23T14:57:30.894 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-23T14:57:30.894 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/3352372536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":3352372536},{"type":"v1","addr":"172.21.15.164:6835","nonce":3352372536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-23T14:56:58.953326+0000","modified":"2024-03-23T14:57:00.999623+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4279":{"gid":4279,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.173:6833/3506402771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":3506402771},{"type":"v1","addr":"172.21.15.173:6833","nonce":3506402771}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4279,"qdb_cluster":[4279]},"id":1}]} 2024-03-23T14:57:30.894 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 6 2024-03-23T14:57:30.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.893+0000 7f4d7f7fb640 1 -- 172.21.15.164:0/487907161 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f4d5c0672d0 msgr2=0x7f4d5c069790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:30.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.893+0000 7f4d7f7fb640 1 --2- 172.21.15.164:0/487907161 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f4d5c0672d0 0x7f4d5c069790 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f4d780727e0 tx=0x7f4d68008040 comp rx=0 tx=0).stop 2024-03-23T14:57:30.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.893+0000 7f4d7f7fb640 1 -- 172.21.15.164:0/487907161 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4d64003980 msgr2=0x7f4d78144f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:30.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.893+0000 7f4d7f7fb640 1 --2- 172.21.15.164:0/487907161 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4d64003980 0x7f4d78144f20 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f4d7400c240 tx=0x7f4d64005800 comp rx=0 tx=0).stop 2024-03-23T14:57:30.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.893+0000 7f4d7f7fb640 1 -- 172.21.15.164:0/487907161 shutdown_connections 2024-03-23T14:57:30.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.893+0000 7f4d7f7fb640 1 --2- 172.21.15.164:0/487907161 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4d7806f4e0 0x7f4d7813ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:30.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.893+0000 7f4d7f7fb640 1 --2- 172.21.15.164:0/487907161 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f4d5c0672d0 0x7f4d5c069790 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:30.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.893+0000 7f4d7f7fb640 1 --2- 172.21.15.164:0/487907161 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4d78071c80 0x7f4d7814ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:30.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.893+0000 7f4d7f7fb640 1 --2- 172.21.15.164:0/487907161 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4d64003980 0x7f4d78144f20 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:30.897 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.893+0000 7f4d7f7fb640 1 -- 172.21.15.164:0/487907161 >> 172.21.15.164:0/487907161 conn(0x7f4d780b8ca0 msgr2=0x7f4d780b43f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:30.897 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.893+0000 7f4d7f7fb640 1 -- 172.21.15.164:0/487907161 shutdown_connections 2024-03-23T14:57:30.897 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:30.893+0000 7f4d7f7fb640 1 -- 172.21.15.164:0/487907161 wait complete. 2024-03-23T14:57:30.912 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T14:57:31.035 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.029+0000 7f83475e5640 1 Processor -- start 2024-03-23T14:57:31.035 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.029+0000 7f83475e5640 1 -- start start 2024-03-23T14:57:31.036 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.033+0000 7f83475e5640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f8340148cf0 0x7f8340149180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:31.037 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.033+0000 7f83475e5640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f834014db70 0x7f834014ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:31.037 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.033+0000 7f83475e5640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83400b51d0 con 0x7f83400b0230 2024-03-23T14:57:31.037 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.033+0000 7f83475e5640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 -- 0x7f83400b0610 con 0x7f8340148cf0 2024-03-23T14:57:31.037 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.033+0000 7f83475e5640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f83400b1500 con 0x7f834014db70 2024-03-23T14:57:31.037 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.033+0000 7f8345381640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f83400b0230 0x7f83400b34d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:42566/0 (socket says 172.21.15.164:42566) 2024-03-23T14:57:31.037 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.033+0000 7f8345381640 1 -- 172.21.15.164:0/4210889818 learned_addr learned my addr 172.21.15.164:0/4210889818 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-03-23T14:57:31.037 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.033+0000 7f8345b82640 1 --2- 172.21.15.164:0/4210889818 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f834014db70 0x7f834014ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:31.037 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.033+0000 7f8344b80640 1 --2- 172.21.15.164:0/4210889818 >> v2:172.21.15.173:3300/0 conn(0x7f8340148cf0 0x7f8340149180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:31.037 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.033+0000 7f8337fff640 1 -- 172.21.15.164:0/4210889818 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1434016841 0 0) 0x7f83400b51d0 con 0x7f83400b0230 2024-03-23T14:57:31.037 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.033+0000 7f8337fff640 1 -- 172.21.15.164:0/4210889818 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f832c003530 con 0x7f83400b0230 2024-03-23T14:57:31.037 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.033+0000 7f8337fff640 1 -- 172.21.15.164:0/4210889818 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1287778998 0 0) 0x7f832c003530 con 0x7f83400b0230 2024-03-23T14:57:31.037 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.033+0000 7f8345b82640 1 -- 172.21.15.164:0/4210889818 >> v2:172.21.15.173:3300/0 conn(0x7f8340148cf0 msgr2=0x7f8340149180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:31.037 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.033+0000 7f8345b82640 1 --2- 172.21.15.164:0/4210889818 >> v2:172.21.15.173:3300/0 conn(0x7f8340148cf0 0x7f8340149180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:31.037 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.033+0000 7f8345b82640 1 -- 172.21.15.164:0/4210889818 >> v1:172.21.15.164:6789/0 conn(0x7f83400b0230 legacy=0x7f83400b34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:31.037 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.033+0000 7f8345b82640 1 -- 172.21.15.164:0/4210889818 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f834006d250 con 0x7f834014db70 2024-03-23T14:57:31.038 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.033+0000 7f8345b82640 1 --2- 172.21.15.164:0/4210889818 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f834014db70 0x7f834014ff60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f8338009d90 tx=0x7f833801c7d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=534fc215e560b86a server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:31.039 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.033+0000 7f8337fff640 1 -- 172.21.15.164:0/4210889818 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f833c001680 con 0x7f834014db70 2024-03-23T14:57:31.039 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.033+0000 7f8337fff640 1 -- 172.21.15.164:0/4210889818 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83380a55d0 con 0x7f834014db70 2024-03-23T14:57:31.039 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.033+0000 7f8337fff640 1 -- 172.21.15.164:0/4210889818 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83380a5940 con 0x7f834014db70 2024-03-23T14:57:31.040 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.033+0000 7f83475e5640 1 -- 172.21.15.164:0/4210889818 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f834014db70 msgr2=0x7f834014ff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:31.040 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.033+0000 7f83475e5640 1 --2- 172.21.15.164:0/4210889818 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f834014db70 0x7f834014ff60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f8338009d90 tx=0x7f833801c7d0 comp rx=0 tx=0).stop 2024-03-23T14:57:31.040 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.037+0000 7f83475e5640 1 -- 172.21.15.164:0/4210889818 shutdown_connections 2024-03-23T14:57:31.040 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.037+0000 7f83475e5640 1 --2- 172.21.15.164:0/4210889818 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f834014db70 0x7f834014ff60 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:31.040 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.037+0000 7f83475e5640 1 --2- 172.21.15.164:0/4210889818 >> v2:172.21.15.173:3300/0 conn(0x7f8340148cf0 0x7f8340149180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:31.040 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.037+0000 7f83475e5640 1 -- 172.21.15.164:0/4210889818 >> 172.21.15.164:0/4210889818 conn(0x7f83400b8ca0 msgr2=0x7f83400b90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:31.040 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.037+0000 7f83475e5640 1 -- 172.21.15.164:0/4210889818 shutdown_connections 2024-03-23T14:57:31.040 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.037+0000 7f83475e5640 1 -- 172.21.15.164:0/4210889818 wait complete. 2024-03-23T14:57:31.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.037+0000 7f83475e5640 1 Processor -- start 2024-03-23T14:57:31.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.037+0000 7f83475e5640 1 -- start start 2024-03-23T14:57:31.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.037+0000 7f83475e5640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f83400b0230 0x7f834014f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:31.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.037+0000 7f83475e5640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8340148cf0 0x7f83401dd270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:31.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.037+0000 7f83475e5640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f834014db70 0x7f834013e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:31.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.037+0000 7f83475e5640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f83401392f0 con 0x7f834014db70 2024-03-23T14:57:31.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.037+0000 7f83475e5640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f83401397d0 con 0x7f83400b0230 2024-03-23T14:57:31.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.037+0000 7f83475e5640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f8340142cf0 con 0x7f8340148cf0 2024-03-23T14:57:31.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.037+0000 7f8345b82640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f834014db70 0x7f834013e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:31.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.037+0000 7f8345b82640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f834014db70 0x7f834013e1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:49482/0 (socket says 172.21.15.164:49482) 2024-03-23T14:57:31.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.037+0000 7f8345b82640 1 -- 172.21.15.164:0/839363020 learned_addr learned my addr 172.21.15.164:0/839363020 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:31.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.037+0000 7f8345381640 1 --2- 172.21.15.164:0/839363020 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f83400b0230 0x7f834014f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:31.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.037+0000 7f8344b80640 1 --2- 172.21.15.164:0/839363020 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8340148cf0 0x7f83401dd270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:31.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.037+0000 7f8345b82640 1 -- 172.21.15.164:0/839363020 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8340148cf0 msgr2=0x7f83401dd270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:31.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.037+0000 7f8345b82640 1 --2- 172.21.15.164:0/839363020 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8340148cf0 0x7f83401dd270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:31.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.037+0000 7f8345b82640 1 -- 172.21.15.164:0/839363020 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f83400b0230 msgr2=0x7f834014f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:31.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.037+0000 7f8345b82640 1 --2- 172.21.15.164:0/839363020 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f83400b0230 0x7f834014f200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:31.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.037+0000 7f8345b82640 1 -- 172.21.15.164:0/839363020 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8338009a70 con 0x7f834014db70 2024-03-23T14:57:31.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.037+0000 7f8345381640 1 --2- 172.21.15.164:0/839363020 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f83400b0230 0x7f834014f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T14:57:31.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.037+0000 7f8345b82640 1 --2- 172.21.15.164:0/839363020 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f834014db70 0x7f834013e1d0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f83380b0040 tx=0x7f8338019570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:31.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.037+0000 7f8335ffb640 1 -- 172.21.15.164:0/839363020 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8338007000 con 0x7f834014db70 2024-03-23T14:57:31.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.037+0000 7f8335ffb640 1 -- 172.21.15.164:0/839363020 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83380ad850 con 0x7f834014db70 2024-03-23T14:57:31.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.037+0000 7f83475e5640 1 -- 172.21.15.164:0/839363020 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f834014f9c0 con 0x7f834014db70 2024-03-23T14:57:31.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.037+0000 7f8335ffb640 1 -- 172.21.15.164:0/839363020 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83380adc20 con 0x7f834014db70 2024-03-23T14:57:31.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.037+0000 7f83475e5640 1 -- 172.21.15.164:0/839363020 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83401e0c40 con 0x7f834014db70 2024-03-23T14:57:31.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.037+0000 7f83475e5640 1 -- 172.21.15.164:0/839363020 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83400690a0 con 0x7f834014db70 2024-03-23T14:57:31.045 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.037+0000 7f8335ffb640 1 -- 172.21.15.164:0/839363020 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7f83380b4050 con 0x7f834014db70 2024-03-23T14:57:31.045 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.041+0000 7f8335ffb640 1 --2- 172.21.15.164:0/839363020 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f83240672a0 0x7f8324069760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:31.045 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.041+0000 7f8335ffb640 1 -- 172.21.15.164:0/839363020 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f833811fcb0 con 0x7f834014db70 2024-03-23T14:57:31.045 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.041+0000 7f8345381640 1 --2- 172.21.15.164:0/839363020 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f83240672a0 0x7f8324069760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:31.045 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.041+0000 7f8345381640 1 --2- 172.21.15.164:0/839363020 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f83240672a0 0x7f8324069760 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f833c0079c0 tx=0x7f833c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:31.048 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.045+0000 7f8335ffb640 1 -- 172.21.15.164:0/839363020 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7f83380f0540 con 0x7f834014db70 2024-03-23T14:57:31.200 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.193+0000 7f83475e5640 1 -- 172.21.15.164:0/839363020 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f83400b2fa0 con 0x7f834014db70 2024-03-23T14:57:31.201 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.197+0000 7f8335ffb640 1 -- 172.21.15.164:0/839363020 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16629 (secure 0 0 0) 0x7f83380b2090 con 0x7f834014db70 2024-03-23T14:57:31.202 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-23T14:57:31.202 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":20,"fsid":"45c7911b-b69d-4e68-907b-fbfeb780aa3b","created":"2024-03-23T14:56:34.367633+0000","modified":"2024-03-23T14:57:00.611265+0000","last_up_change":"2024-03-23T14:56:46.297403+0000","last_in_change":"2024-03-23T14:56:36.629884+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T14:56:47.186157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-23T14:56:56.994057+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-23T14:56:57.953862+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"80abdc62-bd0d-4e60-a554-7ed6fd67cf54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":984716690},{"type":"v1","addr":"172.21.15.164:6801","nonce":984716690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":984716690},{"type":"v1","addr":"172.21.15.164:6803","nonce":984716690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":984716690},{"type":"v1","addr":"172.21.15.164:6807","nonce":984716690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":984716690},{"type":"v1","addr":"172.21.15.164:6805","nonce":984716690}]},"public_addr":"172.21.15.164:6801/984716690","cluster_addr":"172.21.15.164:6803/984716690","heartbeat_back_addr":"172.21.15.164:6807/984716690","heartbeat_front_addr":"172.21.15.164:6805/984716690","state":["exists","up"]},{"osd":1,"uuid":"f9001b06-be4d-4f42-a0e6-bcef2afa0d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":1349657239},{"type":"v1","addr":"172.21.15.164:6817","nonce":1349657239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":1349657239},{"type":"v1","addr":"172.21.15.164:6819","nonce":1349657239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":1349657239},{"type":"v1","addr":"172.21.15.164:6823","nonce":1349657239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":1349657239},{"type":"v1","addr":"172.21.15.164:6821","nonce":1349657239}]},"public_addr":"172.21.15.164:6817/1349657239","cluster_addr":"172.21.15.164:6819/1349657239","heartbeat_back_addr":"172.21.15.164:6823/1349657239","heartbeat_front_addr":"172.21.15.164:6821/1349657239","state":["exists","up"]},{"osd":2,"uuid":"fa0aeb9f-8b54-4831-a188-3e361c7980d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":887861252},{"type":"v1","addr":"172.21.15.164:6809","nonce":887861252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":887861252},{"type":"v1","addr":"172.21.15.164:6811","nonce":887861252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":887861252},{"type":"v1","addr":"172.21.15.164:6815","nonce":887861252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":887861252},{"type":"v1","addr":"172.21.15.164:6813","nonce":887861252}]},"public_addr":"172.21.15.164:6809/887861252","cluster_addr":"172.21.15.164:6811/887861252","heartbeat_back_addr":"172.21.15.164:6815/887861252","heartbeat_front_addr":"172.21.15.164:6813/887861252","state":["exists","up"]},{"osd":3,"uuid":"7e18c957-3091-46d5-8e17-8f3354c1726c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":812540271},{"type":"v1","addr":"172.21.15.164:6825","nonce":812540271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":812540271},{"type":"v1","addr":"172.21.15.164:6827","nonce":812540271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":812540271},{"type":"v1","addr":"172.21.15.164:6831","nonce":812540271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":812540271},{"type":"v1","addr":"172.21.15.164:6829","nonce":812540271}]},"public_addr":"172.21.15.164:6825/812540271","cluster_addr":"172.21.15.164:6827/812540271","heartbeat_back_addr":"172.21.15.164:6831/812540271","heartbeat_front_addr":"172.21.15.164:6829/812540271","state":["exists","up"]},{"osd":4,"uuid":"8e83f315-42bf-4471-9ce8-d81948da0f9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":2656857421},{"type":"v1","addr":"172.21.15.173:6825","nonce":2656857421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2656857421},{"type":"v1","addr":"172.21.15.173:6827","nonce":2656857421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2656857421},{"type":"v1","addr":"172.21.15.173:6831","nonce":2656857421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":2656857421},{"type":"v1","addr":"172.21.15.173:6829","nonce":2656857421}]},"public_addr":"172.21.15.173:6825/2656857421","cluster_addr":"172.21.15.173:6827/2656857421","heartbeat_back_addr":"172.21.15.173:6831/2656857421","heartbeat_front_addr":"172.21.15.173:6829/2656857421","state":["exists","up"]},{"osd":5,"uuid":"a230c872-71f9-40ab-8999-defd0162d4ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":3676483185},{"type":"v1","addr":"172.21.15.173:6801","nonce":3676483185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":3676483185},{"type":"v1","addr":"172.21.15.173:6803","nonce":3676483185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":3676483185},{"type":"v1","addr":"172.21.15.173:6807","nonce":3676483185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":3676483185},{"type":"v1","addr":"172.21.15.173:6805","nonce":3676483185}]},"public_addr":"172.21.15.173:6801/3676483185","cluster_addr":"172.21.15.173:6803/3676483185","heartbeat_back_addr":"172.21.15.173:6807/3676483185","heartbeat_front_addr":"172.21.15.173:6805/3676483185","state":["exists","up"]},{"osd":6,"uuid":"6193bfd7-8432-4f7f-a117-a3fcee21af99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":3631684087},{"type":"v1","addr":"172.21.15.173:6809","nonce":3631684087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":3631684087},{"type":"v1","addr":"172.21.15.173:6811","nonce":3631684087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":3631684087},{"type":"v1","addr":"172.21.15.173:6815","nonce":3631684087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":3631684087},{"type":"v1","addr":"172.21.15.173:6813","nonce":3631684087}]},"public_addr":"172.21.15.173:6809/3631684087","cluster_addr":"172.21.15.173:6811/3631684087","heartbeat_back_addr":"172.21.15.173:6815/3631684087","heartbeat_front_addr":"172.21.15.173:6813/3631684087","state":["exists","up"]},{"osd":7,"uuid":"b0f1b80a-afc4-4538-83b9-f081d2dcb8cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":3261114948},{"type":"v1","addr":"172.21.15.173:6817","nonce":3261114948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":3261114948},{"type":"v1","addr":"172.21.15.173:6819","nonce":3261114948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":3261114948},{"type":"v1","addr":"172.21.15.173:6823","nonce":3261114948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":3261114948},{"type":"v1","addr":"172.21.15.173:6821","nonce":3261114948}]},"public_addr":"172.21.15.173:6817/3261114948","cluster_addr":"172.21.15.173:6819/3261114948","heartbeat_back_addr":"172.21.15.173:6823/3261114948","heartbeat_front_addr":"172.21.15.173:6821/3261114948","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.188390+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.159932+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.191101+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.181776+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.224365+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.177515+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.153631+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.139972+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T14:57:31.204 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.201+0000 7f83475e5640 1 -- 172.21.15.164:0/839363020 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f83240672a0 msgr2=0x7f8324069760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:31.204 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.201+0000 7f83475e5640 1 --2- 172.21.15.164:0/839363020 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f83240672a0 0x7f8324069760 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f833c0079c0 tx=0x7f833c009040 comp rx=0 tx=0).stop 2024-03-23T14:57:31.204 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.201+0000 7f83475e5640 1 -- 172.21.15.164:0/839363020 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f834014db70 msgr2=0x7f834013e1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:31.204 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.201+0000 7f83475e5640 1 --2- 172.21.15.164:0/839363020 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f834014db70 0x7f834013e1d0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f83380b0040 tx=0x7f8338019570 comp rx=0 tx=0).stop 2024-03-23T14:57:31.204 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.201+0000 7f83475e5640 1 -- 172.21.15.164:0/839363020 shutdown_connections 2024-03-23T14:57:31.204 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.201+0000 7f83475e5640 1 --2- 172.21.15.164:0/839363020 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f834014db70 0x7f834013e1d0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:31.204 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.201+0000 7f83475e5640 1 --2- 172.21.15.164:0/839363020 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8340148cf0 0x7f83401dd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:31.204 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.201+0000 7f83475e5640 1 --2- 172.21.15.164:0/839363020 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f83240672a0 0x7f8324069760 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:31.205 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.201+0000 7f83475e5640 1 --2- 172.21.15.164:0/839363020 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f83400b0230 0x7f834014f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:31.205 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.201+0000 7f83475e5640 1 -- 172.21.15.164:0/839363020 >> 172.21.15.164:0/839363020 conn(0x7f83400b8ca0 msgr2=0x7f83400b2350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:31.205 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.201+0000 7f83475e5640 1 -- 172.21.15.164:0/839363020 shutdown_connections 2024-03-23T14:57:31.205 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.201+0000 7f83475e5640 1 -- 172.21.15.164:0/839363020 wait complete. 2024-03-23T14:57:31.219 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-23T14:57:31.329 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.325+0000 7f08ed864640 1 Processor -- start 2024-03-23T14:57:31.329 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.325+0000 7f08ed864640 1 -- start start 2024-03-23T14:57:31.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.325+0000 7f08ed864640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f08e80b56f0 0x7f08e814a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:31.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.325+0000 7f08ed864640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f08e80b4a70 0x7f08e8140f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:31.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.325+0000 7f08ed864640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f08e8141470 0x7f08e80b75f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:31.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.325+0000 7f08ed864640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f08e80b5fa0 con 0x7f08e8141470 2024-03-23T14:57:31.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.325+0000 7f08ed864640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 -- 0x7f08e80b4300 con 0x7f08e80b4a70 2024-03-23T14:57:31.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.325+0000 7f08ed864640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f08e80b45e0 con 0x7f08e80b56f0 2024-03-23T14:57:31.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.325+0000 7f08e77fe640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f08e8141470 0x7f08e80b75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:31.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.325+0000 7f08e77fe640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f08e8141470 0x7f08e80b75f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:49490/0 (socket says 172.21.15.164:49490) 2024-03-23T14:57:31.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.325+0000 7f08e77fe640 1 -- 172.21.15.164:0/3328318255 learned_addr learned my addr 172.21.15.164:0/3328318255 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:31.331 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.325+0000 7f08e6ffd640 1 --2- 172.21.15.164:0/3328318255 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f08e80b56f0 0x7f08e814a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:31.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.325+0000 7f08e67fc640 1 --2- 172.21.15.164:0/3328318255 >> v2:172.21.15.173:3300/0 conn(0x7f08e80b4a70 0x7f08e8140f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:31.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.325+0000 7f08e77fe640 1 -- 172.21.15.164:0/3328318255 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f08e80b56f0 msgr2=0x7f08e814a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:31.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.325+0000 7f08e77fe640 1 --2- 172.21.15.164:0/3328318255 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f08e80b56f0 0x7f08e814a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:31.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.325+0000 7f08e77fe640 1 -- 172.21.15.164:0/3328318255 >> v2:172.21.15.173:3300/0 conn(0x7f08e80b4a70 msgr2=0x7f08e8140f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:31.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.325+0000 7f08e77fe640 1 --2- 172.21.15.164:0/3328318255 >> v2:172.21.15.173:3300/0 conn(0x7f08e80b4a70 0x7f08e8140f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:31.332 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.325+0000 7f08e77fe640 1 -- 172.21.15.164:0/3328318255 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08e814ade0 con 0x7f08e8141470 2024-03-23T14:57:31.333 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.329+0000 7f08e77fe640 1 --2- 172.21.15.164:0/3328318255 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f08e8141470 0x7f08e80b75f0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f08e80ba7b0 tx=0x7f08e001c710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be8dde012d9fc196 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:31.333 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.329+0000 7f08e5ffb640 1 -- 172.21.15.164:0/3328318255 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08e00a5590 con 0x7f08e8141470 2024-03-23T14:57:31.333 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.329+0000 7f08e5ffb640 1 -- 172.21.15.164:0/3328318255 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08e00a56f0 con 0x7f08e8141470 2024-03-23T14:57:31.333 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.329+0000 7f08e5ffb640 1 -- 172.21.15.164:0/3328318255 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08e00a5a80 con 0x7f08e8141470 2024-03-23T14:57:31.334 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.329+0000 7f08ed864640 1 -- 172.21.15.164:0/3328318255 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f08e8141470 msgr2=0x7f08e80b75f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:31.334 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.329+0000 7f08ed864640 1 --2- 172.21.15.164:0/3328318255 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f08e8141470 0x7f08e80b75f0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f08e80ba7b0 tx=0x7f08e001c710 comp rx=0 tx=0).stop 2024-03-23T14:57:31.334 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.329+0000 7f08ed864640 1 -- 172.21.15.164:0/3328318255 shutdown_connections 2024-03-23T14:57:31.334 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.329+0000 7f08ed864640 1 --2- 172.21.15.164:0/3328318255 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f08e8141470 0x7f08e80b75f0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:31.334 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.329+0000 7f08ed864640 1 --2- 172.21.15.164:0/3328318255 >> v2:172.21.15.173:3300/0 conn(0x7f08e80b4a70 0x7f08e8140f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:31.334 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.329+0000 7f08ed864640 1 --2- 172.21.15.164:0/3328318255 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f08e80b56f0 0x7f08e814a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:31.334 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.329+0000 7f08ed864640 1 -- 172.21.15.164:0/3328318255 >> 172.21.15.164:0/3328318255 conn(0x7f08e802c3c0 msgr2=0x7f08e802c7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:31.334 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.329+0000 7f08ed864640 1 -- 172.21.15.164:0/3328318255 shutdown_connections 2024-03-23T14:57:31.334 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.329+0000 7f08ed864640 1 -- 172.21.15.164:0/3328318255 wait complete. 2024-03-23T14:57:31.336 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.329+0000 7f08ed864640 1 Processor -- start 2024-03-23T14:57:31.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.329+0000 7f08ed864640 1 -- start start 2024-03-23T14:57:31.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.329+0000 7f08ed864640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f08e80b4a70 0x7f08e8148be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:31.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.329+0000 7f08ed864640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f08e80b56f0 0x7f08e8135260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:31.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.329+0000 7f08ed864640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f08e8141470 0x7f08e81dff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:31.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.329+0000 7f08ed864640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f08e814a4b0 con 0x7f08e80b56f0 2024-03-23T14:57:31.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.329+0000 7f08ed864640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f08e81e0440 con 0x7f08e80b4a70 2024-03-23T14:57:31.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.329+0000 7f08ed864640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f08e8148470 con 0x7f08e8141470 2024-03-23T14:57:31.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.329+0000 7f08e67fc640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f08e80b56f0 0x7f08e8135260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:31.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.329+0000 7f08e67fc640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f08e80b56f0 0x7f08e8135260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:49504/0 (socket says 172.21.15.164:49504) 2024-03-23T14:57:31.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.329+0000 7f08e67fc640 1 -- 172.21.15.164:0/2464556408 learned_addr learned my addr 172.21.15.164:0/2464556408 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:31.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.329+0000 7f08e6ffd640 1 --2- 172.21.15.164:0/2464556408 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f08e80b4a70 0x7f08e8148be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:31.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.329+0000 7f08e77fe640 1 --2- 172.21.15.164:0/2464556408 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f08e8141470 0x7f08e81dff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:31.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.329+0000 7f08e67fc640 1 -- 172.21.15.164:0/2464556408 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f08e8141470 msgr2=0x7f08e81dff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:31.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.329+0000 7f08e67fc640 1 --2- 172.21.15.164:0/2464556408 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f08e8141470 0x7f08e81dff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:31.337 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.329+0000 7f08e67fc640 1 -- 172.21.15.164:0/2464556408 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f08e80b4a70 msgr2=0x7f08e8148be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:31.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.329+0000 7f08e67fc640 1 --2- 172.21.15.164:0/2464556408 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f08e80b4a70 0x7f08e8148be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:31.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.329+0000 7f08e67fc640 1 -- 172.21.15.164:0/2464556408 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08e0009b30 con 0x7f08e80b56f0 2024-03-23T14:57:31.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.333+0000 7f08e67fc640 1 --2- 172.21.15.164:0/2464556408 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f08e80b56f0 0x7f08e8135260 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f08d800d5f0 tx=0x7f08d800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:31.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.333+0000 7f08cffff640 1 -- 172.21.15.164:0/2464556408 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08d8014620 con 0x7f08e80b56f0 2024-03-23T14:57:31.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.333+0000 7f08cffff640 1 -- 172.21.15.164:0/2464556408 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08d8014780 con 0x7f08e80b56f0 2024-03-23T14:57:31.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.333+0000 7f08ed864640 1 -- 172.21.15.164:0/2464556408 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08e814a6e0 con 0x7f08e80b56f0 2024-03-23T14:57:31.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.333+0000 7f08cffff640 1 -- 172.21.15.164:0/2464556408 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08d8014b30 con 0x7f08e80b56f0 2024-03-23T14:57:31.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.333+0000 7f08ed864640 1 -- 172.21.15.164:0/2464556408 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08e8133600 con 0x7f08e80b56f0 2024-03-23T14:57:31.338 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.333+0000 7f08ed864640 1 -- 172.21.15.164:0/2464556408 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08e80690a0 con 0x7f08e80b56f0 2024-03-23T14:57:31.339 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.333+0000 7f08cffff640 1 -- 172.21.15.164:0/2464556408 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7f08d8014c90 con 0x7f08e80b56f0 2024-03-23T14:57:31.339 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.333+0000 7f08cffff640 1 --2- 172.21.15.164:0/2464556408 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f08d40672a0 0x7f08d4069760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:31.339 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.333+0000 7f08cffff640 1 -- 172.21.15.164:0/2464556408 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f08d8085660 con 0x7f08e80b56f0 2024-03-23T14:57:31.339 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.333+0000 7f08e6ffd640 1 --2- 172.21.15.164:0/2464556408 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f08d40672a0 0x7f08d4069760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:31.339 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.333+0000 7f08e6ffd640 1 --2- 172.21.15.164:0/2464556408 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f08d40672a0 0x7f08d4069760 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f08dc006aa0 tx=0x7f08dc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:31.342 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.337+0000 7f08cffff640 1 -- 172.21.15.164:0/2464556408 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7f08d8052210 con 0x7f08e80b56f0 2024-03-23T14:57:31.520 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.517+0000 7f08ed864640 1 -- 172.21.15.164:0/2464556408 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f08e8148250 con 0x7f08e80b56f0 2024-03-23T14:57:31.521 INFO:tasks.ceph.mon.a.smithi164.stderr:2024-03-23T14:57:31.517+0000 7fd8f965e640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-23T14:57:31.524 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.521+0000 7f08cffff640 1 -- 172.21.15.164:0/2464556408 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v7) ==== 161+0+0 (secure 0 0 0) 0x7f08d80558d0 con 0x7f08e80b56f0 2024-03-23T14:57:31.525 INFO:teuthology.orchestra.run.smithi164.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-23T14:57:31.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.521+0000 7f08ed864640 1 -- 172.21.15.164:0/2464556408 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f08d40672a0 msgr2=0x7f08d4069760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:31.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.521+0000 7f08ed864640 1 --2- 172.21.15.164:0/2464556408 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f08d40672a0 0x7f08d4069760 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f08dc006aa0 tx=0x7f08dc008040 comp rx=0 tx=0).stop 2024-03-23T14:57:31.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.521+0000 7f08ed864640 1 -- 172.21.15.164:0/2464556408 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f08e80b56f0 msgr2=0x7f08e8135260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:31.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.521+0000 7f08ed864640 1 --2- 172.21.15.164:0/2464556408 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f08e80b56f0 0x7f08e8135260 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f08d800d5f0 tx=0x7f08d800dab0 comp rx=0 tx=0).stop 2024-03-23T14:57:31.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.521+0000 7f08ed864640 1 -- 172.21.15.164:0/2464556408 shutdown_connections 2024-03-23T14:57:31.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.521+0000 7f08ed864640 1 --2- 172.21.15.164:0/2464556408 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f08e8141470 0x7f08e81dff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:31.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.521+0000 7f08ed864640 1 --2- 172.21.15.164:0/2464556408 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f08e80b56f0 0x7f08e8135260 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:31.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.521+0000 7f08ed864640 1 --2- 172.21.15.164:0/2464556408 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f08d40672a0 0x7f08d4069760 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:31.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.521+0000 7f08ed864640 1 --2- 172.21.15.164:0/2464556408 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f08e80b4a70 0x7f08e8148be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:31.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.521+0000 7f08ed864640 1 -- 172.21.15.164:0/2464556408 >> 172.21.15.164:0/2464556408 conn(0x7f08e802c3c0 msgr2=0x7f08e813f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:31.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.525+0000 7f08ed864640 1 -- 172.21.15.164:0/2464556408 shutdown_connections 2024-03-23T14:57:31.528 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.525+0000 7f08ed864640 1 -- 172.21.15.164:0/2464556408 wait complete. 2024-03-23T14:57:31.545 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-23T14:57:31.573 INFO:tasks.ceph.mds.b.smithi173.stdout:starting mds.b at 2024-03-23T14:57:31.661 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.657+0000 7f653ab52640 1 Processor -- start 2024-03-23T14:57:31.661 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.657+0000 7f653ab52640 1 -- start start 2024-03-23T14:57:31.663 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.657+0000 7f653ab52640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f653414eb80 0x7f6534150f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:31.663 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.657+0000 7f653ab52640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65340b51d0 con 0x7f6534148cf0 2024-03-23T14:57:31.663 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.657+0000 7f653ab52640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65340b0610 con 0x7f65340b0230 2024-03-23T14:57:31.663 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.657+0000 7f653ab52640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f65340b1500 con 0x7f653414eb80 2024-03-23T14:57:31.663 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.657+0000 7f6533fff640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f6534148cf0 0x7f6534149180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:42570/0 (socket says 172.21.15.164:42570) 2024-03-23T14:57:31.663 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.657+0000 7f6533fff640 1 -- 172.21.15.164:0/330784496 learned_addr learned my addr 172.21.15.164:0/330784496 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-03-23T14:57:31.663 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.657+0000 7f65390ef640 1 --2- 172.21.15.164:0/330784496 >> v2:172.21.15.164:3300/0 conn(0x7f653414eb80 0x7f6534150f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:31.663 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.657+0000 7f65337fe640 1 -- 172.21.15.164:0/330784496 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1752014298 0 0) 0x7f65340b51d0 con 0x7f6534148cf0 2024-03-23T14:57:31.663 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.657+0000 7f65337fe640 1 -- 172.21.15.164:0/330784496 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6520003530 con 0x7f6534148cf0 2024-03-23T14:57:31.663 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.657+0000 7f65390ef640 1 -- 172.21.15.164:0/330784496 >> v1:172.21.15.173:6789/0 conn(0x7f65340b0230 legacy=0x7f65340b34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:31.663 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.657+0000 7f65337fe640 1 -- 172.21.15.164:0/330784496 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 347491030 0 0) 0x7f6520003530 con 0x7f6534148cf0 2024-03-23T14:57:31.663 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.657+0000 7f65390ef640 1 -- 172.21.15.164:0/330784496 >> v1:172.21.15.164:6789/0 conn(0x7f6534148cf0 legacy=0x7f6534149180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:31.664 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.657+0000 7f65390ef640 1 -- 172.21.15.164:0/330784496 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f653406d250 con 0x7f653414eb80 2024-03-23T14:57:31.666 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.661+0000 7f65390ef640 1 --2- 172.21.15.164:0/330784496 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f653414eb80 0x7f6534150f70 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f65340b0800 tx=0x7f652401c270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8c94f490dc7ab3 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:31.666 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.661+0000 7f65337fe640 1 -- 172.21.15.164:0/330784496 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f652c001680 con 0x7f653414eb80 2024-03-23T14:57:31.666 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.661+0000 7f65337fe640 1 -- 172.21.15.164:0/330784496 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65240a7050 con 0x7f653414eb80 2024-03-23T14:57:31.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.661+0000 7f65337fe640 1 -- 172.21.15.164:0/330784496 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65240ab620 con 0x7f653414eb80 2024-03-23T14:57:31.670 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.661+0000 7f653ab52640 1 -- 172.21.15.164:0/330784496 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f653414eb80 msgr2=0x7f6534150f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:31.670 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.661+0000 7f653ab52640 1 --2- 172.21.15.164:0/330784496 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f653414eb80 0x7f6534150f70 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f65340b0800 tx=0x7f652401c270 comp rx=0 tx=0).stop 2024-03-23T14:57:31.670 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.661+0000 7f653ab52640 1 -- 172.21.15.164:0/330784496 shutdown_connections 2024-03-23T14:57:31.670 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.661+0000 7f653ab52640 1 --2- 172.21.15.164:0/330784496 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f653414eb80 0x7f6534150f70 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:31.670 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.661+0000 7f653ab52640 1 -- 172.21.15.164:0/330784496 >> 172.21.15.164:0/330784496 conn(0x7f65340b8ca0 msgr2=0x7f65340b90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:31.670 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.661+0000 7f653ab52640 1 -- 172.21.15.164:0/330784496 shutdown_connections 2024-03-23T14:57:31.670 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.661+0000 7f653ab52640 1 -- 172.21.15.164:0/330784496 wait complete. 2024-03-23T14:57:31.670 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.665+0000 7f653ab52640 1 Processor -- start 2024-03-23T14:57:31.670 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.665+0000 7f653ab52640 1 -- start start 2024-03-23T14:57:31.670 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.665+0000 7f653ab52640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f65340b0230 0x7f6534144520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:31.670 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.665+0000 7f653ab52640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6534148cf0 0x7f65341e1a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:31.670 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.665+0000 7f653ab52640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f653414eb80 0x7f653409c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:31.671 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.665+0000 7f653ab52640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6534144a60 con 0x7f65340b0230 2024-03-23T14:57:31.671 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.665+0000 7f653ab52640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f65341e15b0 con 0x7f653414eb80 2024-03-23T14:57:31.671 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.665+0000 7f653ab52640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f65341e1f80 con 0x7f6534148cf0 2024-03-23T14:57:31.671 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.665+0000 7f65388ee640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f65340b0230 0x7f6534144520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:31.671 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.665+0000 7f65388ee640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f65340b0230 0x7f6534144520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:49510/0 (socket says 172.21.15.164:49510) 2024-03-23T14:57:31.671 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.665+0000 7f65388ee640 1 -- 172.21.15.164:0/2503946685 learned_addr learned my addr 172.21.15.164:0/2503946685 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:31.671 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.665+0000 7f6533fff640 1 --2- 172.21.15.164:0/2503946685 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6534148cf0 0x7f65341e1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:31.671 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.665+0000 7f65390ef640 1 --2- 172.21.15.164:0/2503946685 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f653414eb80 0x7f653409c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:31.671 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.665+0000 7f65388ee640 1 -- 172.21.15.164:0/2503946685 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6534148cf0 msgr2=0x7f65341e1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:31.671 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.665+0000 7f65388ee640 1 --2- 172.21.15.164:0/2503946685 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6534148cf0 0x7f65341e1a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:31.671 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.665+0000 7f65388ee640 1 -- 172.21.15.164:0/2503946685 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f653414eb80 msgr2=0x7f653409c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:31.671 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.665+0000 7f65388ee640 1 --2- 172.21.15.164:0/2503946685 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f653414eb80 0x7f653409c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:31.671 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.665+0000 7f65388ee640 1 -- 172.21.15.164:0/2503946685 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6524009420 con 0x7f65340b0230 2024-03-23T14:57:31.671 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.665+0000 7f6533fff640 1 --2- 172.21.15.164:0/2503946685 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6534148cf0 0x7f65341e1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T14:57:31.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.665+0000 7f65390ef640 1 --2- 172.21.15.164:0/2503946685 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f653414eb80 0x7f653409c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T14:57:31.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.665+0000 7f65388ee640 1 --2- 172.21.15.164:0/2503946685 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f65340b0230 0x7f6534144520 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f651c00c7f0 tx=0x7f651c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:31.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.665+0000 7f65317fa640 1 -- 172.21.15.164:0/2503946685 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f651c003b80 con 0x7f65340b0230 2024-03-23T14:57:31.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.665+0000 7f65317fa640 1 -- 172.21.15.164:0/2503946685 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f651c003ce0 con 0x7f65340b0230 2024-03-23T14:57:31.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.665+0000 7f65317fa640 1 -- 172.21.15.164:0/2503946685 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f651c00aa90 con 0x7f65340b0230 2024-03-23T14:57:31.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.665+0000 7f653ab52640 1 -- 172.21.15.164:0/2503946685 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65340b0f70 con 0x7f65340b0230 2024-03-23T14:57:31.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.665+0000 7f653ab52640 1 -- 172.21.15.164:0/2503946685 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65341e1240 con 0x7f65340b0230 2024-03-23T14:57:31.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.665+0000 7f653ab52640 1 -- 172.21.15.164:0/2503946685 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65341e10a0 con 0x7f65340b0230 2024-03-23T14:57:31.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.665+0000 7f65317fa640 1 -- 172.21.15.164:0/2503946685 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7f651c00abf0 con 0x7f65340b0230 2024-03-23T14:57:31.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.665+0000 7f65317fa640 1 --2- 172.21.15.164:0/2503946685 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f65140672d0 0x7f6514069790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:31.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.665+0000 7f6533fff640 1 --2- 172.21.15.164:0/2503946685 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f65140672d0 0x7f6514069790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:31.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.665+0000 7f65317fa640 1 -- 172.21.15.164:0/2503946685 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5795+0+0 (secure 0 0 0) 0x7f651c084e80 con 0x7f65340b0230 2024-03-23T14:57:31.673 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.669+0000 7f6533fff640 1 --2- 172.21.15.164:0/2503946685 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f65140672d0 0x7f6514069790 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f652c0011f0 tx=0x7f652c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:31.675 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.673+0000 7f65317fa640 1 -- 172.21.15.164:0/2503946685 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7f65341e10a0 con 0x7f65340b0230 2024-03-23T14:57:31.867 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:31.861+0000 7f653ab52640 1 -- 172.21.15.164:0/2503946685 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f65341e10a0 con 0x7f65340b0230 2024-03-23T14:57:32.527 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:32.521+0000 7f65317fa640 1 -- 172.21.15.164:0/2503946685 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v8) ==== 104+0+0 (secure 0 0 0) 0x7f65341e10a0 con 0x7f65340b0230 2024-03-23T14:57:32.530 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:32.525+0000 7f653ab52640 1 -- 172.21.15.164:0/2503946685 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f65140672d0 msgr2=0x7f6514069790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:32.530 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:32.525+0000 7f653ab52640 1 --2- 172.21.15.164:0/2503946685 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f65140672d0 0x7f6514069790 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f652c0011f0 tx=0x7f652c009040 comp rx=0 tx=0).stop 2024-03-23T14:57:32.530 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:32.525+0000 7f653ab52640 1 -- 172.21.15.164:0/2503946685 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f65340b0230 msgr2=0x7f6534144520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:32.530 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:32.525+0000 7f653ab52640 1 --2- 172.21.15.164:0/2503946685 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f65340b0230 0x7f6534144520 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f651c00c7f0 tx=0x7f651c00ccb0 comp rx=0 tx=0).stop 2024-03-23T14:57:32.531 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:32.525+0000 7f653ab52640 1 -- 172.21.15.164:0/2503946685 shutdown_connections 2024-03-23T14:57:32.531 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:32.525+0000 7f653ab52640 1 --2- 172.21.15.164:0/2503946685 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f65140672d0 0x7f6514069790 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:32.531 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:32.525+0000 7f653ab52640 1 --2- 172.21.15.164:0/2503946685 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f653414eb80 0x7f653409c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:32.531 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:32.525+0000 7f653ab52640 1 --2- 172.21.15.164:0/2503946685 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6534148cf0 0x7f65341e1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:32.531 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:32.525+0000 7f653ab52640 1 --2- 172.21.15.164:0/2503946685 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f65340b0230 0x7f6534144520 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:32.531 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:32.525+0000 7f653ab52640 1 -- 172.21.15.164:0/2503946685 >> 172.21.15.164:0/2503946685 conn(0x7f65340b8ca0 msgr2=0x7f65340b43f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:32.531 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:32.525+0000 7f653ab52640 1 -- 172.21.15.164:0/2503946685 shutdown_connections 2024-03-23T14:57:32.531 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:32.525+0000 7f653ab52640 1 -- 172.21.15.164:0/2503946685 wait complete. 2024-03-23T14:57:32.550 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-23T14:57:32.676 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:32.673+0000 7f721f533640 1 Processor -- start 2024-03-23T14:57:32.676 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:32.673+0000 7f721f533640 1 -- start start 2024-03-23T14:57:32.678 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:32.673+0000 7f721f533640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f72180b4e90 0x7f721814f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:32.678 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:32.673+0000 7f721f533640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f721809e2d0 con 0x7f72180b4a00 2024-03-23T14:57:32.678 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:32.673+0000 7f721f533640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f721806cb70 con 0x7f721809def0 2024-03-23T14:57:32.678 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:32.673+0000 7f721f533640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f7218146de0 con 0x7f72180b4e90 2024-03-23T14:57:32.678 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:32.673+0000 7f721cace640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f72180b4a00 0x7f7218147420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:58716/0 (socket says 172.21.15.164:58716) 2024-03-23T14:57:32.678 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:32.673+0000 7f721cace640 1 -- 172.21.15.164:0/1630736803 learned_addr learned my addr 172.21.15.164:0/1630736803 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-03-23T14:57:32.678 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:32.673+0000 7f721dad0640 1 --2- 172.21.15.164:0/1630736803 >> v2:172.21.15.164:3300/0 conn(0x7f72180b4e90 0x7f721814f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:32.678 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:32.673+0000 7f720ffff640 1 -- 172.21.15.164:0/1630736803 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2425546321 0 0) 0x7f721809e2d0 con 0x7f72180b4a00 2024-03-23T14:57:32.678 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:32.673+0000 7f720ffff640 1 -- 172.21.15.164:0/1630736803 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7204003530 con 0x7f72180b4a00 2024-03-23T14:57:32.678 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:32.673+0000 7f721dad0640 1 -- 172.21.15.164:0/1630736803 >> v1:172.21.15.173:6789/0 conn(0x7f721809def0 legacy=0x7f72180ae620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:32.678 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:32.673+0000 7f721dad0640 1 -- 172.21.15.164:0/1630736803 >> v1:172.21.15.164:6789/0 conn(0x7f72180b4a00 legacy=0x7f7218147420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:32.679 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:32.673+0000 7f721dad0640 1 -- 172.21.15.164:0/1630736803 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f721806ce50 con 0x7f72180b4e90 2024-03-23T14:57:32.680 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:32.677+0000 7f721dad0640 1 --2- 172.21.15.164:0/1630736803 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f72180b4e90 0x7f721814f120 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f7210009d90 tx=0x7f721001c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ee95353b5182ec5 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:32.680 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:32.677+0000 7f720ffff640 1 -- 172.21.15.164:0/1630736803 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72100a8630 con 0x7f72180b4e90 2024-03-23T14:57:32.680 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:32.677+0000 7f720ffff640 1 -- 172.21.15.164:0/1630736803 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72100a8790 con 0x7f72180b4e90 2024-03-23T14:57:32.680 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:32.677+0000 7f720ffff640 1 -- 172.21.15.164:0/1630736803 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72100a8b20 con 0x7f72180b4e90 2024-03-23T14:57:32.681 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:32.677+0000 7f721f533640 1 -- 172.21.15.164:0/1630736803 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f72180b4e90 msgr2=0x7f721814f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:32.681 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:32.677+0000 7f721f533640 1 --2- 172.21.15.164:0/1630736803 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f72180b4e90 0x7f721814f120 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f7210009d90 tx=0x7f721001c810 comp rx=0 tx=0).stop 2024-03-23T14:57:32.681 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:32.677+0000 7f721f533640 1 -- 172.21.15.164:0/1630736803 shutdown_connections 2024-03-23T14:57:32.681 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:32.677+0000 7f721f533640 1 --2- 172.21.15.164:0/1630736803 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f72180b4e90 0x7f721814f120 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:32.681 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:32.677+0000 7f721f533640 1 -- 172.21.15.164:0/1630736803 >> 172.21.15.164:0/1630736803 conn(0x7f72180b8ca0 msgr2=0x7f72180b90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:32.681 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:32.677+0000 7f721f533640 1 -- 172.21.15.164:0/1630736803 shutdown_connections 2024-03-23T14:57:32.681 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:32.677+0000 7f721f533640 1 -- 172.21.15.164:0/1630736803 wait complete. 2024-03-23T14:57:32.683 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:32.677+0000 7f721f533640 1 Processor -- start 2024-03-23T14:57:32.683 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:32.677+0000 7f721f533640 1 -- start start 2024-03-23T14:57:32.683 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:32.677+0000 7f721f533640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f721809def0 0x7f72181e32a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:32.684 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:32.677+0000 7f721f533640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f72180b4a00 0x7f721814f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:32.684 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:32.677+0000 7f721f533640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f72180b4e90 0x7f721813a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:32.684 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:32.677+0000 7f721f533640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f721813e800 con 0x7f721809def0 2024-03-23T14:57:32.684 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:32.677+0000 7f721f533640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f721813d330 con 0x7f72180b4a00 2024-03-23T14:57:32.684 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:32.677+0000 7f721f533640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f721814d0b0 con 0x7f72180b4e90 2024-03-23T14:57:32.684 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:32.677+0000 7f721d2cf640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f721809def0 0x7f72181e32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:32.684 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:32.677+0000 7f721d2cf640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f721809def0 0x7f72181e32a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:33118/0 (socket says 172.21.15.164:33118) 2024-03-23T14:57:32.684 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:32.677+0000 7f721d2cf640 1 -- 172.21.15.164:0/288739678 learned_addr learned my addr 172.21.15.164:0/288739678 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:32.684 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:32.677+0000 7f721d2cf640 1 -- 172.21.15.164:0/288739678 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f72180b4e90 msgr2=0x7f721813a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:32.684 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:32.677+0000 7f721d2cf640 1 --2- 172.21.15.164:0/288739678 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f72180b4e90 0x7f721813a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:32.684 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:32.677+0000 7f721d2cf640 1 -- 172.21.15.164:0/288739678 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f72180b4a00 msgr2=0x7f721814f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:32.684 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:32.677+0000 7f721d2cf640 1 --2- 172.21.15.164:0/288739678 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f72180b4a00 0x7f721814f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:32.684 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:32.677+0000 7f721d2cf640 1 -- 172.21.15.164:0/288739678 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7210009a70 con 0x7f721809def0 2024-03-23T14:57:32.684 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:32.677+0000 7f721d2cf640 1 --2- 172.21.15.164:0/288739678 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f721809def0 0x7f72181e32a0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f720800c450 tx=0x7f720800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:32.685 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:32.677+0000 7f720dffb640 1 -- 172.21.15.164:0/288739678 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7208012830 con 0x7f721809def0 2024-03-23T14:57:32.685 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:32.677+0000 7f720dffb640 1 -- 172.21.15.164:0/288739678 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72080129b0 con 0x7f721809def0 2024-03-23T14:57:32.685 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:32.677+0000 7f721f533640 1 -- 172.21.15.164:0/288739678 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7218147d40 con 0x7f721809def0 2024-03-23T14:57:32.685 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:32.677+0000 7f720dffb640 1 -- 172.21.15.164:0/288739678 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7208012d60 con 0x7f721809def0 2024-03-23T14:57:32.685 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:32.677+0000 7f721f533640 1 -- 172.21.15.164:0/288739678 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f721813e490 con 0x7f721809def0 2024-03-23T14:57:32.686 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:32.681+0000 7f721f533640 1 -- 172.21.15.164:0/288739678 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f721813d190 con 0x7f721809def0 2024-03-23T14:57:32.686 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:32.681+0000 7f720dffb640 1 -- 172.21.15.164:0/288739678 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7f720801c460 con 0x7f721809def0 2024-03-23T14:57:32.686 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:32.681+0000 7f720dffb640 1 --2- 172.21.15.164:0/288739678 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f71fc0672a0 0x7f71fc069760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:32.686 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:32.681+0000 7f720dffb640 1 -- 172.21.15.164:0/288739678 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5795+0+0 (secure 0 0 0) 0x7f7208084ec0 con 0x7f721809def0 2024-03-23T14:57:32.686 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:32.681+0000 7f721cace640 1 --2- 172.21.15.164:0/288739678 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f71fc0672a0 0x7f71fc069760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:32.686 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:32.681+0000 7f721cace640 1 --2- 172.21.15.164:0/288739678 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f71fc0672a0 0x7f71fc069760 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7214007fd0 tx=0x7f7214007900 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:32.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:32.685+0000 7f720dffb640 1 -- 172.21.15.164:0/288739678 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7f721813d190 con 0x7f721809def0 2024-03-23T14:57:32.856 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:32.849+0000 7f721f533640 1 -- 172.21.15.164:0/288739678 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f721813d190 con 0x7f721809def0 2024-03-23T14:57:33.536 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:33.533+0000 7f720dffb640 1 -- 172.21.15.164:0/288739678 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v22) ==== 181+0+0 (secure 0 0 0) 0x7f721813d190 con 0x7f721809def0 2024-03-23T14:57:33.537 INFO:teuthology.orchestra.run.smithi164.stderr:pool 'cephfs_metadata' removed 2024-03-23T14:57:33.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:33.537+0000 7f721f533640 1 -- 172.21.15.164:0/288739678 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f71fc0672a0 msgr2=0x7f71fc069760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:33.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:33.537+0000 7f721f533640 1 --2- 172.21.15.164:0/288739678 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f71fc0672a0 0x7f71fc069760 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7214007fd0 tx=0x7f7214007900 comp rx=0 tx=0).stop 2024-03-23T14:57:33.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:33.537+0000 7f721f533640 1 -- 172.21.15.164:0/288739678 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f721809def0 msgr2=0x7f72181e32a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:33.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:33.537+0000 7f721f533640 1 --2- 172.21.15.164:0/288739678 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f721809def0 0x7f72181e32a0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f720800c450 tx=0x7f720800c910 comp rx=0 tx=0).stop 2024-03-23T14:57:33.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:33.537+0000 7f721f533640 1 -- 172.21.15.164:0/288739678 shutdown_connections 2024-03-23T14:57:33.542 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:33.537+0000 7f721f533640 1 --2- 172.21.15.164:0/288739678 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f72180b4e90 0x7f721813a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:33.542 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:33.537+0000 7f721f533640 1 --2- 172.21.15.164:0/288739678 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f71fc0672a0 0x7f71fc069760 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:33.542 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:33.537+0000 7f721f533640 1 --2- 172.21.15.164:0/288739678 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f72180b4a00 0x7f721814f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:33.542 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:33.537+0000 7f721f533640 1 --2- 172.21.15.164:0/288739678 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f721809def0 0x7f72181e32a0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:33.542 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:33.537+0000 7f721f533640 1 -- 172.21.15.164:0/288739678 >> 172.21.15.164:0/288739678 conn(0x7f72180b8ca0 msgr2=0x7f721813f870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:33.542 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:33.537+0000 7f721f533640 1 -- 172.21.15.164:0/288739678 shutdown_connections 2024-03-23T14:57:33.542 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:33.537+0000 7f721f533640 1 -- 172.21.15.164:0/288739678 wait complete. 2024-03-23T14:57:33.564 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-23T14:57:33.685 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:33.681+0000 7f8b344e2640 1 Processor -- start 2024-03-23T14:57:33.685 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:33.681+0000 7f8b344e2640 1 -- start start 2024-03-23T14:57:33.688 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:33.681+0000 7f8b344e2640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8b2c0add50 0x7f8b2c147770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:33.688 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:33.681+0000 7f8b344e2640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8b2c06d650 con 0x7f8b2c147cb0 2024-03-23T14:57:33.688 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:33.681+0000 7f8b344e2640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8b2c0af4a0 con 0x7f8b2c09def0 2024-03-23T14:57:33.688 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:33.681+0000 7f8b344e2640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f8b2c09e2d0 con 0x7f8b2c0add50 2024-03-23T14:57:33.688 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:33.681+0000 7f8b32a7f640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f8b2c147cb0 0x7f8b2c14ed80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:58728/0 (socket says 172.21.15.164:58728) 2024-03-23T14:57:33.688 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:33.681+0000 7f8b32a7f640 1 -- 172.21.15.164:0/914905230 learned_addr learned my addr 172.21.15.164:0/914905230 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-03-23T14:57:33.688 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:33.681+0000 7f8b3127c640 1 -- 172.21.15.164:0/914905230 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1170044736 0 0) 0x7f8b2c06d650 con 0x7f8b2c147cb0 2024-03-23T14:57:33.688 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:33.681+0000 7f8b3127c640 1 -- 172.21.15.164:0/914905230 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8b18005740 con 0x7f8b2c147cb0 2024-03-23T14:57:33.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:33.681+0000 7f8b31a7d640 1 --2- 172.21.15.164:0/914905230 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8b2c0add50 0x7f8b2c147770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:33.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:33.681+0000 7f8b3127c640 1 -- 172.21.15.164:0/914905230 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3729555520 0 0) 0x7f8b18005740 con 0x7f8b2c147cb0 2024-03-23T14:57:33.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:33.681+0000 7f8b3127c640 1 -- 172.21.15.164:0/914905230 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8b2c0add50 msgr2=0x7f8b2c147770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:33.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:33.681+0000 7f8b3127c640 1 --2- 172.21.15.164:0/914905230 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8b2c0add50 0x7f8b2c147770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:33.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:33.681+0000 7f8b3127c640 1 -- 172.21.15.164:0/914905230 >> v1:172.21.15.173:6789/0 conn(0x7f8b2c09def0 legacy=0x7f8b2c0b45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:33.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:33.681+0000 7f8b3127c640 1 -- 172.21.15.164:0/914905230 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b2c0aeca0 con 0x7f8b2c147cb0 2024-03-23T14:57:33.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:33.681+0000 7f8b3127c640 1 -- 172.21.15.164:0/914905230 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2526390548 0 0) 0x7f8b28001680 con 0x7f8b2c147cb0 2024-03-23T14:57:33.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:33.681+0000 7f8b3127c640 1 -- 172.21.15.164:0/914905230 >> v1:172.21.15.164:6789/0 conn(0x7f8b2c147cb0 legacy=0x7f8b2c14ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:33.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:33.681+0000 7f8b3127c640 1 --2- 172.21.15.164:0/914905230 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8b18005bc0 0x7f8b18005fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:33.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:33.681+0000 7f8b3127c640 1 --2- 172.21.15.164:0/914905230 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8b180069e0 0x7f8b18006e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:33.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:33.681+0000 7f8b3127c640 1 --2- 172.21.15.164:0/914905230 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8b2c0add50 0x7f8b18007470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:33.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:33.681+0000 7f8b3127c640 1 -- 172.21.15.164:0/914905230 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8b2c09e2d0 con 0x7f8b18005bc0 2024-03-23T14:57:33.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:33.681+0000 7f8b3127c640 1 -- 172.21.15.164:0/914905230 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f8b1800dd90 con 0x7f8b180069e0 2024-03-23T14:57:33.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:33.681+0000 7f8b3127c640 1 -- 172.21.15.164:0/914905230 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f8b1800df50 con 0x7f8b2c0add50 2024-03-23T14:57:33.690 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:33.681+0000 7f8b3227e640 1 --2- 172.21.15.164:0/914905230 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8b18005bc0 0x7f8b18005fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:33.690 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:33.681+0000 7f8b31a7d640 1 --2- 172.21.15.164:0/914905230 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8b180069e0 0x7f8b18006e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:33.690 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:33.681+0000 7f8b3227e640 1 -- 172.21.15.164:0/914905230 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8b2c0add50 msgr2=0x7f8b18007470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:33.690 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:33.681+0000 7f8b32a7f640 1 --2- 172.21.15.164:0/914905230 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8b2c0add50 0x7f8b18007470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:33.690 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:33.681+0000 7f8b3227e640 1 --2- 172.21.15.164:0/914905230 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8b2c0add50 0x7f8b18007470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:33.690 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:33.681+0000 7f8b3227e640 1 -- 172.21.15.164:0/914905230 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8b180069e0 msgr2=0x7f8b18006e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:33.690 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:33.681+0000 7f8b3227e640 1 --2- 172.21.15.164:0/914905230 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8b180069e0 0x7f8b18006e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:33.690 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:33.681+0000 7f8b3227e640 1 -- 172.21.15.164:0/914905230 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b18005740 con 0x7f8b18005bc0 2024-03-23T14:57:33.690 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:33.681+0000 7f8b31a7d640 1 --2- 172.21.15.164:0/914905230 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8b180069e0 0x7f8b18006e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T14:57:33.691 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:33.685+0000 7f8b3227e640 1 --2- 172.21.15.164:0/914905230 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8b18005bc0 0x7f8b18005fe0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f8b24001e70 tx=0x7f8b2401c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe221112e4997f4e server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:33.691 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:33.685+0000 7f8b3127c640 1 -- 172.21.15.164:0/914905230 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b24003b80 con 0x7f8b18005bc0 2024-03-23T14:57:33.692 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:33.685+0000 7f8b3127c640 1 -- 172.21.15.164:0/914905230 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b240a6040 con 0x7f8b18005bc0 2024-03-23T14:57:33.692 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:33.685+0000 7f8b3127c640 1 -- 172.21.15.164:0/914905230 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b240050d0 con 0x7f8b18005bc0 2024-03-23T14:57:33.692 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:33.685+0000 7f8b344e2640 1 -- 172.21.15.164:0/914905230 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8b18005bc0 msgr2=0x7f8b18005fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:33.692 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:33.685+0000 7f8b344e2640 1 --2- 172.21.15.164:0/914905230 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8b18005bc0 0x7f8b18005fe0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f8b24001e70 tx=0x7f8b2401c780 comp rx=0 tx=0).stop 2024-03-23T14:57:33.692 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:33.685+0000 7f8b32a7f640 1 -- 172.21.15.164:0/914905230 reap_dead start 2024-03-23T14:57:33.692 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:33.685+0000 7f8b344e2640 1 -- 172.21.15.164:0/914905230 shutdown_connections 2024-03-23T14:57:33.692 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:33.685+0000 7f8b344e2640 1 -- 172.21.15.164:0/914905230 >> 172.21.15.164:0/914905230 conn(0x7f8b2c0b8ca0 msgr2=0x7f8b2c0b90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:33.692 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:33.685+0000 7f8b344e2640 1 -- 172.21.15.164:0/914905230 shutdown_connections 2024-03-23T14:57:33.692 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:33.685+0000 7f8b344e2640 1 -- 172.21.15.164:0/914905230 wait complete. 2024-03-23T14:57:33.692 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:33.685+0000 7f8b344e2640 1 Processor -- start 2024-03-23T14:57:33.692 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:33.685+0000 7f8b344e2640 1 -- start start 2024-03-23T14:57:33.694 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:33.689+0000 7f8b344e2640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8b18005bc0 0x7f8b2c0b1e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:33.694 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:33.689+0000 7f8b344e2640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8b2c06f3b0 0x7f8b2c06fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:33.694 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:33.689+0000 7f8b344e2640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8b2c14a140 0x7f8b2c140d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:33.694 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:33.689+0000 7f8b344e2640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8b2c0720b0 con 0x7f8b2c06f3b0 2024-03-23T14:57:33.694 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:33.689+0000 7f8b344e2640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f8b2c0718c0 con 0x7f8b2c14a140 2024-03-23T14:57:33.695 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:33.689+0000 7f8b344e2640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f8b2c071150 con 0x7f8b18005bc0 2024-03-23T14:57:33.695 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:33.689+0000 7f8b31a7d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8b2c06f3b0 0x7f8b2c06fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:33.695 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:33.689+0000 7f8b31a7d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8b2c06f3b0 0x7f8b2c06fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:33132/0 (socket says 172.21.15.164:33132) 2024-03-23T14:57:33.695 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:33.689+0000 7f8b31a7d640 1 -- 172.21.15.164:0/3298573023 learned_addr learned my addr 172.21.15.164:0/3298573023 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:33.695 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:33.689+0000 7f8b3227e640 1 --2- 172.21.15.164:0/3298573023 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8b18005bc0 0x7f8b2c0b1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:33.695 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:33.689+0000 7f8b32a7f640 1 --2- 172.21.15.164:0/3298573023 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8b2c14a140 0x7f8b2c140d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:33.695 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:33.689+0000 7f8b31a7d640 1 -- 172.21.15.164:0/3298573023 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8b18005bc0 msgr2=0x7f8b2c0b1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:33.695 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:33.689+0000 7f8b31a7d640 1 --2- 172.21.15.164:0/3298573023 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8b18005bc0 0x7f8b2c0b1e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:33.695 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:33.689+0000 7f8b31a7d640 1 -- 172.21.15.164:0/3298573023 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8b2c14a140 msgr2=0x7f8b2c140d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:33.695 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:33.689+0000 7f8b31a7d640 1 --2- 172.21.15.164:0/3298573023 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8b2c14a140 0x7f8b2c140d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:33.695 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:33.689+0000 7f8b31a7d640 1 -- 172.21.15.164:0/3298573023 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b2c071af0 con 0x7f8b2c06f3b0 2024-03-23T14:57:33.695 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:33.689+0000 7f8b31a7d640 1 --2- 172.21.15.164:0/3298573023 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8b2c06f3b0 0x7f8b2c06fdc0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f8b1c010890 tx=0x7f8b1c010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:33.695 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:33.689+0000 7f8b22ffd640 1 -- 172.21.15.164:0/3298573023 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b1c004220 con 0x7f8b2c06f3b0 2024-03-23T14:57:33.695 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:33.689+0000 7f8b22ffd640 1 -- 172.21.15.164:0/3298573023 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b1c004380 con 0x7f8b2c06f3b0 2024-03-23T14:57:33.695 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:33.689+0000 7f8b344e2640 1 -- 172.21.15.164:0/3298573023 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b2c071dd0 con 0x7f8b2c06f3b0 2024-03-23T14:57:33.696 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:33.689+0000 7f8b22ffd640 1 -- 172.21.15.164:0/3298573023 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b1c00eb80 con 0x7f8b2c06f3b0 2024-03-23T14:57:33.696 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:33.689+0000 7f8b344e2640 1 -- 172.21.15.164:0/3298573023 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b2c070300 con 0x7f8b2c06f3b0 2024-03-23T14:57:33.696 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:33.689+0000 7f8b344e2640 1 -- 172.21.15.164:0/3298573023 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b2c145080 con 0x7f8b2c06f3b0 2024-03-23T14:57:33.696 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:33.689+0000 7f8b22ffd640 1 -- 172.21.15.164:0/3298573023 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7f8b1c00ece0 con 0x7f8b2c06f3b0 2024-03-23T14:57:33.696 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:33.689+0000 7f8b22ffd640 1 --2- 172.21.15.164:0/3298573023 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f8b100672a0 0x7f8b10069760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:33.696 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:33.689+0000 7f8b22ffd640 1 -- 172.21.15.164:0/3298573023 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5357+0+0 (secure 0 0 0) 0x7f8b1c084fe0 con 0x7f8b2c06f3b0 2024-03-23T14:57:33.697 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:33.689+0000 7f8b3227e640 1 --2- 172.21.15.164:0/3298573023 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f8b100672a0 0x7f8b10069760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:33.697 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:33.689+0000 7f8b3227e640 1 --2- 172.21.15.164:0/3298573023 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f8b100672a0 0x7f8b10069760 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f8b240ac000 tx=0x7f8b24001520 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:33.699 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:33.693+0000 7f8b22ffd640 1 -- 172.21.15.164:0/3298573023 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7f8b1c051d80 con 0x7f8b2c06f3b0 2024-03-23T14:57:33.859 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:33.853+0000 7f8b344e2640 1 -- 172.21.15.164:0/3298573023 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f8b2c0b4c70 con 0x7f8b2c06f3b0 2024-03-23T14:57:34.539 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:34.533+0000 7f8b22ffd640 1 -- 172.21.15.164:0/3298573023 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v23) ==== 169+0+0 (secure 0 0 0) 0x7f8b1c0044f0 con 0x7f8b2c06f3b0 2024-03-23T14:57:34.539 INFO:teuthology.orchestra.run.smithi164.stderr:pool 'cephfs_data' removed 2024-03-23T14:57:34.542 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:34.537+0000 7f8b344e2640 1 -- 172.21.15.164:0/3298573023 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f8b100672a0 msgr2=0x7f8b10069760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:34.542 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:34.537+0000 7f8b344e2640 1 --2- 172.21.15.164:0/3298573023 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f8b100672a0 0x7f8b10069760 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f8b240ac000 tx=0x7f8b24001520 comp rx=0 tx=0).stop 2024-03-23T14:57:34.542 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:34.537+0000 7f8b344e2640 1 -- 172.21.15.164:0/3298573023 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8b2c06f3b0 msgr2=0x7f8b2c06fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:34.542 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:34.537+0000 7f8b344e2640 1 --2- 172.21.15.164:0/3298573023 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8b2c06f3b0 0x7f8b2c06fdc0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f8b1c010890 tx=0x7f8b1c010d50 comp rx=0 tx=0).stop 2024-03-23T14:57:34.545 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:34.541+0000 7f8b344e2640 1 -- 172.21.15.164:0/3298573023 shutdown_connections 2024-03-23T14:57:34.545 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:34.541+0000 7f8b344e2640 1 --2- 172.21.15.164:0/3298573023 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f8b100672a0 0x7f8b10069760 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:34.545 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:34.541+0000 7f8b344e2640 1 --2- 172.21.15.164:0/3298573023 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8b2c14a140 0x7f8b2c140d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:34.545 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:34.541+0000 7f8b344e2640 1 --2- 172.21.15.164:0/3298573023 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8b2c06f3b0 0x7f8b2c06fdc0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:34.545 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:34.541+0000 7f8b344e2640 1 --2- 172.21.15.164:0/3298573023 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8b18005bc0 0x7f8b2c0b1e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:34.545 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:34.541+0000 7f8b344e2640 1 -- 172.21.15.164:0/3298573023 >> 172.21.15.164:0/3298573023 conn(0x7f8b2c0b8ca0 msgr2=0x7f8b2c145e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:34.545 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:34.541+0000 7f8b344e2640 1 -- 172.21.15.164:0/3298573023 shutdown_connections 2024-03-23T14:57:34.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:34.541+0000 7f8b344e2640 1 -- 172.21.15.164:0/3298573023 wait complete. 2024-03-23T14:57:34.569 INFO:tasks.ceph.mds.a:Restarting daemon 2024-03-23T14:57:34.569 INFO:tasks.ceph.mds.a:Stopping old one... 2024-03-23T14:57:34.569 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-03-23T14:57:34.569 INFO:teuthology.orchestra.run:waiting for 300 2024-03-23T14:57:34.569 INFO:tasks.ceph.mds.b:Restarting daemon 2024-03-23T14:57:34.569 INFO:tasks.ceph.mds.b:Stopping old one... 2024-03-23T14:57:34.569 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-03-23T14:57:34.570 INFO:teuthology.orchestra.run:waiting for 300 2024-03-23T14:57:34.579 INFO:tasks.ceph.mds.a:Stopped 2024-03-23T14:57:34.579 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-03-23T14:57:34.584 INFO:tasks.ceph.mds.b:Stopped 2024-03-23T14:57:34.584 DEBUG:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-03-23T14:57:34.625 INFO:tasks.ceph.mds.a:Started 2024-03-23T14:57:34.625 DEBUG:teuthology.parallel:result is None 2024-03-23T14:57:34.627 INFO:tasks.ceph.mds.b:Started 2024-03-23T14:57:34.627 DEBUG:teuthology.parallel:result is None 2024-03-23T14:57:34.628 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-03-23T14:57:34.690 INFO:tasks.ceph.mds.a.smithi164.stdout:starting mds.a at 2024-03-23T14:57:34.699 INFO:tasks.ceph.mds.b.smithi173.stdout:starting mds.b at 2024-03-23T14:57:34.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:34.749+0000 7f30dad24640 1 Processor -- start 2024-03-23T14:57:34.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:34.749+0000 7f30dad24640 1 -- start start 2024-03-23T14:57:34.756 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:34.749+0000 7f30dad24640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f30d40b56f0 0x7f30d414a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:34.756 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:34.749+0000 7f30dad24640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30d40b4300 con 0x7f30d4141640 2024-03-23T14:57:34.756 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:34.749+0000 7f30dad24640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30d40b45e0 con 0x7f30d40b4a70 2024-03-23T14:57:34.756 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:34.749+0000 7f30dad24640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 -- 0x7f30d40b5fa0 con 0x7f30d40b56f0 2024-03-23T14:57:34.756 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:34.749+0000 7f30d92c1640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f30d4141640 0x7f30d4151c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:58736/0 (socket says 172.21.15.164:58736) 2024-03-23T14:57:34.756 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:34.749+0000 7f30d92c1640 1 -- 172.21.15.164:0/4066314833 learned_addr learned my addr 172.21.15.164:0/4066314833 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-03-23T14:57:34.756 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:34.749+0000 7f30d8ac0640 1 --2- 172.21.15.164:0/4066314833 >> v2:172.21.15.173:3300/0 conn(0x7f30d40b56f0 0x7f30d414a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:34.756 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:34.749+0000 7f30d37fe640 1 -- 172.21.15.164:0/4066314833 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3733222395 0 0) 0x7f30d40b4300 con 0x7f30d4141640 2024-03-23T14:57:34.757 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:34.749+0000 7f30d37fe640 1 -- 172.21.15.164:0/4066314833 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f30c0003530 con 0x7f30d4141640 2024-03-23T14:57:34.757 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:34.749+0000 7f30d37fe640 1 -- 172.21.15.164:0/4066314833 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2286066539 0 0) 0x7f30c0003530 con 0x7f30d4141640 2024-03-23T14:57:34.757 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:34.749+0000 7f30d37fe640 1 -- 172.21.15.164:0/4066314833 >> v2:172.21.15.173:3300/0 conn(0x7f30d40b56f0 msgr2=0x7f30d414a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:34.757 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:34.749+0000 7f30d37fe640 1 --2- 172.21.15.164:0/4066314833 >> v2:172.21.15.173:3300/0 conn(0x7f30d40b56f0 0x7f30d414a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:34.757 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:34.749+0000 7f30d37fe640 1 -- 172.21.15.164:0/4066314833 >> v1:172.21.15.173:6789/0 conn(0x7f30d40b4a70 legacy=0x7f30d4140f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:34.757 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:34.749+0000 7f30d37fe640 1 -- 172.21.15.164:0/4066314833 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30d414ade0 con 0x7f30d4141640 2024-03-23T14:57:34.757 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:34.749+0000 7f30d37fe640 1 -- 172.21.15.164:0/4066314833 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2526390548 0 0) 0x7f30c4001a00 con 0x7f30d4141640 2024-03-23T14:57:34.757 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:34.749+0000 7f30d37fe640 1 -- 172.21.15.164:0/4066314833 >> v1:172.21.15.164:6789/0 conn(0x7f30d4141640 legacy=0x7f30d4151c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:34.757 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:34.749+0000 7f30d37fe640 1 --2- 172.21.15.164:0/4066314833 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f30c0003980 0x7f30c0003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:34.757 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:34.749+0000 7f30d37fe640 1 --2- 172.21.15.164:0/4066314833 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f30c00047d0 0x7f30c0004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:34.758 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:34.749+0000 7f30d37fe640 1 --2- 172.21.15.164:0/4066314833 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f30c0005190 0x7f30c000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:34.758 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:34.749+0000 7f30d37fe640 1 -- 172.21.15.164:0/4066314833 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f30c000c000 con 0x7f30c0005190 2024-03-23T14:57:34.758 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:34.749+0000 7f30d37fe640 1 -- 172.21.15.164:0/4066314833 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f30c000c170 con 0x7f30c00047d0 2024-03-23T14:57:34.758 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:34.749+0000 7f30d37fe640 1 -- 172.21.15.164:0/4066314833 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f30c000c330 con 0x7f30c0003980 2024-03-23T14:57:34.758 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:34.749+0000 7f30d92c1640 1 --2- 172.21.15.164:0/4066314833 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f30c0005190 0x7f30c000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:34.758 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:34.749+0000 7f30d3fff640 1 --2- 172.21.15.164:0/4066314833 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f30c00047d0 0x7f30c0004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:34.758 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:34.749+0000 7f30d8ac0640 1 --2- 172.21.15.164:0/4066314833 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f30c0003980 0x7f30c0003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:34.758 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:34.749+0000 7f30d92c1640 1 -- 172.21.15.164:0/4066314833 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f30c0003980 msgr2=0x7f30c0003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:34.758 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:34.749+0000 7f30d92c1640 1 --2- 172.21.15.164:0/4066314833 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f30c0003980 0x7f30c0003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:34.758 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:34.749+0000 7f30d92c1640 1 -- 172.21.15.164:0/4066314833 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f30c00047d0 msgr2=0x7f30c0004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:34.758 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:34.749+0000 7f30d92c1640 1 --2- 172.21.15.164:0/4066314833 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f30c00047d0 0x7f30c0004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:34.759 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:34.749+0000 7f30d92c1640 1 -- 172.21.15.164:0/4066314833 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30c0003530 con 0x7f30c0005190 2024-03-23T14:57:34.759 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:34.749+0000 7f30d3fff640 1 --2- 172.21.15.164:0/4066314833 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f30c00047d0 0x7f30c0004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T14:57:34.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:34.753+0000 7f30d92c1640 1 -- 172.21.15.164:0/4066314833 reap_dead start 2024-03-23T14:57:34.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:34.753+0000 7f30d92c1640 1 --2- 172.21.15.164:0/4066314833 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f30c0005190 0x7f30c000b990 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f30c400a830 tx=0x7f30c401d360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ec07a6b6249cd5d server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:34.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:34.753+0000 7f30d37fe640 1 -- 172.21.15.164:0/4066314833 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c40072d0 con 0x7f30c0005190 2024-03-23T14:57:34.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:34.753+0000 7f30d37fe640 1 -- 172.21.15.164:0/4066314833 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30c40a8070 con 0x7f30c0005190 2024-03-23T14:57:34.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:34.753+0000 7f30d37fe640 1 -- 172.21.15.164:0/4066314833 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c4007650 con 0x7f30c0005190 2024-03-23T14:57:34.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:34.753+0000 7f30dad24640 1 -- 172.21.15.164:0/4066314833 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f30c0005190 msgr2=0x7f30c000b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:34.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:34.753+0000 7f30dad24640 1 --2- 172.21.15.164:0/4066314833 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f30c0005190 0x7f30c000b990 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f30c400a830 tx=0x7f30c401d360 comp rx=0 tx=0).stop 2024-03-23T14:57:34.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:34.753+0000 7f30dad24640 1 -- 172.21.15.164:0/4066314833 shutdown_connections 2024-03-23T14:57:34.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:34.753+0000 7f30dad24640 1 --2- 172.21.15.164:0/4066314833 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f30c0005190 0x7f30c000b990 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:34.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:34.753+0000 7f30dad24640 1 -- 172.21.15.164:0/4066314833 >> 172.21.15.164:0/4066314833 conn(0x7f30d402c3c0 msgr2=0x7f30d402c7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:34.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:34.753+0000 7f30dad24640 1 -- 172.21.15.164:0/4066314833 shutdown_connections 2024-03-23T14:57:34.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:34.753+0000 7f30dad24640 1 -- 172.21.15.164:0/4066314833 wait complete. 2024-03-23T14:57:34.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:34.753+0000 7f30dad24640 1 Processor -- start 2024-03-23T14:57:34.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:34.753+0000 7f30dad24640 1 -- start start 2024-03-23T14:57:34.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:34.757+0000 7f30dad24640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f30c0005190 0x7f30d41478b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:34.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:34.757+0000 7f30dad24640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f30d4147df0 0x7f30d41ddd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:34.763 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:34.757+0000 7f30dad24640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f30d41422f0 0x7f30d41de280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:34.763 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:34.757+0000 7f30dad24640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f30d4142ea0 con 0x7f30d4147df0 2024-03-23T14:57:34.763 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:34.757+0000 7f30dad24640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f30d4143bb0 con 0x7f30c0005190 2024-03-23T14:57:34.763 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:34.757+0000 7f30dad24640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f30d41427e0 con 0x7f30d41422f0 2024-03-23T14:57:34.763 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:34.757+0000 7f30d3fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f30d4147df0 0x7f30d41ddd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:34.763 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:34.757+0000 7f30d3fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f30d4147df0 0x7f30d41ddd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:33172/0 (socket says 172.21.15.164:33172) 2024-03-23T14:57:34.763 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:34.757+0000 7f30d3fff640 1 -- 172.21.15.164:0/2897815113 learned_addr learned my addr 172.21.15.164:0/2897815113 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:34.763 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:34.757+0000 7f30d8ac0640 1 --2- 172.21.15.164:0/2897815113 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f30c0005190 0x7f30d41478b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:34.763 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:34.757+0000 7f30d92c1640 1 --2- 172.21.15.164:0/2897815113 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f30d41422f0 0x7f30d41de280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:34.763 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:34.757+0000 7f30d3fff640 1 -- 172.21.15.164:0/2897815113 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f30d41422f0 msgr2=0x7f30d41de280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:34.763 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:34.757+0000 7f30d3fff640 1 --2- 172.21.15.164:0/2897815113 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f30d41422f0 0x7f30d41de280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:34.763 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:34.757+0000 7f30d3fff640 1 -- 172.21.15.164:0/2897815113 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f30c0005190 msgr2=0x7f30d41478b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:34.763 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:34.757+0000 7f30d3fff640 1 --2- 172.21.15.164:0/2897815113 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f30c0005190 0x7f30d41478b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:34.763 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:34.757+0000 7f30d3fff640 1 -- 172.21.15.164:0/2897815113 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30d4142ac0 con 0x7f30d4147df0 2024-03-23T14:57:34.763 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:34.757+0000 7f30d92c1640 1 --2- 172.21.15.164:0/2897815113 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f30d41422f0 0x7f30d41de280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T14:57:34.764 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:34.757+0000 7f30d3fff640 1 --2- 172.21.15.164:0/2897815113 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f30d4147df0 0x7f30d41ddd40 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f30bc00e440 tx=0x7f30bc00e900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:34.764 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:34.757+0000 7f30d17fa640 1 -- 172.21.15.164:0/2897815113 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30bc0052e0 con 0x7f30d4147df0 2024-03-23T14:57:34.764 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:34.757+0000 7f30d17fa640 1 -- 172.21.15.164:0/2897815113 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30bc005440 con 0x7f30d4147df0 2024-03-23T14:57:34.764 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:34.757+0000 7f30dad24640 1 -- 172.21.15.164:0/2897815113 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30d41437b0 con 0x7f30d4147df0 2024-03-23T14:57:34.764 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:34.757+0000 7f30d17fa640 1 -- 172.21.15.164:0/2897815113 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30bc003820 con 0x7f30d4147df0 2024-03-23T14:57:34.764 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:34.757+0000 7f30dad24640 1 -- 172.21.15.164:0/2897815113 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30d41dfce0 con 0x7f30d4147df0 2024-03-23T14:57:34.764 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:34.757+0000 7f30dad24640 1 -- 172.21.15.164:0/2897815113 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30d40690a0 con 0x7f30d4147df0 2024-03-23T14:57:34.765 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:34.757+0000 7f30d17fa640 1 -- 172.21.15.164:0/2897815113 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7f30bc003980 con 0x7f30d4147df0 2024-03-23T14:57:34.765 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:34.757+0000 7f30d17fa640 1 --2- 172.21.15.164:0/2897815113 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f30b40672a0 0x7f30b4069760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:34.765 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:34.757+0000 7f30d17fa640 1 -- 172.21.15.164:0/2897815113 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 4943+0+0 (secure 0 0 0) 0x7f30bc084910 con 0x7f30d4147df0 2024-03-23T14:57:34.765 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:34.757+0000 7f30d8ac0640 1 --2- 172.21.15.164:0/2897815113 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f30b40672a0 0x7f30b4069760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:34.765 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:34.757+0000 7f30d8ac0640 1 --2- 172.21.15.164:0/2897815113 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f30b40672a0 0x7f30b4069760 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f30cc009a60 tx=0x7f30cc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:34.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:34.761+0000 7f30d17fa640 1 -- 172.21.15.164:0/2897815113 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7f30bc051850 con 0x7f30d4147df0 2024-03-23T14:57:34.937 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:34.933+0000 7f30dad24640 1 -- 172.21.15.164:0/2897815113 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f30d4148250 con 0x7f30d4147df0 2024-03-23T14:57:35.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:35.569+0000 7f30d17fa640 1 -- 172.21.15.164:0/2897815113 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v25) ==== 97+0+0 (secure 0 0 0) 0x7f30bc007e60 con 0x7f30d4147df0 2024-03-23T14:57:35.575 INFO:teuthology.orchestra.run.smithi164.stderr: removed all blocklist entries 2024-03-23T14:57:35.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:35.573+0000 7f30dad24640 1 -- 172.21.15.164:0/2897815113 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f30b40672a0 msgr2=0x7f30b4069760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:35.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:35.573+0000 7f30dad24640 1 --2- 172.21.15.164:0/2897815113 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f30b40672a0 0x7f30b4069760 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f30cc009a60 tx=0x7f30cc008040 comp rx=0 tx=0).stop 2024-03-23T14:57:35.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:35.573+0000 7f30dad24640 1 -- 172.21.15.164:0/2897815113 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f30d4147df0 msgr2=0x7f30d41ddd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:35.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:35.573+0000 7f30dad24640 1 --2- 172.21.15.164:0/2897815113 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f30d4147df0 0x7f30d41ddd40 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f30bc00e440 tx=0x7f30bc00e900 comp rx=0 tx=0).stop 2024-03-23T14:57:35.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:35.573+0000 7f30dad24640 1 -- 172.21.15.164:0/2897815113 shutdown_connections 2024-03-23T14:57:35.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:35.573+0000 7f30dad24640 1 --2- 172.21.15.164:0/2897815113 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f30d41422f0 0x7f30d41de280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:35.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:35.573+0000 7f30dad24640 1 --2- 172.21.15.164:0/2897815113 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f30d4147df0 0x7f30d41ddd40 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:35.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:35.573+0000 7f30dad24640 1 --2- 172.21.15.164:0/2897815113 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f30b40672a0 0x7f30b4069760 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:35.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:35.573+0000 7f30dad24640 1 --2- 172.21.15.164:0/2897815113 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f30c0005190 0x7f30d41478b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:35.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:35.573+0000 7f30dad24640 1 -- 172.21.15.164:0/2897815113 >> 172.21.15.164:0/2897815113 conn(0x7f30d402c3c0 msgr2=0x7f30d402c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:35.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:35.573+0000 7f30dad24640 1 -- 172.21.15.164:0/2897815113 shutdown_connections 2024-03-23T14:57:35.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:35.573+0000 7f30dad24640 1 -- 172.21.15.164:0/2897815113 wait complete. 2024-03-23T14:57:35.603 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-03-23T14:57:35.719 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:35.713+0000 7f1b9e5b4640 1 Processor -- start 2024-03-23T14:57:35.720 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:35.713+0000 7f1b9e5b4640 1 -- start start 2024-03-23T14:57:35.721 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:35.717+0000 7f1b9e5b4640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1b980add50 0x7f1b98147770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:35.721 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:35.717+0000 7f1b9e5b4640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f1b98147cb0 0x7f1b98148140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:35.721 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:35.717+0000 7f1b9e5b4640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b9806d650 con 0x7f1b9809def0 2024-03-23T14:57:35.721 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:35.717+0000 7f1b9e5b4640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f1b980af4a0 con 0x7f1b98147cb0 2024-03-23T14:57:35.721 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:35.717+0000 7f1b9e5b4640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f1b9809e2d0 con 0x7f1b980add50 2024-03-23T14:57:35.721 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:35.717+0000 7f1b9cb51640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f1b98147cb0 0x7f1b98148140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:35.721 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:35.717+0000 7f1b97fff640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f1b9809def0 0x7f1b980b45e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:58744/0 (socket says 172.21.15.164:58744) 2024-03-23T14:57:35.722 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:35.717+0000 7f1b97fff640 1 -- 172.21.15.164:0/480104458 learned_addr learned my addr 172.21.15.164:0/480104458 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-03-23T14:57:35.722 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:35.717+0000 7f1b977fe640 1 --2- 172.21.15.164:0/480104458 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1b980add50 0x7f1b98147770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:35.722 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:35.717+0000 7f1b9cb51640 1 -- 172.21.15.164:0/480104458 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1b980add50 msgr2=0x7f1b98147770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:35.722 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:35.717+0000 7f1b9cb51640 1 --2- 172.21.15.164:0/480104458 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1b980add50 0x7f1b98147770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:35.722 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:35.717+0000 7f1b9cb51640 1 -- 172.21.15.164:0/480104458 >> v1:172.21.15.164:6789/0 conn(0x7f1b9809def0 legacy=0x7f1b980b45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:35.722 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:35.717+0000 7f1b9cb51640 1 -- 172.21.15.164:0/480104458 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b980aeca0 con 0x7f1b98147cb0 2024-03-23T14:57:35.722 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:35.717+0000 7f1b96ffd640 1 -- 172.21.15.164:0/480104458 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 751244837 0 0) 0x7f1b9806d650 con 0x7f1b9809def0 2024-03-23T14:57:35.722 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:35.717+0000 7f1b977fe640 1 --2- 172.21.15.164:0/480104458 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1b980add50 0x7f1b98147770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T14:57:35.723 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:35.717+0000 7f1b9cb51640 1 --2- 172.21.15.164:0/480104458 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b98147cb0 0x7f1b98148140 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f1b900097c0 tx=0x7f1b9001c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2243ac6a88cd4ba server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:35.723 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:35.717+0000 7f1b96ffd640 1 -- 172.21.15.164:0/480104458 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b900ab600 con 0x7f1b98147cb0 2024-03-23T14:57:35.723 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:35.717+0000 7f1b96ffd640 1 -- 172.21.15.164:0/480104458 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b900a7050 con 0x7f1b98147cb0 2024-03-23T14:57:35.723 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:35.717+0000 7f1b96ffd640 1 -- 172.21.15.164:0/480104458 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b900ab980 con 0x7f1b98147cb0 2024-03-23T14:57:35.724 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:35.721+0000 7f1b9e5b4640 1 -- 172.21.15.164:0/480104458 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b98147cb0 msgr2=0x7f1b98148140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:35.724 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:35.721+0000 7f1b9e5b4640 1 --2- 172.21.15.164:0/480104458 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b98147cb0 0x7f1b98148140 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f1b900097c0 tx=0x7f1b9001c380 comp rx=0 tx=0).stop 2024-03-23T14:57:35.724 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:35.721+0000 7f1b9e5b4640 1 -- 172.21.15.164:0/480104458 shutdown_connections 2024-03-23T14:57:35.724 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:35.721+0000 7f1b9e5b4640 1 --2- 172.21.15.164:0/480104458 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b98147cb0 0x7f1b98148140 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:35.724 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:35.721+0000 7f1b9e5b4640 1 --2- 172.21.15.164:0/480104458 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1b980add50 0x7f1b98147770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:35.725 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:35.721+0000 7f1b9e5b4640 1 -- 172.21.15.164:0/480104458 >> 172.21.15.164:0/480104458 conn(0x7f1b980b8ca0 msgr2=0x7f1b980b90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:35.725 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:35.721+0000 7f1b9e5b4640 1 -- 172.21.15.164:0/480104458 shutdown_connections 2024-03-23T14:57:35.725 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:35.721+0000 7f1b9e5b4640 1 -- 172.21.15.164:0/480104458 wait complete. 2024-03-23T14:57:35.725 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:35.721+0000 7f1b9e5b4640 1 Processor -- start 2024-03-23T14:57:35.725 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:35.721+0000 7f1b9e5b4640 1 -- start start 2024-03-23T14:57:35.727 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:35.721+0000 7f1b9e5b4640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b9809def0 0x7f1b980b3730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:35.727 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:35.721+0000 7f1b9e5b4640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1b980add50 0x7f1b9814d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:35.727 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:35.721+0000 7f1b9e5b4640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1b98147cb0 0x7f1b9814d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:35.727 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:35.721+0000 7f1b9e5b4640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1b9809b690 con 0x7f1b9809def0 2024-03-23T14:57:35.727 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:35.721+0000 7f1b9e5b4640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f1b9814eb30 con 0x7f1b98147cb0 2024-03-23T14:57:35.727 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:35.721+0000 7f1b9e5b4640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f1b9814e570 con 0x7f1b980add50 2024-03-23T14:57:35.727 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:35.721+0000 7f1b97fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b9809def0 0x7f1b980b3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:35.728 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:35.721+0000 7f1b97fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b9809def0 0x7f1b980b3730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:33186/0 (socket says 172.21.15.164:33186) 2024-03-23T14:57:35.728 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:35.721+0000 7f1b97fff640 1 -- 172.21.15.164:0/642495976 learned_addr learned my addr 172.21.15.164:0/642495976 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:35.728 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:35.721+0000 7f1b977fe640 1 --2- 172.21.15.164:0/642495976 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1b980add50 0x7f1b9814d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:35.728 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:35.721+0000 7f1b9cb51640 1 --2- 172.21.15.164:0/642495976 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1b98147cb0 0x7f1b9814d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:35.728 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:35.721+0000 7f1b97fff640 1 -- 172.21.15.164:0/642495976 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1b980add50 msgr2=0x7f1b9814d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:35.728 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:35.721+0000 7f1b97fff640 1 --2- 172.21.15.164:0/642495976 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1b980add50 0x7f1b9814d180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:35.728 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:35.721+0000 7f1b97fff640 1 -- 172.21.15.164:0/642495976 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1b98147cb0 msgr2=0x7f1b9814d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:35.728 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:35.721+0000 7f1b97fff640 1 --2- 172.21.15.164:0/642495976 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1b98147cb0 0x7f1b9814d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:35.728 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:35.721+0000 7f1b97fff640 1 -- 172.21.15.164:0/642495976 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b90009420 con 0x7f1b9809def0 2024-03-23T14:57:35.728 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:35.721+0000 7f1b97fff640 1 --2- 172.21.15.164:0/642495976 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b9809def0 0x7f1b980b3730 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f1b8800c7c0 tx=0x7f1b8800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:35.728 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:35.721+0000 7f1b94ff9640 1 -- 172.21.15.164:0/642495976 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b8800a640 con 0x7f1b9809def0 2024-03-23T14:57:35.728 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:35.721+0000 7f1b94ff9640 1 -- 172.21.15.164:0/642495976 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b8800a7c0 con 0x7f1b9809def0 2024-03-23T14:57:35.728 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:35.721+0000 7f1b9e5b4640 1 -- 172.21.15.164:0/642495976 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b980b4e70 con 0x7f1b9809def0 2024-03-23T14:57:35.728 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:35.721+0000 7f1b94ff9640 1 -- 172.21.15.164:0/642495976 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b8800ab70 con 0x7f1b9809def0 2024-03-23T14:57:35.728 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:35.721+0000 7f1b9e5b4640 1 -- 172.21.15.164:0/642495976 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b9809b8e0 con 0x7f1b9809def0 2024-03-23T14:57:35.729 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:35.725+0000 7f1b9e5b4640 1 -- 172.21.15.164:0/642495976 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b9814cea0 con 0x7f1b9809def0 2024-03-23T14:57:35.729 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:35.725+0000 7f1b94ff9640 1 -- 172.21.15.164:0/642495976 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7f1b8800acd0 con 0x7f1b9809def0 2024-03-23T14:57:35.729 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:35.725+0000 7f1b94ff9640 1 --2- 172.21.15.164:0/642495976 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f1b840672d0 0x7f1b84069790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:35.729 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:35.725+0000 7f1b977fe640 1 --2- 172.21.15.164:0/642495976 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f1b840672d0 0x7f1b84069790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:35.729 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:35.725+0000 7f1b94ff9640 1 -- 172.21.15.164:0/642495976 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 4857+0+0 (secure 0 0 0) 0x7f1b88084400 con 0x7f1b9809def0 2024-03-23T14:57:35.730 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:35.725+0000 7f1b977fe640 1 --2- 172.21.15.164:0/642495976 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f1b840672d0 0x7f1b84069790 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c006fd0 tx=0x7f1b8c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:35.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:35.729+0000 7f1b94ff9640 1 -- 172.21.15.164:0/642495976 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7f1b9814cea0 con 0x7f1b9809def0 2024-03-23T14:57:35.947 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:35.941+0000 7f1b9e5b4640 1 -- 172.21.15.164:0/642495976 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f1b9814cea0 con 0x7f1b9809def0 2024-03-23T14:57:35.947 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:35.941+0000 7f1b94ff9640 1 -- 172.21.15.164:0/642495976 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v6) ==== 80+0+5207 (secure 0 0 0) 0x7f1b9814cea0 con 0x7f1b9809def0 2024-03-23T14:57:35.947 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-23T14:57:35.947 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-03-23T14:57:35.948 INFO:teuthology.orchestra.run.smithi164.stdout: "auth_dump": [ 2024-03-23T14:57:35.948 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-03-23T14:57:35.948 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "mds.a", 2024-03-23T14:57:35.948 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCA7f5lorXGNhAAjdjj6HC1um5/oohvuTLBAw==", 2024-03-23T14:57:35.948 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-03-23T14:57:35.948 INFO:teuthology.orchestra.run.smithi164.stdout: "mds": "allow", 2024-03-23T14:57:35.948 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow *", 2024-03-23T14:57:35.948 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow *", 2024-03-23T14:57:35.948 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow *" 2024-03-23T14:57:35.948 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-03-23T14:57:35.948 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-03-23T14:57:35.948 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-03-23T14:57:35.948 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "mds.b", 2024-03-23T14:57:35.948 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCB7f5lZfVOARAAVbo4snMrB3dMWB8BpS9UHA==", 2024-03-23T14:57:35.948 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-03-23T14:57:35.949 INFO:teuthology.orchestra.run.smithi164.stdout: "mds": "allow", 2024-03-23T14:57:35.949 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow *", 2024-03-23T14:57:35.949 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow *", 2024-03-23T14:57:35.949 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow *" 2024-03-23T14:57:35.949 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-03-23T14:57:35.949 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-03-23T14:57:35.949 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-03-23T14:57:35.949 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "osd.0", 2024-03-23T14:57:35.949 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCE7f5l0od2BhAAxjRnHvlZPAOTJcg/vVeEHg==", 2024-03-23T14:57:35.949 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-03-23T14:57:35.949 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow profile osd", 2024-03-23T14:57:35.949 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow profile osd", 2024-03-23T14:57:35.949 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow *" 2024-03-23T14:57:35.949 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-03-23T14:57:35.949 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-03-23T14:57:35.950 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-03-23T14:57:35.950 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "osd.1", 2024-03-23T14:57:35.950 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCH7f5lFsGOCxAAMHsOolSe7Mk/oxuT/crNuQ==", 2024-03-23T14:57:35.950 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-03-23T14:57:35.950 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow profile osd", 2024-03-23T14:57:35.950 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow profile osd", 2024-03-23T14:57:35.950 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow *" 2024-03-23T14:57:35.950 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-03-23T14:57:35.950 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-03-23T14:57:35.950 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-03-23T14:57:35.950 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "osd.2", 2024-03-23T14:57:35.950 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCK7f5lNig1EhAAr1K4ulKZ5udSrhqmWOjsOg==", 2024-03-23T14:57:35.950 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-03-23T14:57:35.950 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow profile osd", 2024-03-23T14:57:35.950 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow profile osd", 2024-03-23T14:57:35.951 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow *" 2024-03-23T14:57:35.951 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-03-23T14:57:35.951 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-03-23T14:57:35.951 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-03-23T14:57:35.951 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "osd.3", 2024-03-23T14:57:35.951 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCN7f5lM7A2GRAAkLg1AGmOL1GTDqrrBAsC/Q==", 2024-03-23T14:57:35.951 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-03-23T14:57:35.951 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow profile osd", 2024-03-23T14:57:35.951 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow profile osd", 2024-03-23T14:57:35.951 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow *" 2024-03-23T14:57:35.951 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-03-23T14:57:35.951 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-03-23T14:57:35.951 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-03-23T14:57:35.951 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "osd.4", 2024-03-23T14:57:35.952 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCT7f5lP3FwHRAAspF4SEUl4WLR8bQJGKnJfQ==", 2024-03-23T14:57:35.952 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-03-23T14:57:35.952 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow profile osd", 2024-03-23T14:57:35.952 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow profile osd", 2024-03-23T14:57:35.952 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow *" 2024-03-23T14:57:35.952 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-03-23T14:57:35.952 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-03-23T14:57:35.952 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-03-23T14:57:35.952 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "osd.5", 2024-03-23T14:57:35.952 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCW7f5l24HAIhAAt6VXAYDqJlBP317/cnwJbQ==", 2024-03-23T14:57:35.952 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-03-23T14:57:35.952 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow profile osd", 2024-03-23T14:57:35.952 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow profile osd", 2024-03-23T14:57:35.952 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow *" 2024-03-23T14:57:35.952 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-03-23T14:57:35.953 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-03-23T14:57:35.953 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-03-23T14:57:35.953 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "osd.6", 2024-03-23T14:57:35.953 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCZ7f5lpQfmKRAAnME++cPS67LD/BgEXA931w==", 2024-03-23T14:57:35.953 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-03-23T14:57:35.953 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow profile osd", 2024-03-23T14:57:35.953 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow profile osd", 2024-03-23T14:57:35.953 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow *" 2024-03-23T14:57:35.953 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-03-23T14:57:35.953 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-03-23T14:57:35.953 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-03-23T14:57:35.953 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "osd.7", 2024-03-23T14:57:35.953 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCc7f5lIlQTMxAAAZNbA78j5XuIcbfNLgQsuA==", 2024-03-23T14:57:35.953 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-03-23T14:57:35.953 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow profile osd", 2024-03-23T14:57:35.954 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow profile osd", 2024-03-23T14:57:35.954 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow *" 2024-03-23T14:57:35.954 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-03-23T14:57:35.954 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-03-23T14:57:35.954 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-03-23T14:57:35.954 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "client.0", 2024-03-23T14:57:35.954 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCB7f5ldBX+BhAAqsE6K6jE0UUaUO+MTCYmOg==", 2024-03-23T14:57:35.954 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-03-23T14:57:35.954 INFO:teuthology.orchestra.run.smithi164.stdout: "mds": "allow", 2024-03-23T14:57:35.954 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow r", 2024-03-23T14:57:35.954 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow rw", 2024-03-23T14:57:35.954 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow rwx" 2024-03-23T14:57:35.954 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-03-23T14:57:35.955 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-03-23T14:57:35.955 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-03-23T14:57:35.955 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "client.admin", 2024-03-23T14:57:35.955 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCA7f5l9GyVHxAAlaySBix8zvAMgGItIkL0yA==", 2024-03-23T14:57:35.955 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-03-23T14:57:35.955 INFO:teuthology.orchestra.run.smithi164.stdout: "mds": "allow *", 2024-03-23T14:57:35.955 INFO:teuthology.orchestra.run.smithi164.stdout: "mgr": "allow *", 2024-03-23T14:57:35.955 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow *", 2024-03-23T14:57:35.955 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow *" 2024-03-23T14:57:35.955 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-03-23T14:57:35.955 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-03-23T14:57:35.955 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-03-23T14:57:35.955 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "client.bootstrap-mds", 2024-03-23T14:57:35.956 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCi7f5lBrH6FRAAippxAhDdlQxVAAL1JrYnbQ==", 2024-03-23T14:57:35.956 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-03-23T14:57:35.956 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow profile bootstrap-mds" 2024-03-23T14:57:35.956 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-03-23T14:57:35.956 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-03-23T14:57:35.956 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-03-23T14:57:35.956 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "client.bootstrap-mgr", 2024-03-23T14:57:35.956 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCi7f5lgcn6FRAAYmI6Oj1hKN8oWewHft5vfg==", 2024-03-23T14:57:35.956 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-03-23T14:57:35.956 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow profile bootstrap-mgr" 2024-03-23T14:57:35.956 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-03-23T14:57:35.956 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-03-23T14:57:35.956 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-03-23T14:57:35.956 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "client.bootstrap-osd", 2024-03-23T14:57:35.957 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCi7f5laeH6FRAABK6XnOQCBiC4j4A4vbHH3Q==", 2024-03-23T14:57:35.957 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-03-23T14:57:35.957 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow profile bootstrap-osd" 2024-03-23T14:57:35.957 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-03-23T14:57:35.957 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-03-23T14:57:35.957 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-03-23T14:57:35.957 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "client.bootstrap-rbd", 2024-03-23T14:57:35.957 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCi7f5lfyD7FRAAfuqEjk18mqo5zHtRzkxveA==", 2024-03-23T14:57:35.957 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-03-23T14:57:35.957 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow profile bootstrap-rbd" 2024-03-23T14:57:35.957 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-03-23T14:57:35.957 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-03-23T14:57:35.957 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-03-23T14:57:35.957 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-03-23T14:57:35.958 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCi7f5l2zj7FRAAHeJ3BWLppu4/Oa1r7HrpSQ==", 2024-03-23T14:57:35.958 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-03-23T14:57:35.958 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-03-23T14:57:35.958 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-03-23T14:57:35.958 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-03-23T14:57:35.958 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-03-23T14:57:35.958 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "client.bootstrap-rgw", 2024-03-23T14:57:35.958 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCi7f5lEk/7FRAAUNhpqkJsUocLZGK0lpWeZQ==", 2024-03-23T14:57:35.958 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-03-23T14:57:35.958 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow profile bootstrap-rgw" 2024-03-23T14:57:35.958 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-03-23T14:57:35.958 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-03-23T14:57:35.958 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-03-23T14:57:35.959 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "mgr.x", 2024-03-23T14:57:35.959 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCA7f5l7BaWMxAAeIPg+NXPYnVBG+rK9VUZAg==", 2024-03-23T14:57:35.959 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-03-23T14:57:35.959 INFO:teuthology.orchestra.run.smithi164.stdout: "mds": "allow *", 2024-03-23T14:57:35.959 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow profile mgr", 2024-03-23T14:57:35.959 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow *" 2024-03-23T14:57:35.959 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-03-23T14:57:35.959 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-03-23T14:57:35.959 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-03-23T14:57:35.959 INFO:teuthology.orchestra.run.smithi164.stdout: "entity": "mgr.y", 2024-03-23T14:57:35.959 INFO:teuthology.orchestra.run.smithi164.stdout: "key": "AQCA7f5lZsO0MBAAuIBRk8br/Uai/75hGuHnpA==", 2024-03-23T14:57:35.959 INFO:teuthology.orchestra.run.smithi164.stdout: "caps": { 2024-03-23T14:57:35.959 INFO:teuthology.orchestra.run.smithi164.stdout: "mds": "allow *", 2024-03-23T14:57:35.959 INFO:teuthology.orchestra.run.smithi164.stdout: "mon": "allow profile mgr", 2024-03-23T14:57:35.959 INFO:teuthology.orchestra.run.smithi164.stdout: "osd": "allow *" 2024-03-23T14:57:35.960 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-03-23T14:57:35.960 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-03-23T14:57:35.960 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-03-23T14:57:35.960 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-03-23T14:57:35.961 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:35.945+0000 7f1b9e5b4640 1 -- 172.21.15.164:0/642495976 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f1b840672d0 msgr2=0x7f1b84069790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:35.961 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:35.945+0000 7f1b9e5b4640 1 --2- 172.21.15.164:0/642495976 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f1b840672d0 0x7f1b84069790 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c006fd0 tx=0x7f1b8c008040 comp rx=0 tx=0).stop 2024-03-23T14:57:35.961 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:35.945+0000 7f1b9e5b4640 1 -- 172.21.15.164:0/642495976 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b9809def0 msgr2=0x7f1b980b3730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:35.961 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:35.945+0000 7f1b9e5b4640 1 --2- 172.21.15.164:0/642495976 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b9809def0 0x7f1b980b3730 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f1b8800c7c0 tx=0x7f1b8800cc80 comp rx=0 tx=0).stop 2024-03-23T14:57:35.961 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:35.945+0000 7f1b9e5b4640 1 -- 172.21.15.164:0/642495976 shutdown_connections 2024-03-23T14:57:35.961 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:35.945+0000 7f1b9e5b4640 1 --2- 172.21.15.164:0/642495976 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f1b840672d0 0x7f1b84069790 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:35.961 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:35.945+0000 7f1b9e5b4640 1 --2- 172.21.15.164:0/642495976 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1b98147cb0 0x7f1b9814d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:35.961 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:35.945+0000 7f1b9e5b4640 1 --2- 172.21.15.164:0/642495976 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1b980add50 0x7f1b9814d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:35.961 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:35.945+0000 7f1b9e5b4640 1 --2- 172.21.15.164:0/642495976 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b9809def0 0x7f1b980b3730 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:35.961 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:35.945+0000 7f1b9e5b4640 1 -- 172.21.15.164:0/642495976 >> 172.21.15.164:0/642495976 conn(0x7f1b980b8ca0 msgr2=0x7f1b98146ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:35.961 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:35.945+0000 7f1b9e5b4640 1 -- 172.21.15.164:0/642495976 shutdown_connections 2024-03-23T14:57:35.961 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:35.945+0000 7f1b9e5b4640 1 -- 172.21.15.164:0/642495976 wait complete. 2024-03-23T14:57:35.967 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-03-23T14:57:36.082 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.077+0000 7ff64b1d1640 1 Processor -- start 2024-03-23T14:57:36.082 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.077+0000 7ff64b1d1640 1 -- start start 2024-03-23T14:57:36.084 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.077+0000 7ff64b1d1640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7ff64406bf70 0x7ff6440aec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:36.084 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.077+0000 7ff64b1d1640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff644072cb0 con 0x7ff64406c350 2024-03-23T14:57:36.085 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.077+0000 7ff64b1d1640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6440b53e0 con 0x7ff64409da80 2024-03-23T14:57:36.085 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.077+0000 7ff64b1d1640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 -- 0x7ff6440ae770 con 0x7ff64406bf70 2024-03-23T14:57:36.085 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.077+0000 7ff643fff640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7ff64406c350 0x7ff6440b34d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:58754/0 (socket says 172.21.15.164:58754) 2024-03-23T14:57:36.085 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.077+0000 7ff643fff640 1 -- 172.21.15.164:0/3556041450 learned_addr learned my addr 172.21.15.164:0/3556041450 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-03-23T14:57:36.085 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.077+0000 7ff648f6d640 1 --2- 172.21.15.164:0/3556041450 >> v2:172.21.15.173:3300/0 conn(0x7ff64406bf70 0x7ff6440aec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:36.085 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.077+0000 7ff6437fe640 1 -- 172.21.15.164:0/3556041450 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2155092754 0 0) 0x7ff644072cb0 con 0x7ff64406c350 2024-03-23T14:57:36.085 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.077+0000 7ff6437fe640 1 -- 172.21.15.164:0/3556041450 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff630003530 con 0x7ff64406c350 2024-03-23T14:57:36.085 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.077+0000 7ff6437fe640 1 -- 172.21.15.164:0/3556041450 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2718786193 0 0) 0x7ff630003530 con 0x7ff64406c350 2024-03-23T14:57:36.085 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.077+0000 7ff6437fe640 1 -- 172.21.15.164:0/3556041450 >> v2:172.21.15.173:3300/0 conn(0x7ff64406bf70 msgr2=0x7ff6440aec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:36.085 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.077+0000 7ff6437fe640 1 --2- 172.21.15.164:0/3556041450 >> v2:172.21.15.173:3300/0 conn(0x7ff64406bf70 0x7ff6440aec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:36.085 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.077+0000 7ff6437fe640 1 -- 172.21.15.164:0/3556041450 >> v1:172.21.15.173:6789/0 conn(0x7ff64409da80 legacy=0x7ff64414ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:36.085 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.077+0000 7ff6437fe640 1 -- 172.21.15.164:0/3556041450 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6440b3be0 con 0x7ff64406c350 2024-03-23T14:57:36.085 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.077+0000 7ff6437fe640 1 -- 172.21.15.164:0/3556041450 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2526390548 0 0) 0x7ff62c001680 con 0x7ff64406c350 2024-03-23T14:57:36.085 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.077+0000 7ff6437fe640 1 -- 172.21.15.164:0/3556041450 >> v1:172.21.15.164:6789/0 conn(0x7ff64406c350 legacy=0x7ff6440b34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:36.085 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.077+0000 7ff6437fe640 1 --2- 172.21.15.164:0/3556041450 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff630003980 0x7ff630003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:36.086 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.077+0000 7ff6437fe640 1 --2- 172.21.15.164:0/3556041450 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff6300047d0 0x7ff630004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:36.086 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.077+0000 7ff6437fe640 1 --2- 172.21.15.164:0/3556041450 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff6300052b0 0x7ff63000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:36.086 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.077+0000 7ff6437fe640 1 -- 172.21.15.164:0/3556041450 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff6440ae770 con 0x7ff6300052b0 2024-03-23T14:57:36.086 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.077+0000 7ff6437fe640 1 -- 172.21.15.164:0/3556041450 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ff63000c180 con 0x7ff630003980 2024-03-23T14:57:36.086 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.077+0000 7ff6437fe640 1 -- 172.21.15.164:0/3556041450 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7ff63000c340 con 0x7ff6300047d0 2024-03-23T14:57:36.086 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.081+0000 7ff64976e640 1 --2- 172.21.15.164:0/3556041450 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff6300052b0 0x7ff63000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:36.086 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.081+0000 7ff643fff640 1 --2- 172.21.15.164:0/3556041450 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff6300047d0 0x7ff630004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:36.086 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.081+0000 7ff648f6d640 1 --2- 172.21.15.164:0/3556041450 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff630003980 0x7ff630003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:36.086 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.081+0000 7ff64976e640 1 -- 172.21.15.164:0/3556041450 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff6300047d0 msgr2=0x7ff630004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:36.086 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.081+0000 7ff64976e640 1 --2- 172.21.15.164:0/3556041450 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff6300047d0 0x7ff630004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:36.086 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.081+0000 7ff64976e640 1 -- 172.21.15.164:0/3556041450 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff630003980 msgr2=0x7ff630003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:36.086 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.081+0000 7ff64976e640 1 --2- 172.21.15.164:0/3556041450 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff630003980 0x7ff630003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:36.086 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.081+0000 7ff64976e640 1 -- 172.21.15.164:0/3556041450 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff630003530 con 0x7ff6300052b0 2024-03-23T14:57:36.088 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.081+0000 7ff64976e640 1 -- 172.21.15.164:0/3556041450 reap_dead start 2024-03-23T14:57:36.088 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.081+0000 7ff64976e640 1 --2- 172.21.15.164:0/3556041450 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff6300052b0 0x7ff63000b990 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7ff634001750 tx=0x7ff63401c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3382b6c2c74f5581 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:36.088 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.081+0000 7ff6437fe640 1 -- 172.21.15.164:0/3556041450 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6340a4ea0 con 0x7ff6300052b0 2024-03-23T14:57:36.088 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.081+0000 7ff6437fe640 1 -- 172.21.15.164:0/3556041450 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6340a7050 con 0x7ff6300052b0 2024-03-23T14:57:36.088 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.081+0000 7ff6437fe640 1 -- 172.21.15.164:0/3556041450 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6340054f0 con 0x7ff6300052b0 2024-03-23T14:57:36.088 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.081+0000 7ff64b1d1640 1 -- 172.21.15.164:0/3556041450 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff6300052b0 msgr2=0x7ff63000b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:36.088 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.081+0000 7ff64b1d1640 1 --2- 172.21.15.164:0/3556041450 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff6300052b0 0x7ff63000b990 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7ff634001750 tx=0x7ff63401c3e0 comp rx=0 tx=0).stop 2024-03-23T14:57:36.088 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.081+0000 7ff64b1d1640 1 -- 172.21.15.164:0/3556041450 shutdown_connections 2024-03-23T14:57:36.088 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.081+0000 7ff64b1d1640 1 --2- 172.21.15.164:0/3556041450 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff6300052b0 0x7ff63000b990 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:36.088 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.081+0000 7ff64b1d1640 1 -- 172.21.15.164:0/3556041450 >> 172.21.15.164:0/3556041450 conn(0x7ff6440b8ca0 msgr2=0x7ff6440b90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:36.088 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.081+0000 7ff64b1d1640 1 -- 172.21.15.164:0/3556041450 shutdown_connections 2024-03-23T14:57:36.088 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.081+0000 7ff64b1d1640 1 -- 172.21.15.164:0/3556041450 wait complete. 2024-03-23T14:57:36.088 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.085+0000 7ff64b1d1640 1 Processor -- start 2024-03-23T14:57:36.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.085+0000 7ff64b1d1640 1 -- start start 2024-03-23T14:57:36.091 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.085+0000 7ff64b1d1640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff6300052b0 0x7ff6441411e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:36.091 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.085+0000 7ff64b1d1640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff644142650 0x7ff644141b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:36.091 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.085+0000 7ff64b1d1640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff6441463a0 0x7ff64409bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:36.091 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.085+0000 7ff64b1d1640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff644139f20 con 0x7ff644142650 2024-03-23T14:57:36.091 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.085+0000 7ff64b1d1640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ff64413f8e0 con 0x7ff6441463a0 2024-03-23T14:57:36.091 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.085+0000 7ff64b1d1640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7ff644146960 con 0x7ff6300052b0 2024-03-23T14:57:36.091 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.085+0000 7ff643fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff644142650 0x7ff644141b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:36.091 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.085+0000 7ff643fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff644142650 0x7ff644141b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:33198/0 (socket says 172.21.15.164:33198) 2024-03-23T14:57:36.092 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.085+0000 7ff643fff640 1 -- 172.21.15.164:0/546501517 learned_addr learned my addr 172.21.15.164:0/546501517 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:36.092 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.085+0000 7ff64976e640 1 --2- 172.21.15.164:0/546501517 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff6441463a0 0x7ff64409bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:36.092 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.085+0000 7ff648f6d640 1 --2- 172.21.15.164:0/546501517 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff6300052b0 0x7ff6441411e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:36.092 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.085+0000 7ff643fff640 1 -- 172.21.15.164:0/546501517 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff6300052b0 msgr2=0x7ff6441411e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:36.092 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.085+0000 7ff643fff640 1 --2- 172.21.15.164:0/546501517 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff6300052b0 0x7ff6441411e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:36.092 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.085+0000 7ff643fff640 1 -- 172.21.15.164:0/546501517 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff6441463a0 msgr2=0x7ff64409bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:36.092 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.085+0000 7ff643fff640 1 --2- 172.21.15.164:0/546501517 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff6441463a0 0x7ff64409bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:36.092 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.085+0000 7ff643fff640 1 -- 172.21.15.164:0/546501517 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff644146c40 con 0x7ff644142650 2024-03-23T14:57:36.092 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.085+0000 7ff643fff640 1 --2- 172.21.15.164:0/546501517 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff644142650 0x7ff644141b90 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7ff62c00e7b0 tx=0x7ff62c00ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:36.092 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.085+0000 7ff6417fa640 1 -- 172.21.15.164:0/546501517 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff62c00c6c0 con 0x7ff644142650 2024-03-23T14:57:36.092 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.085+0000 7ff6417fa640 1 -- 172.21.15.164:0/546501517 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff62c00c840 con 0x7ff644142650 2024-03-23T14:57:36.092 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.085+0000 7ff64b1d1640 1 -- 172.21.15.164:0/546501517 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff644140cc0 con 0x7ff644142650 2024-03-23T14:57:36.092 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.085+0000 7ff6417fa640 1 -- 172.21.15.164:0/546501517 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff62c00cbf0 con 0x7ff644142650 2024-03-23T14:57:36.092 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.085+0000 7ff64b1d1640 1 -- 172.21.15.164:0/546501517 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff64413d960 con 0x7ff644142650 2024-03-23T14:57:36.092 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.085+0000 7ff64b1d1640 1 -- 172.21.15.164:0/546501517 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6440adb60 con 0x7ff644142650 2024-03-23T14:57:36.093 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.085+0000 7ff6417fa640 1 -- 172.21.15.164:0/546501517 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7ff62c00cd50 con 0x7ff644142650 2024-03-23T14:57:36.093 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.085+0000 7ff6417fa640 1 --2- 172.21.15.164:0/546501517 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7ff6240672a0 0x7ff624069760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:36.093 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.085+0000 7ff6417fa640 1 -- 172.21.15.164:0/546501517 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 4857+0+0 (secure 0 0 0) 0x7ff62c084ad0 con 0x7ff644142650 2024-03-23T14:57:36.093 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.085+0000 7ff648f6d640 1 --2- 172.21.15.164:0/546501517 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7ff6240672a0 0x7ff624069760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:36.093 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.085+0000 7ff648f6d640 1 --2- 172.21.15.164:0/546501517 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7ff6240672a0 0x7ff624069760 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff64413ffe0 tx=0x7ff63c00b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:36.095 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.089+0000 7ff6417fa640 1 -- 172.21.15.164:0/546501517 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7ff6440adb60 con 0x7ff644142650 2024-03-23T14:57:36.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.305+0000 7ff64b1d1640 1 -- 172.21.15.164:0/546501517 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) -- 0x7ff644140f00 con 0x7ff644142650 2024-03-23T14:57:36.314 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.309+0000 7ff6417fa640 1 -- 172.21.15.164:0/546501517 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v7) ==== 90+0+0 (secure 0 0 0) 0x7ff644140f00 con 0x7ff644142650 2024-03-23T14:57:36.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.313+0000 7ff64b1d1640 1 -- 172.21.15.164:0/546501517 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7ff6240672a0 msgr2=0x7ff624069760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:36.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.313+0000 7ff64b1d1640 1 --2- 172.21.15.164:0/546501517 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7ff6240672a0 0x7ff624069760 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff64413ffe0 tx=0x7ff63c00b040 comp rx=0 tx=0).stop 2024-03-23T14:57:36.316 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.313+0000 7ff64b1d1640 1 -- 172.21.15.164:0/546501517 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff644142650 msgr2=0x7ff644141b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:36.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.313+0000 7ff64b1d1640 1 --2- 172.21.15.164:0/546501517 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff644142650 0x7ff644141b90 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7ff62c00e7b0 tx=0x7ff62c00ec70 comp rx=0 tx=0).stop 2024-03-23T14:57:36.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.313+0000 7ff64b1d1640 1 -- 172.21.15.164:0/546501517 shutdown_connections 2024-03-23T14:57:36.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.313+0000 7ff64b1d1640 1 --2- 172.21.15.164:0/546501517 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7ff6240672a0 0x7ff624069760 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:36.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.313+0000 7ff64b1d1640 1 --2- 172.21.15.164:0/546501517 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff6441463a0 0x7ff64409bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:36.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.313+0000 7ff64b1d1640 1 --2- 172.21.15.164:0/546501517 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff644142650 0x7ff644141b90 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:36.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.313+0000 7ff64b1d1640 1 --2- 172.21.15.164:0/546501517 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff6300052b0 0x7ff6441411e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:36.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.313+0000 7ff64b1d1640 1 -- 172.21.15.164:0/546501517 >> 172.21.15.164:0/546501517 conn(0x7ff6440b8ca0 msgr2=0x7ff6440b9080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:36.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.313+0000 7ff64b1d1640 1 -- 172.21.15.164:0/546501517 shutdown_connections 2024-03-23T14:57:36.317 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.313+0000 7ff64b1d1640 1 -- 172.21.15.164:0/546501517 wait complete. 2024-03-23T14:57:36.332 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-03-23T14:57:36.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.437+0000 7f2b0ab76640 1 Processor -- start 2024-03-23T14:57:36.441 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.437+0000 7f2b0ab76640 1 -- start start 2024-03-23T14:57:36.443 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.437+0000 7f2b0ab76640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f2b0414eb80 0x7f2b04150f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:36.443 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.437+0000 7f2b0ab76640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b040b51d0 con 0x7f2b040b0230 2024-03-23T14:57:36.443 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.437+0000 7f2b0ab76640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b040b0610 con 0x7f2b04148cf0 2024-03-23T14:57:36.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.437+0000 7f2b0ab76640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 -- 0x7f2b040b1500 con 0x7f2b0414eb80 2024-03-23T14:57:36.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.437+0000 7f2b08912640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f2b040b0230 0x7f2b040b34d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:58768/0 (socket says 172.21.15.164:58768) 2024-03-23T14:57:36.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.437+0000 7f2b08912640 1 -- 172.21.15.164:0/1452353690 learned_addr learned my addr 172.21.15.164:0/1452353690 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-03-23T14:57:36.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.437+0000 7f2b09113640 1 --2- 172.21.15.164:0/1452353690 >> v2:172.21.15.173:3300/0 conn(0x7f2b0414eb80 0x7f2b04150f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:36.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.437+0000 7f2b037fe640 1 -- 172.21.15.164:0/1452353690 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1799546335 0 0) 0x7f2b040b51d0 con 0x7f2b040b0230 2024-03-23T14:57:36.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.437+0000 7f2b037fe640 1 -- 172.21.15.164:0/1452353690 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2aec003530 con 0x7f2b040b0230 2024-03-23T14:57:36.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.437+0000 7f2b037fe640 1 -- 172.21.15.164:0/1452353690 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 976996026 0 0) 0x7f2aec003530 con 0x7f2b040b0230 2024-03-23T14:57:36.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.437+0000 7f2b037fe640 1 -- 172.21.15.164:0/1452353690 >> v2:172.21.15.173:3300/0 conn(0x7f2b0414eb80 msgr2=0x7f2b04150f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:36.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.437+0000 7f2b037fe640 1 --2- 172.21.15.164:0/1452353690 >> v2:172.21.15.173:3300/0 conn(0x7f2b0414eb80 0x7f2b04150f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:36.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.437+0000 7f2b037fe640 1 -- 172.21.15.164:0/1452353690 >> v1:172.21.15.173:6789/0 conn(0x7f2b04148cf0 legacy=0x7f2b04149180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:36.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.437+0000 7f2b037fe640 1 -- 172.21.15.164:0/1452353690 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b0406d250 con 0x7f2b040b0230 2024-03-23T14:57:36.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.437+0000 7f2b037fe640 1 -- 172.21.15.164:0/1452353690 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2526390548 0 0) 0x7f2af4001680 con 0x7f2b040b0230 2024-03-23T14:57:36.444 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.437+0000 7f2b037fe640 1 -- 172.21.15.164:0/1452353690 >> v1:172.21.15.164:6789/0 conn(0x7f2b040b0230 legacy=0x7f2b040b34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:36.445 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.437+0000 7f2b037fe640 1 --2- 172.21.15.164:0/1452353690 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2aec003980 0x7f2aec003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:36.445 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.437+0000 7f2b037fe640 1 --2- 172.21.15.164:0/1452353690 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2aec0047d0 0x7f2aec004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:36.445 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.437+0000 7f2b037fe640 1 --2- 172.21.15.164:0/1452353690 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2aec005390 0x7f2aec00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:36.445 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.437+0000 7f2b037fe640 1 -- 172.21.15.164:0/1452353690 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2b040b1500 con 0x7f2aec0047d0 2024-03-23T14:57:36.445 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.437+0000 7f2b037fe640 1 -- 172.21.15.164:0/1452353690 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f2aec00c230 con 0x7f2aec003980 2024-03-23T14:57:36.445 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.437+0000 7f2b037fe640 1 -- 172.21.15.164:0/1452353690 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f2aec00c3f0 con 0x7f2aec005390 2024-03-23T14:57:36.445 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.437+0000 7f2b03fff640 1 --2- 172.21.15.164:0/1452353690 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2aec0047d0 0x7f2aec004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:36.445 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.437+0000 7f2b08912640 1 --2- 172.21.15.164:0/1452353690 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2aec003980 0x7f2aec003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:36.445 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.437+0000 7f2b09113640 1 --2- 172.21.15.164:0/1452353690 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2aec005390 0x7f2aec00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:36.445 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.437+0000 7f2b03fff640 1 -- 172.21.15.164:0/1452353690 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2aec005390 msgr2=0x7f2aec00b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:36.445 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.437+0000 7f2b03fff640 1 --2- 172.21.15.164:0/1452353690 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2aec005390 0x7f2aec00b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:36.445 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.437+0000 7f2b03fff640 1 -- 172.21.15.164:0/1452353690 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2aec003980 msgr2=0x7f2aec003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:36.445 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.437+0000 7f2b03fff640 1 --2- 172.21.15.164:0/1452353690 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2aec003980 0x7f2aec003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:36.445 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.437+0000 7f2b09113640 1 -- 172.21.15.164:0/1452353690 reap_dead start 2024-03-23T14:57:36.446 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.437+0000 7f2b03fff640 1 -- 172.21.15.164:0/1452353690 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2aec003530 con 0x7f2aec0047d0 2024-03-23T14:57:36.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.441+0000 7f2b03fff640 1 --2- 172.21.15.164:0/1452353690 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2aec0047d0 0x7f2aec004c50 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f2af8001780 tx=0x7f2af801c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0922c8dd71153f3 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:36.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.441+0000 7f2b037fe640 1 -- 172.21.15.164:0/1452353690 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af80052d0 con 0x7f2aec0047d0 2024-03-23T14:57:36.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.441+0000 7f2b037fe640 1 -- 172.21.15.164:0/1452353690 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2af80a7050 con 0x7f2aec0047d0 2024-03-23T14:57:36.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.441+0000 7f2b037fe640 1 -- 172.21.15.164:0/1452353690 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af8003600 con 0x7f2aec0047d0 2024-03-23T14:57:36.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.441+0000 7f2b0ab76640 1 -- 172.21.15.164:0/1452353690 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2aec0047d0 msgr2=0x7f2aec004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:36.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.441+0000 7f2b0ab76640 1 --2- 172.21.15.164:0/1452353690 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2aec0047d0 0x7f2aec004c50 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f2af8001780 tx=0x7f2af801c3d0 comp rx=0 tx=0).stop 2024-03-23T14:57:36.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.441+0000 7f2b0ab76640 1 -- 172.21.15.164:0/1452353690 shutdown_connections 2024-03-23T14:57:36.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.441+0000 7f2b0ab76640 1 --2- 172.21.15.164:0/1452353690 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2aec0047d0 0x7f2aec004c50 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:36.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.441+0000 7f2b0ab76640 1 -- 172.21.15.164:0/1452353690 >> 172.21.15.164:0/1452353690 conn(0x7f2b040b8ca0 msgr2=0x7f2b040b90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:36.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.441+0000 7f2b0ab76640 1 -- 172.21.15.164:0/1452353690 shutdown_connections 2024-03-23T14:57:36.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.441+0000 7f2b0ab76640 1 -- 172.21.15.164:0/1452353690 wait complete. 2024-03-23T14:57:36.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.441+0000 7f2b0ab76640 1 Processor -- start 2024-03-23T14:57:36.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.441+0000 7f2b0ab76640 1 -- start start 2024-03-23T14:57:36.450 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.445+0000 7f2b0ab76640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2aec0047d0 0x7f2b040af3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:36.450 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.445+0000 7f2b0ab76640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2b04144310 0x7f2b0413fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:36.450 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.445+0000 7f2b0ab76640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2b041405d0 0x7f2b04140a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:36.450 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.445+0000 7f2b0ab76640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2b041402a0 con 0x7f2b04144310 2024-03-23T14:57:36.450 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.445+0000 7f2b0ab76640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f2b040afa90 con 0x7f2b041405d0 2024-03-23T14:57:36.450 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.445+0000 7f2b0ab76640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f2b04142ab0 con 0x7f2aec0047d0 2024-03-23T14:57:36.450 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.445+0000 7f2b03fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2b04144310 0x7f2b0413fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:36.450 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.445+0000 7f2b03fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2b04144310 0x7f2b0413fd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:33208/0 (socket says 172.21.15.164:33208) 2024-03-23T14:57:36.450 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.445+0000 7f2b03fff640 1 -- 172.21.15.164:0/2925699002 learned_addr learned my addr 172.21.15.164:0/2925699002 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:36.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.445+0000 7f2b09113640 1 --2- 172.21.15.164:0/2925699002 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2b041405d0 0x7f2b04140a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:36.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.445+0000 7f2b03fff640 1 -- 172.21.15.164:0/2925699002 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2aec0047d0 msgr2=0x7f2b040af3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:36.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.445+0000 7f2b03fff640 1 --2- 172.21.15.164:0/2925699002 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2aec0047d0 0x7f2b040af3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:36.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.445+0000 7f2b03fff640 1 -- 172.21.15.164:0/2925699002 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2b041405d0 msgr2=0x7f2b04140a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:36.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.445+0000 7f2b03fff640 1 --2- 172.21.15.164:0/2925699002 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2b041405d0 0x7f2b04140a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:36.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.445+0000 7f2b03fff640 1 -- 172.21.15.164:0/2925699002 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b04141070 con 0x7f2b04144310 2024-03-23T14:57:36.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.445+0000 7f2b09113640 1 --2- 172.21.15.164:0/2925699002 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2b041405d0 0x7f2b04140a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T14:57:36.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.445+0000 7f2b03fff640 1 --2- 172.21.15.164:0/2925699002 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2b04144310 0x7f2b0413fd60 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f2af8005040 tx=0x7f2aec005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:36.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.445+0000 7f2b017fa640 1 -- 172.21.15.164:0/2925699002 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af80072f0 con 0x7f2b04144310 2024-03-23T14:57:36.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.445+0000 7f2b017fa640 1 -- 172.21.15.164:0/2925699002 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2af8007450 con 0x7f2b04144310 2024-03-23T14:57:36.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.445+0000 7f2b017fa640 1 -- 172.21.15.164:0/2925699002 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af80067e0 con 0x7f2b04144310 2024-03-23T14:57:36.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.445+0000 7f2b0ab76640 1 -- 172.21.15.164:0/2925699002 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b04141350 con 0x7f2b04144310 2024-03-23T14:57:36.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.445+0000 7f2b0ab76640 1 -- 172.21.15.164:0/2925699002 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b041e3ae0 con 0x7f2b04144310 2024-03-23T14:57:36.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.445+0000 7f2b0ab76640 1 -- 172.21.15.164:0/2925699002 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b0409e110 con 0x7f2b04144310 2024-03-23T14:57:36.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.445+0000 7f2b017fa640 1 -- 172.21.15.164:0/2925699002 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7f2af8006940 con 0x7f2b04144310 2024-03-23T14:57:36.452 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.445+0000 7f2b017fa640 1 --2- 172.21.15.164:0/2925699002 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f2af00672a0 0x7f2af0069760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:36.452 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.445+0000 7f2b017fa640 1 -- 172.21.15.164:0/2925699002 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 4857+0+0 (secure 0 0 0) 0x7f2af811ed60 con 0x7f2b04144310 2024-03-23T14:57:36.452 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.445+0000 7f2b08912640 1 --2- 172.21.15.164:0/2925699002 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f2af00672a0 0x7f2af0069760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:36.452 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.445+0000 7f2b08912640 1 --2- 172.21.15.164:0/2925699002 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f2af00672a0 0x7f2af0069760 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f2b040688b0 tx=0x7f2af4001000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:36.454 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.449+0000 7f2b017fa640 1 -- 172.21.15.164:0/2925699002 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7f2af80ef7a0 con 0x7f2b04144310 2024-03-23T14:57:36.659 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.653+0000 7f2b0ab76640 1 -- 172.21.15.164:0/2925699002 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) -- 0x7f2b0409dcc0 con 0x7f2b04144310 2024-03-23T14:57:36.662 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.657+0000 7f2b017fa640 1 -- 172.21.15.164:0/2925699002 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v8) ==== 90+0+0 (secure 0 0 0) 0x7f2b0409dcc0 con 0x7f2b04144310 2024-03-23T14:57:36.665 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.661+0000 7f2b0ab76640 1 -- 172.21.15.164:0/2925699002 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f2af00672a0 msgr2=0x7f2af0069760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:36.665 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.661+0000 7f2b0ab76640 1 --2- 172.21.15.164:0/2925699002 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f2af00672a0 0x7f2af0069760 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f2b040688b0 tx=0x7f2af4001000 comp rx=0 tx=0).stop 2024-03-23T14:57:36.665 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.661+0000 7f2b0ab76640 1 -- 172.21.15.164:0/2925699002 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2b04144310 msgr2=0x7f2b0413fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:36.665 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.661+0000 7f2b0ab76640 1 --2- 172.21.15.164:0/2925699002 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2b04144310 0x7f2b0413fd60 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f2af8005040 tx=0x7f2aec005800 comp rx=0 tx=0).stop 2024-03-23T14:57:36.665 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.661+0000 7f2b0ab76640 1 -- 172.21.15.164:0/2925699002 shutdown_connections 2024-03-23T14:57:36.665 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.661+0000 7f2b0ab76640 1 --2- 172.21.15.164:0/2925699002 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f2af00672a0 0x7f2af0069760 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:36.665 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.661+0000 7f2b0ab76640 1 --2- 172.21.15.164:0/2925699002 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2b041405d0 0x7f2b04140a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:36.665 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.661+0000 7f2b0ab76640 1 --2- 172.21.15.164:0/2925699002 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2b04144310 0x7f2b0413fd60 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:36.665 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.661+0000 7f2b0ab76640 1 --2- 172.21.15.164:0/2925699002 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2aec0047d0 0x7f2b040af3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:36.665 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.661+0000 7f2b0ab76640 1 -- 172.21.15.164:0/2925699002 >> 172.21.15.164:0/2925699002 conn(0x7f2b040b8ca0 msgr2=0x7f2b040b5560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:36.665 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.661+0000 7f2b0ab76640 1 -- 172.21.15.164:0/2925699002 shutdown_connections 2024-03-23T14:57:36.665 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.661+0000 7f2b0ab76640 1 -- 172.21.15.164:0/2925699002 wait complete. 2024-03-23T14:57:36.680 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-03-23T14:57:36.798 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.793+0000 7fb781680640 1 Processor -- start 2024-03-23T14:57:36.798 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.793+0000 7fb781680640 1 -- start start 2024-03-23T14:57:36.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.793+0000 7fb781680640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fb77c09def0 0x7fb77c0ae620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:36.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.793+0000 7fb781680640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fb77c0b4e90 0x7fb77c14e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:36.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.793+0000 7fb781680640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb77c09e2d0 con 0x7fb77c0b4a00 2024-03-23T14:57:36.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.793+0000 7fb781680640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fb77c146de0 con 0x7fb77c0b4e90 2024-03-23T14:57:36.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.793+0000 7fb781680640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 -- 0x7fb77c0b52e0 con 0x7fb77c09def0 2024-03-23T14:57:36.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.793+0000 7fb77b7fe640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fb77c0b4e90 0x7fb77c14e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:36.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.793+0000 7fb77b7fe640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fb77c0b4e90 0x7fb77c14e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:33210/0 (socket says 172.21.15.164:33210) 2024-03-23T14:57:36.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.793+0000 7fb77b7fe640 1 -- 172.21.15.164:0/1455486758 learned_addr learned my addr 172.21.15.164:0/1455486758 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:36.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.793+0000 7fb77affd640 1 --2- 172.21.15.164:0/1455486758 >> v2:172.21.15.173:3300/0 conn(0x7fb77c09def0 0x7fb77c0ae620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:36.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.793+0000 7fb77b7fe640 1 -- 172.21.15.164:0/1455486758 >> v2:172.21.15.173:3300/0 conn(0x7fb77c09def0 msgr2=0x7fb77c0ae620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:36.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.793+0000 7fb77b7fe640 1 --2- 172.21.15.164:0/1455486758 >> v2:172.21.15.173:3300/0 conn(0x7fb77c09def0 0x7fb77c0ae620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:36.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.793+0000 7fb77b7fe640 1 -- 172.21.15.164:0/1455486758 >> v1:172.21.15.173:6789/0 conn(0x7fb77c0b4a00 legacy=0x7fb77c147420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:36.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.793+0000 7fb77b7fe640 1 -- 172.21.15.164:0/1455486758 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb77c06cb70 con 0x7fb77c0b4e90 2024-03-23T14:57:36.801 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.797+0000 7fb77b7fe640 1 --2- 172.21.15.164:0/1455486758 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb77c0b4e90 0x7fb77c14e110 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fb77c06da30 tx=0x7fb77401ea90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=969e9af13324c45a server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:36.801 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.797+0000 7fb77a7fc640 1 -- 172.21.15.164:0/1455486758 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb77400b2d0 con 0x7fb77c0b4e90 2024-03-23T14:57:36.801 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.797+0000 7fb77a7fc640 1 -- 172.21.15.164:0/1455486758 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb77400b430 con 0x7fb77c0b4e90 2024-03-23T14:57:36.801 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.797+0000 7fb77a7fc640 1 -- 172.21.15.164:0/1455486758 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7740a7a70 con 0x7fb77c0b4e90 2024-03-23T14:57:36.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.797+0000 7fb781680640 1 -- 172.21.15.164:0/1455486758 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb77c0b4e90 msgr2=0x7fb77c14e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:36.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.797+0000 7fb781680640 1 --2- 172.21.15.164:0/1455486758 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb77c0b4e90 0x7fb77c14e110 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fb77c06da30 tx=0x7fb77401ea90 comp rx=0 tx=0).stop 2024-03-23T14:57:36.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.797+0000 7fb781680640 1 -- 172.21.15.164:0/1455486758 shutdown_connections 2024-03-23T14:57:36.803 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.797+0000 7fb781680640 1 --2- 172.21.15.164:0/1455486758 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb77c0b4e90 0x7fb77c14e110 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:36.803 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.797+0000 7fb781680640 1 --2- 172.21.15.164:0/1455486758 >> v2:172.21.15.173:3300/0 conn(0x7fb77c09def0 0x7fb77c0ae620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:36.803 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.797+0000 7fb781680640 1 -- 172.21.15.164:0/1455486758 >> 172.21.15.164:0/1455486758 conn(0x7fb77c0b8ca0 msgr2=0x7fb77c0b90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:36.803 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.797+0000 7fb781680640 1 -- 172.21.15.164:0/1455486758 shutdown_connections 2024-03-23T14:57:36.803 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.797+0000 7fb781680640 1 -- 172.21.15.164:0/1455486758 wait complete. 2024-03-23T14:57:36.805 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.797+0000 7fb781680640 1 Processor -- start 2024-03-23T14:57:36.805 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.797+0000 7fb781680640 1 -- start start 2024-03-23T14:57:36.805 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.801+0000 7fb781680640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb77c09def0 0x7fb77c140980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:36.805 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.801+0000 7fb781680640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb77c0b4a00 0x7fb77c0b0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:36.805 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.801+0000 7fb781680640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb77c0b4e90 0x7fb77c0b17d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:36.805 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.801+0000 7fb781680640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb77c0b0cf0 con 0x7fb77c0b4e90 2024-03-23T14:57:36.805 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.801+0000 7fb781680640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fb77c13bff0 con 0x7fb77c0b4a00 2024-03-23T14:57:36.805 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.801+0000 7fb781680640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fb77c14dd40 con 0x7fb77c09def0 2024-03-23T14:57:36.805 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.801+0000 7fb77b7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb77c0b4e90 0x7fb77c0b17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:36.805 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.801+0000 7fb77b7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb77c0b4e90 0x7fb77c0b17d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:33214/0 (socket says 172.21.15.164:33214) 2024-03-23T14:57:36.805 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.801+0000 7fb77b7fe640 1 -- 172.21.15.164:0/4194145229 learned_addr learned my addr 172.21.15.164:0/4194145229 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:36.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.801+0000 7fb77affd640 1 --2- 172.21.15.164:0/4194145229 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb77c09def0 0x7fb77c140980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:36.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.801+0000 7fb773fff640 1 --2- 172.21.15.164:0/4194145229 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb77c0b4a00 0x7fb77c0b0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:36.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.801+0000 7fb77b7fe640 1 -- 172.21.15.164:0/4194145229 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb77c09def0 msgr2=0x7fb77c140980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:36.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.801+0000 7fb77b7fe640 1 --2- 172.21.15.164:0/4194145229 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb77c09def0 0x7fb77c140980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:36.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.801+0000 7fb77b7fe640 1 -- 172.21.15.164:0/4194145229 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb77c0b4a00 msgr2=0x7fb77c0b0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:36.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.801+0000 7fb77b7fe640 1 --2- 172.21.15.164:0/4194145229 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb77c0b4a00 0x7fb77c0b0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:36.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.801+0000 7fb77b7fe640 1 -- 172.21.15.164:0/4194145229 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb77400bd40 con 0x7fb77c0b4e90 2024-03-23T14:57:36.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.801+0000 7fb77b7fe640 1 --2- 172.21.15.164:0/4194145229 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb77c0b4e90 0x7fb77c0b17d0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fb7740b1040 tx=0x7fb77401b830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:36.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.801+0000 7fb7737fe640 1 -- 172.21.15.164:0/4194145229 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb774009800 con 0x7fb77c0b4e90 2024-03-23T14:57:36.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.801+0000 7fb7737fe640 1 -- 172.21.15.164:0/4194145229 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb774009960 con 0x7fb77c0b4e90 2024-03-23T14:57:36.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.801+0000 7fb781680640 1 -- 172.21.15.164:0/4194145229 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb77c0addc0 con 0x7fb77c0b4e90 2024-03-23T14:57:36.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.801+0000 7fb7737fe640 1 -- 172.21.15.164:0/4194145229 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb774009d10 con 0x7fb77c0b4e90 2024-03-23T14:57:36.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.801+0000 7fb781680640 1 -- 172.21.15.164:0/4194145229 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb77c072620 con 0x7fb77c0b4e90 2024-03-23T14:57:36.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.801+0000 7fb781680640 1 -- 172.21.15.164:0/4194145229 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb77c072480 con 0x7fb77c0b4e90 2024-03-23T14:57:36.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.801+0000 7fb7737fe640 1 -- 172.21.15.164:0/4194145229 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7fb7740c3460 con 0x7fb77c0b4e90 2024-03-23T14:57:36.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.801+0000 7fb7737fe640 1 --2- 172.21.15.164:0/4194145229 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fb75c0672a0 0x7fb75c069760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:36.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.801+0000 7fb7737fe640 1 -- 172.21.15.164:0/4194145229 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 4857+0+0 (secure 0 0 0) 0x7fb774120360 con 0x7fb77c0b4e90 2024-03-23T14:57:36.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.801+0000 7fb77affd640 1 --2- 172.21.15.164:0/4194145229 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fb75c0672a0 0x7fb75c069760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:36.807 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.801+0000 7fb77affd640 1 --2- 172.21.15.164:0/4194145229 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fb75c0672a0 0x7fb75c069760 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb76c00a970 tx=0x7fb76c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:36.811 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:36.805+0000 7fb7737fe640 1 -- 172.21.15.164:0/4194145229 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7fb77c072480 con 0x7fb77c0b4e90 2024-03-23T14:57:37.024 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.021+0000 7fb781680640 1 -- 172.21.15.164:0/4194145229 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) -- 0x7fb77c0b14f0 con 0x7fb77c0b4e90 2024-03-23T14:57:37.027 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.021+0000 7fb7737fe640 1 -- 172.21.15.164:0/4194145229 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v9) ==== 90+0+0 (secure 0 0 0) 0x7fb77c0b14f0 con 0x7fb77c0b4e90 2024-03-23T14:57:37.029 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.025+0000 7fb781680640 1 -- 172.21.15.164:0/4194145229 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fb75c0672a0 msgr2=0x7fb75c069760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:37.029 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.025+0000 7fb781680640 1 --2- 172.21.15.164:0/4194145229 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fb75c0672a0 0x7fb75c069760 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb76c00a970 tx=0x7fb76c008040 comp rx=0 tx=0).stop 2024-03-23T14:57:37.029 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.025+0000 7fb781680640 1 -- 172.21.15.164:0/4194145229 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb77c0b4e90 msgr2=0x7fb77c0b17d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:37.029 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.025+0000 7fb781680640 1 --2- 172.21.15.164:0/4194145229 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb77c0b4e90 0x7fb77c0b17d0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fb7740b1040 tx=0x7fb77401b830 comp rx=0 tx=0).stop 2024-03-23T14:57:37.029 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.025+0000 7fb781680640 1 -- 172.21.15.164:0/4194145229 shutdown_connections 2024-03-23T14:57:37.029 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.025+0000 7fb781680640 1 --2- 172.21.15.164:0/4194145229 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb77c0b4e90 0x7fb77c0b17d0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:37.029 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.025+0000 7fb781680640 1 --2- 172.21.15.164:0/4194145229 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fb75c0672a0 0x7fb75c069760 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:37.030 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.025+0000 7fb781680640 1 --2- 172.21.15.164:0/4194145229 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb77c0b4a00 0x7fb77c0b0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:37.030 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.025+0000 7fb781680640 1 --2- 172.21.15.164:0/4194145229 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb77c09def0 0x7fb77c140980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:37.030 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.025+0000 7fb781680640 1 -- 172.21.15.164:0/4194145229 >> 172.21.15.164:0/4194145229 conn(0x7fb77c0b8ca0 msgr2=0x7fb77c1455f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:37.030 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.025+0000 7fb781680640 1 -- 172.21.15.164:0/4194145229 shutdown_connections 2024-03-23T14:57:37.030 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.025+0000 7fb781680640 1 -- 172.21.15.164:0/4194145229 wait complete. 2024-03-23T14:57:37.045 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-03-23T14:57:37.153 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.149+0000 7fd361c74640 1 Processor -- start 2024-03-23T14:57:37.154 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.149+0000 7fd361c74640 1 -- start start 2024-03-23T14:57:37.155 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.149+0000 7fd361c74640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd35c09def0 0x7fd35c0b45e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:37.155 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.149+0000 7fd361c74640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fd35c0b4b20 0x7fd35c06c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:37.155 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.149+0000 7fd361c74640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd35c06d650 con 0x7fd35c0add50 2024-03-23T14:57:37.155 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.149+0000 7fd361c74640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 -- 0x7fd35c0af4a0 con 0x7fd35c0b4b20 2024-03-23T14:57:37.156 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.149+0000 7fd361c74640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fd35c09e2d0 con 0x7fd35c09def0 2024-03-23T14:57:37.156 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.149+0000 7fd35b7fe640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd35c09def0 0x7fd35c0b45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:37.156 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.149+0000 7fd35b7fe640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd35c09def0 0x7fd35c0b45e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.164:56792/0 (socket says 172.21.15.164:56792) 2024-03-23T14:57:37.156 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.149+0000 7fd35b7fe640 1 -- 172.21.15.164:0/3112190688 learned_addr learned my addr 172.21.15.164:0/3112190688 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:37.156 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.149+0000 7fd35bfff640 1 --2- 172.21.15.164:0/3112190688 >> v2:172.21.15.173:3300/0 conn(0x7fd35c0b4b20 0x7fd35c06c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:37.156 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.149+0000 7fd35a7fc640 1 -- 172.21.15.164:0/3112190688 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3417101726 0 0) 0x7fd35c06d650 con 0x7fd35c0add50 2024-03-23T14:57:37.156 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.149+0000 7fd35a7fc640 1 -- 172.21.15.164:0/3112190688 --> v1:172.21.15.173:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd344003530 con 0x7fd35c0add50 2024-03-23T14:57:37.156 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.149+0000 7fd35b7fe640 1 -- 172.21.15.164:0/3112190688 >> v2:172.21.15.173:3300/0 conn(0x7fd35c0b4b20 msgr2=0x7fd35c06c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:37.156 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.149+0000 7fd35b7fe640 1 --2- 172.21.15.164:0/3112190688 >> v2:172.21.15.173:3300/0 conn(0x7fd35c0b4b20 0x7fd35c06c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:37.156 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.149+0000 7fd35b7fe640 1 -- 172.21.15.164:0/3112190688 >> v1:172.21.15.173:6789/0 conn(0x7fd35c0add50 legacy=0x7fd35c06bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:37.156 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.149+0000 7fd35b7fe640 1 -- 172.21.15.164:0/3112190688 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd35c0aeca0 con 0x7fd35c09def0 2024-03-23T14:57:37.157 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.153+0000 7fd35b7fe640 1 --2- 172.21.15.164:0/3112190688 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd35c09def0 0x7fd35c0b45e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd350009870 tx=0x7fd35001c440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29f4e6c172e066c5 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:37.158 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.153+0000 7fd35a7fc640 1 -- 172.21.15.164:0/3112190688 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3500ac600 con 0x7fd35c09def0 2024-03-23T14:57:37.158 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.153+0000 7fd35a7fc640 1 -- 172.21.15.164:0/3112190688 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3500a8050 con 0x7fd35c09def0 2024-03-23T14:57:37.158 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.153+0000 7fd35a7fc640 1 -- 172.21.15.164:0/3112190688 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3500ac980 con 0x7fd35c09def0 2024-03-23T14:57:37.159 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.153+0000 7fd361c74640 1 -- 172.21.15.164:0/3112190688 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd35c09def0 msgr2=0x7fd35c0b45e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:37.159 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.153+0000 7fd361c74640 1 --2- 172.21.15.164:0/3112190688 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd35c09def0 0x7fd35c0b45e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd350009870 tx=0x7fd35001c440 comp rx=0 tx=0).stop 2024-03-23T14:57:37.159 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.153+0000 7fd361c74640 1 -- 172.21.15.164:0/3112190688 shutdown_connections 2024-03-23T14:57:37.159 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.153+0000 7fd361c74640 1 --2- 172.21.15.164:0/3112190688 >> v2:172.21.15.173:3300/0 conn(0x7fd35c0b4b20 0x7fd35c06c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:37.159 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.153+0000 7fd361c74640 1 --2- 172.21.15.164:0/3112190688 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd35c09def0 0x7fd35c0b45e0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:37.159 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.153+0000 7fd361c74640 1 -- 172.21.15.164:0/3112190688 >> 172.21.15.164:0/3112190688 conn(0x7fd35c0b8ca0 msgr2=0x7fd35c0b90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:37.159 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.153+0000 7fd361c74640 1 -- 172.21.15.164:0/3112190688 shutdown_connections 2024-03-23T14:57:37.159 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.153+0000 7fd361c74640 1 -- 172.21.15.164:0/3112190688 wait complete. 2024-03-23T14:57:37.160 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.157+0000 7fd361c74640 1 Processor -- start 2024-03-23T14:57:37.160 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.157+0000 7fd361c74640 1 -- start start 2024-03-23T14:57:37.162 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.157+0000 7fd361c74640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd35c09def0 0x7fd35c0b1e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:37.162 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.157+0000 7fd361c74640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd35c0add50 0x7fd35c138a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:37.162 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.157+0000 7fd361c74640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd35c0b4b20 0x7fd35c138fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:37.163 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.157+0000 7fd361c74640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd35c072160 con 0x7fd35c0add50 2024-03-23T14:57:37.163 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.157+0000 7fd361c74640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd35c06f690 con 0x7fd35c09def0 2024-03-23T14:57:37.163 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.157+0000 7fd361c74640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fd35c09b750 con 0x7fd35c0b4b20 2024-03-23T14:57:37.163 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.157+0000 7fd35affd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd35c0add50 0x7fd35c138a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:37.163 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.157+0000 7fd35affd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd35c0add50 0x7fd35c138a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:33226/0 (socket says 172.21.15.164:33226) 2024-03-23T14:57:37.163 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.157+0000 7fd35affd640 1 -- 172.21.15.164:0/4040487101 learned_addr learned my addr 172.21.15.164:0/4040487101 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:37.163 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.157+0000 7fd35b7fe640 1 --2- 172.21.15.164:0/4040487101 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd35c09def0 0x7fd35c0b1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:37.163 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.157+0000 7fd35bfff640 1 --2- 172.21.15.164:0/4040487101 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd35c0b4b20 0x7fd35c138fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:37.163 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.157+0000 7fd35affd640 1 -- 172.21.15.164:0/4040487101 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd35c0b4b20 msgr2=0x7fd35c138fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:37.163 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.157+0000 7fd35affd640 1 --2- 172.21.15.164:0/4040487101 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd35c0b4b20 0x7fd35c138fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:37.163 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.157+0000 7fd35affd640 1 -- 172.21.15.164:0/4040487101 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd35c09def0 msgr2=0x7fd35c0b1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:37.163 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.157+0000 7fd35affd640 1 --2- 172.21.15.164:0/4040487101 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd35c09def0 0x7fd35c0b1e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:37.164 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.157+0000 7fd35affd640 1 -- 172.21.15.164:0/4040487101 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3500094d0 con 0x7fd35c0add50 2024-03-23T14:57:37.164 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.157+0000 7fd35b7fe640 1 --2- 172.21.15.164:0/4040487101 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd35c09def0 0x7fd35c0b1e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T14:57:37.164 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.157+0000 7fd35affd640 1 --2- 172.21.15.164:0/4040487101 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd35c0add50 0x7fd35c138a60 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fd34c00c530 tx=0x7fd34c00c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:37.164 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.157+0000 7fd343fff640 1 -- 172.21.15.164:0/4040487101 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd34c0052d0 con 0x7fd35c0add50 2024-03-23T14:57:37.164 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.157+0000 7fd343fff640 1 -- 172.21.15.164:0/4040487101 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd34c005450 con 0x7fd35c0add50 2024-03-23T14:57:37.164 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.157+0000 7fd361c74640 1 -- 172.21.15.164:0/4040487101 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd35c09ba30 con 0x7fd35c0add50 2024-03-23T14:57:37.164 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.157+0000 7fd343fff640 1 -- 172.21.15.164:0/4040487101 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd34c012870 con 0x7fd35c0add50 2024-03-23T14:57:37.164 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.157+0000 7fd361c74640 1 -- 172.21.15.164:0/4040487101 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd35c06fbf0 con 0x7fd35c0add50 2024-03-23T14:57:37.164 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.157+0000 7fd361c74640 1 -- 172.21.15.164:0/4040487101 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd35c0690a0 con 0x7fd35c0add50 2024-03-23T14:57:37.165 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.157+0000 7fd343fff640 1 -- 172.21.15.164:0/4040487101 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7fd34c0129d0 con 0x7fd35c0add50 2024-03-23T14:57:37.165 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.157+0000 7fd343fff640 1 --2- 172.21.15.164:0/4040487101 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fd3480672a0 0x7fd348069760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:37.165 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.157+0000 7fd35b7fe640 1 --2- 172.21.15.164:0/4040487101 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fd3480672a0 0x7fd348069760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:37.165 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.157+0000 7fd343fff640 1 -- 172.21.15.164:0/4040487101 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 4857+0+0 (secure 0 0 0) 0x7fd34c084820 con 0x7fd35c0add50 2024-03-23T14:57:37.165 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.161+0000 7fd35b7fe640 1 --2- 172.21.15.164:0/4040487101 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fd3480672a0 0x7fd348069760 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fd3500ae000 tx=0x7fd350008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:37.167 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.161+0000 7fd343fff640 1 -- 172.21.15.164:0/4040487101 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7fd34c051730 con 0x7fd35c0add50 2024-03-23T14:57:37.367 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.361+0000 7fd361c74640 1 -- 172.21.15.164:0/4040487101 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) -- 0x7fd35c0722e0 con 0x7fd35c0add50 2024-03-23T14:57:37.369 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.365+0000 7fd343fff640 1 -- 172.21.15.164:0/4040487101 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v10) ==== 90+0+0 (secure 0 0 0) 0x7fd34c054df0 con 0x7fd35c0add50 2024-03-23T14:57:37.372 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.369+0000 7fd361c74640 1 -- 172.21.15.164:0/4040487101 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fd3480672a0 msgr2=0x7fd348069760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:37.372 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.369+0000 7fd361c74640 1 --2- 172.21.15.164:0/4040487101 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fd3480672a0 0x7fd348069760 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fd3500ae000 tx=0x7fd350008040 comp rx=0 tx=0).stop 2024-03-23T14:57:37.373 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.369+0000 7fd361c74640 1 -- 172.21.15.164:0/4040487101 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd35c0add50 msgr2=0x7fd35c138a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:37.373 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.369+0000 7fd361c74640 1 --2- 172.21.15.164:0/4040487101 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd35c0add50 0x7fd35c138a60 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fd34c00c530 tx=0x7fd34c00c9f0 comp rx=0 tx=0).stop 2024-03-23T14:57:37.373 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.369+0000 7fd361c74640 1 -- 172.21.15.164:0/4040487101 shutdown_connections 2024-03-23T14:57:37.373 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.369+0000 7fd361c74640 1 --2- 172.21.15.164:0/4040487101 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd35c0b4b20 0x7fd35c138fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:37.373 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.369+0000 7fd361c74640 1 --2- 172.21.15.164:0/4040487101 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd35c0add50 0x7fd35c138a60 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:37.373 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.369+0000 7fd361c74640 1 --2- 172.21.15.164:0/4040487101 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fd3480672a0 0x7fd348069760 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:37.373 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.369+0000 7fd361c74640 1 --2- 172.21.15.164:0/4040487101 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd35c09def0 0x7fd35c0b1e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:37.373 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.369+0000 7fd361c74640 1 -- 172.21.15.164:0/4040487101 >> 172.21.15.164:0/4040487101 conn(0x7fd35c0b8ca0 msgr2=0x7fd35c142040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:37.373 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.369+0000 7fd361c74640 1 -- 172.21.15.164:0/4040487101 shutdown_connections 2024-03-23T14:57:37.373 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.369+0000 7fd361c74640 1 -- 172.21.15.164:0/4040487101 wait complete. 2024-03-23T14:57:37.387 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-03-23T14:57:37.504 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.501+0000 7febcd721640 1 Processor -- start 2024-03-23T14:57:37.504 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.501+0000 7febcd721640 1 -- start start 2024-03-23T14:57:37.505 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.501+0000 7febcd721640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7febc814eb80 0x7febc8150f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:37.505 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.501+0000 7febcd721640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7febc80b51d0 con 0x7febc80b0230 2024-03-23T14:57:37.505 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.501+0000 7febcd721640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7febc80b0610 con 0x7febc8148cf0 2024-03-23T14:57:37.505 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.501+0000 7febcd721640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7febc80b1500 con 0x7febc814eb80 2024-03-23T14:57:37.505 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.501+0000 7febc6ffd640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7febc80b0230 0x7febc80b34d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:58782/0 (socket says 172.21.15.164:58782) 2024-03-23T14:57:37.506 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.501+0000 7febc77fe640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7febc814eb80 0x7febc8150f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:37.506 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.501+0000 7febc6ffd640 1 -- 172.21.15.164:0/258042642 learned_addr learned my addr 172.21.15.164:0/258042642 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-03-23T14:57:37.506 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.501+0000 7febc77fe640 1 -- 172.21.15.164:0/258042642 >> v1:172.21.15.173:6789/0 conn(0x7febc8148cf0 legacy=0x7febc8149180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:37.506 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.501+0000 7febc77fe640 1 -- 172.21.15.164:0/258042642 >> v1:172.21.15.164:6789/0 conn(0x7febc80b0230 legacy=0x7febc80b34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:37.506 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.501+0000 7febc77fe640 1 -- 172.21.15.164:0/258042642 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febc806d250 con 0x7febc814eb80 2024-03-23T14:57:37.508 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.505+0000 7febc77fe640 1 --2- 172.21.15.164:0/258042642 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7febc814eb80 0x7febc8150f70 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7febc80b0800 tx=0x7febc001c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8371c45d3687653 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:37.508 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.505+0000 7febc5ffb640 1 -- 172.21.15.164:0/258042642 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc00a5880 con 0x7febc814eb80 2024-03-23T14:57:37.508 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.505+0000 7febc5ffb640 1 -- 172.21.15.164:0/258042642 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febc00a59e0 con 0x7febc814eb80 2024-03-23T14:57:37.508 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.505+0000 7febc5ffb640 1 -- 172.21.15.164:0/258042642 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc00a5d70 con 0x7febc814eb80 2024-03-23T14:57:37.509 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.505+0000 7febcd721640 1 -- 172.21.15.164:0/258042642 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7febc814eb80 msgr2=0x7febc8150f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:37.509 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.505+0000 7febcd721640 1 --2- 172.21.15.164:0/258042642 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7febc814eb80 0x7febc8150f70 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7febc80b0800 tx=0x7febc001c7c0 comp rx=0 tx=0).stop 2024-03-23T14:57:37.509 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.505+0000 7febcd721640 1 -- 172.21.15.164:0/258042642 shutdown_connections 2024-03-23T14:57:37.509 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.505+0000 7febcd721640 1 --2- 172.21.15.164:0/258042642 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7febc814eb80 0x7febc8150f70 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:37.509 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.505+0000 7febcd721640 1 -- 172.21.15.164:0/258042642 >> 172.21.15.164:0/258042642 conn(0x7febc80b8ca0 msgr2=0x7febc80b90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:37.509 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.505+0000 7febcd721640 1 -- 172.21.15.164:0/258042642 shutdown_connections 2024-03-23T14:57:37.509 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.505+0000 7febcd721640 1 -- 172.21.15.164:0/258042642 wait complete. 2024-03-23T14:57:37.511 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.505+0000 7febcd721640 1 Processor -- start 2024-03-23T14:57:37.511 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.505+0000 7febcd721640 1 -- start start 2024-03-23T14:57:37.511 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.505+0000 7febcd721640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7febc80b0230 0x7febc81424e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:37.511 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.505+0000 7febcd721640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7febc8148cf0 0x7febc8140fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:37.511 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.505+0000 7febcd721640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7febc814eb80 0x7febc809cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:37.511 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.505+0000 7febcd721640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7febc8142b80 con 0x7febc8148cf0 2024-03-23T14:57:37.511 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.505+0000 7febcd721640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7febc8140740 con 0x7febc814eb80 2024-03-23T14:57:37.511 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.505+0000 7febcd721640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7febc8141a80 con 0x7febc80b0230 2024-03-23T14:57:37.512 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.505+0000 7febc67fc640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7febc8148cf0 0x7febc8140fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:37.512 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.505+0000 7febc67fc640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7febc8148cf0 0x7febc8140fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:33240/0 (socket says 172.21.15.164:33240) 2024-03-23T14:57:37.512 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.505+0000 7febc67fc640 1 -- 172.21.15.164:0/2297792705 learned_addr learned my addr 172.21.15.164:0/2297792705 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:37.512 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.505+0000 7febc67fc640 1 -- 172.21.15.164:0/2297792705 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7febc80b0230 msgr2=0x7febc81424e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:37.512 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.505+0000 7febc67fc640 1 --2- 172.21.15.164:0/2297792705 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7febc80b0230 0x7febc81424e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:37.512 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.505+0000 7febc67fc640 1 -- 172.21.15.164:0/2297792705 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7febc814eb80 msgr2=0x7febc809cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:37.512 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.505+0000 7febc67fc640 1 --2- 172.21.15.164:0/2297792705 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7febc814eb80 0x7febc809cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:37.512 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.505+0000 7febc67fc640 1 -- 172.21.15.164:0/2297792705 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febc0009a70 con 0x7febc8148cf0 2024-03-23T14:57:37.512 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.505+0000 7febc67fc640 1 --2- 172.21.15.164:0/2297792705 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7febc8148cf0 0x7febc8140fe0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7febbc00c450 tx=0x7febbc00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:37.512 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.505+0000 7febaf7fe640 1 -- 172.21.15.164:0/2297792705 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febbc012830 con 0x7febc8148cf0 2024-03-23T14:57:37.512 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.505+0000 7febaf7fe640 1 -- 172.21.15.164:0/2297792705 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febbc0129b0 con 0x7febc8148cf0 2024-03-23T14:57:37.512 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.505+0000 7febaf7fe640 1 -- 172.21.15.164:0/2297792705 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febbc012d60 con 0x7febc8148cf0 2024-03-23T14:57:37.512 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.505+0000 7febcd721640 1 -- 172.21.15.164:0/2297792705 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febc80b0f70 con 0x7febc8148cf0 2024-03-23T14:57:37.512 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.505+0000 7febcd721640 1 -- 172.21.15.164:0/2297792705 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febc8140a00 con 0x7febc8148cf0 2024-03-23T14:57:37.513 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.509+0000 7febcd721640 1 -- 172.21.15.164:0/2297792705 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febc8140bf0 con 0x7febc8148cf0 2024-03-23T14:57:37.513 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.509+0000 7febaf7fe640 1 -- 172.21.15.164:0/2297792705 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7febbc01c460 con 0x7febc8148cf0 2024-03-23T14:57:37.513 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.509+0000 7febaf7fe640 1 --2- 172.21.15.164:0/2297792705 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7febb40672d0 0x7febb4069790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:37.513 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.509+0000 7febc6ffd640 1 --2- 172.21.15.164:0/2297792705 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7febb40672d0 0x7febb4069790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:37.513 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.509+0000 7febaf7fe640 1 -- 172.21.15.164:0/2297792705 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 4857+0+0 (secure 0 0 0) 0x7febbc084a80 con 0x7febc8148cf0 2024-03-23T14:57:37.514 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.509+0000 7febc6ffd640 1 --2- 172.21.15.164:0/2297792705 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7febb40672d0 0x7febb4069790 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7febb8000f70 tx=0x7febb8007a90 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:37.517 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.513+0000 7febaf7fe640 1 -- 172.21.15.164:0/2297792705 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7febbc051a10 con 0x7febc8148cf0 2024-03-23T14:57:37.729 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.725+0000 7febcd721640 1 -- 172.21.15.164:0/2297792705 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) -- 0x7febc80690a0 con 0x7febc8148cf0 2024-03-23T14:57:37.733 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.729+0000 7febaf7fe640 1 -- 172.21.15.164:0/2297792705 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v11) ==== 97+0+0 (secure 0 0 0) 0x7febbc0550d0 con 0x7febc8148cf0 2024-03-23T14:57:37.736 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.733+0000 7febcd721640 1 -- 172.21.15.164:0/2297792705 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7febb40672d0 msgr2=0x7febb4069790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:37.736 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.733+0000 7febcd721640 1 --2- 172.21.15.164:0/2297792705 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7febb40672d0 0x7febb4069790 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7febb8000f70 tx=0x7febb8007a90 comp rx=0 tx=0).stop 2024-03-23T14:57:37.736 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.733+0000 7febcd721640 1 -- 172.21.15.164:0/2297792705 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7febc8148cf0 msgr2=0x7febc8140fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:37.736 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.733+0000 7febcd721640 1 --2- 172.21.15.164:0/2297792705 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7febc8148cf0 0x7febc8140fe0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7febbc00c450 tx=0x7febbc00c910 comp rx=0 tx=0).stop 2024-03-23T14:57:37.736 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.733+0000 7febcd721640 1 -- 172.21.15.164:0/2297792705 shutdown_connections 2024-03-23T14:57:37.737 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.733+0000 7febcd721640 1 --2- 172.21.15.164:0/2297792705 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7febb40672d0 0x7febb4069790 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:37.737 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.733+0000 7febcd721640 1 --2- 172.21.15.164:0/2297792705 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7febc814eb80 0x7febc809cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:37.737 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.733+0000 7febcd721640 1 --2- 172.21.15.164:0/2297792705 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7febc8148cf0 0x7febc8140fe0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:37.737 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.733+0000 7febcd721640 1 --2- 172.21.15.164:0/2297792705 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7febc80b0230 0x7febc81424e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:37.737 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.733+0000 7febcd721640 1 -- 172.21.15.164:0/2297792705 >> 172.21.15.164:0/2297792705 conn(0x7febc80b8ca0 msgr2=0x7febc80b43f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:37.737 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.733+0000 7febcd721640 1 -- 172.21.15.164:0/2297792705 shutdown_connections 2024-03-23T14:57:37.737 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.733+0000 7febcd721640 1 -- 172.21.15.164:0/2297792705 wait complete. 2024-03-23T14:57:37.751 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-03-23T14:57:37.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.857+0000 7faf2e554640 1 Processor -- start 2024-03-23T14:57:37.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.857+0000 7faf2e554640 1 -- start start 2024-03-23T14:57:37.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.857+0000 7faf2e554640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7faf28148cf0 0x7faf28149180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:37.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.857+0000 7faf2e554640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7faf2814db70 0x7faf2814ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:37.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.857+0000 7faf2e554640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf280b51d0 con 0x7faf280b0230 2024-03-23T14:57:37.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.857+0000 7faf2e554640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 -- 0x7faf280b0610 con 0x7faf2814db70 2024-03-23T14:57:37.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.857+0000 7faf2e554640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7faf280b1500 con 0x7faf28148cf0 2024-03-23T14:57:37.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.857+0000 7faf27fff640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7faf280b0230 0x7faf280b34d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:58796/0 (socket says 172.21.15.164:58796) 2024-03-23T14:57:37.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.857+0000 7faf27fff640 1 -- 172.21.15.164:0/1092053624 learned_addr learned my addr 172.21.15.164:0/1092053624 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-03-23T14:57:37.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.857+0000 7faf277fe640 1 --2- 172.21.15.164:0/1092053624 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7faf28148cf0 0x7faf28149180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:37.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.857+0000 7faf26ffd640 1 -- 172.21.15.164:0/1092053624 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1702170758 0 0) 0x7faf280b51d0 con 0x7faf280b0230 2024-03-23T14:57:37.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.857+0000 7faf2caf1640 1 --2- 172.21.15.164:0/1092053624 >> v2:172.21.15.173:3300/0 conn(0x7faf2814db70 0x7faf2814ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:37.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.857+0000 7faf26ffd640 1 -- 172.21.15.164:0/1092053624 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faf10003530 con 0x7faf280b0230 2024-03-23T14:57:37.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.857+0000 7faf26ffd640 1 -- 172.21.15.164:0/1092053624 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3023109935 0 0) 0x7faf10003530 con 0x7faf280b0230 2024-03-23T14:57:37.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.857+0000 7faf26ffd640 1 -- 172.21.15.164:0/1092053624 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7faf28148cf0 msgr2=0x7faf28149180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:37.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.857+0000 7faf26ffd640 1 --2- 172.21.15.164:0/1092053624 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7faf28148cf0 0x7faf28149180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:37.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.857+0000 7faf26ffd640 1 -- 172.21.15.164:0/1092053624 >> v2:172.21.15.173:3300/0 conn(0x7faf2814db70 msgr2=0x7faf2814ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:37.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.857+0000 7faf26ffd640 1 --2- 172.21.15.164:0/1092053624 >> v2:172.21.15.173:3300/0 conn(0x7faf2814db70 0x7faf2814ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:37.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.857+0000 7faf26ffd640 1 -- 172.21.15.164:0/1092053624 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf2806d250 con 0x7faf280b0230 2024-03-23T14:57:37.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.857+0000 7faf26ffd640 1 -- 172.21.15.164:0/1092053624 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2526390548 0 0) 0x7faf18001680 con 0x7faf280b0230 2024-03-23T14:57:37.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.857+0000 7faf277fe640 1 --2- 172.21.15.164:0/1092053624 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7faf28148cf0 0x7faf28149180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T14:57:37.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.857+0000 7faf26ffd640 1 -- 172.21.15.164:0/1092053624 >> v1:172.21.15.164:6789/0 conn(0x7faf280b0230 legacy=0x7faf280b34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:37.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.857+0000 7faf26ffd640 1 --2- 172.21.15.164:0/1092053624 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7faf28148cf0 0x7faf10003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:37.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.857+0000 7faf26ffd640 1 --2- 172.21.15.164:0/1092053624 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faf10003f70 0x7faf100047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:37.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.857+0000 7faf26ffd640 1 --2- 172.21.15.164:0/1092053624 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf10004d10 0x7faf10005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:37.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.857+0000 7faf26ffd640 1 -- 172.21.15.164:0/1092053624 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7faf280b0610 con 0x7faf10004d10 2024-03-23T14:57:37.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.857+0000 7faf26ffd640 1 -- 172.21.15.164:0/1092053624 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7faf280b1500 con 0x7faf10003f70 2024-03-23T14:57:37.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.857+0000 7faf26ffd640 1 -- 172.21.15.164:0/1092053624 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7faf1000baf0 con 0x7faf28148cf0 2024-03-23T14:57:37.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.857+0000 7faf2caf1640 1 --2- 172.21.15.164:0/1092053624 >> v2:172.21.15.173:3300/0 conn(0x7faf2814db70 0x7faf2814ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T14:57:37.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.857+0000 7faf2caf1640 1 --2- 172.21.15.164:0/1092053624 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf10004d10 0x7faf10005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:37.866 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.857+0000 7faf277fe640 1 --2- 172.21.15.164:0/1092053624 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faf10003f70 0x7faf100047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:37.866 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.857+0000 7faf27fff640 1 --2- 172.21.15.164:0/1092053624 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7faf28148cf0 0x7faf10003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:37.866 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.857+0000 7faf2caf1640 1 -- 172.21.15.164:0/1092053624 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7faf28148cf0 msgr2=0x7faf10003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:37.866 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.857+0000 7faf2caf1640 1 --2- 172.21.15.164:0/1092053624 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7faf28148cf0 0x7faf10003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:37.866 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.857+0000 7faf2caf1640 1 -- 172.21.15.164:0/1092053624 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faf10003f70 msgr2=0x7faf100047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:37.866 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.857+0000 7faf2caf1640 1 --2- 172.21.15.164:0/1092053624 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faf10003f70 0x7faf100047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:37.866 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.857+0000 7faf2caf1640 1 -- 172.21.15.164:0/1092053624 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf10003530 con 0x7faf10004d10 2024-03-23T14:57:37.870 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.861+0000 7faf2caf1640 1 --2- 172.21.15.164:0/1092053624 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf10004d10 0x7faf10005190 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7faf200073b0 tx=0x7faf2001cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7012b65d78bc4abb server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:37.870 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.861+0000 7faf26ffd640 1 -- 172.21.15.164:0/1092053624 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf200a5d10 con 0x7faf10004d10 2024-03-23T14:57:37.870 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.861+0000 7faf26ffd640 1 -- 172.21.15.164:0/1092053624 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf200a5e70 con 0x7faf10004d10 2024-03-23T14:57:37.870 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.861+0000 7faf26ffd640 1 -- 172.21.15.164:0/1092053624 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf200ad640 con 0x7faf10004d10 2024-03-23T14:57:37.870 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.861+0000 7faf2e554640 1 -- 172.21.15.164:0/1092053624 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf10004d10 msgr2=0x7faf10005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:37.871 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.861+0000 7faf2e554640 1 --2- 172.21.15.164:0/1092053624 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf10004d10 0x7faf10005190 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7faf200073b0 tx=0x7faf2001cf20 comp rx=0 tx=0).stop 2024-03-23T14:57:37.871 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.861+0000 7faf2caf1640 1 -- 172.21.15.164:0/1092053624 reap_dead start 2024-03-23T14:57:37.871 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.861+0000 7faf2e554640 1 -- 172.21.15.164:0/1092053624 shutdown_connections 2024-03-23T14:57:37.871 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.861+0000 7faf2e554640 1 -- 172.21.15.164:0/1092053624 >> 172.21.15.164:0/1092053624 conn(0x7faf280b8ca0 msgr2=0x7faf280b90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:37.871 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.861+0000 7faf2e554640 1 -- 172.21.15.164:0/1092053624 shutdown_connections 2024-03-23T14:57:37.871 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.861+0000 7faf2e554640 1 -- 172.21.15.164:0/1092053624 wait complete. 2024-03-23T14:57:37.871 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.861+0000 7faf2e554640 1 Processor -- start 2024-03-23T14:57:37.871 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.861+0000 7faf2e554640 1 -- start start 2024-03-23T14:57:37.871 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.865+0000 7faf2e554640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faf2809c550 0x7faf2813bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:37.871 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.865+0000 7faf2e554640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7faf2809b9a0 0x7faf2809ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:37.871 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.865+0000 7faf2e554640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf2809d3f0 0x7faf281de1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:37.871 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.865+0000 7faf2e554640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7faf2809b510 con 0x7faf2809d3f0 2024-03-23T14:57:37.871 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.865+0000 7faf2e554640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7faf2813c0f0 con 0x7faf2809c550 2024-03-23T14:57:37.871 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.865+0000 7faf2e554640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7faf2809af30 con 0x7faf2809b9a0 2024-03-23T14:57:37.872 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.865+0000 7faf2caf1640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf2809d3f0 0x7faf281de1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:37.872 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.865+0000 7faf2caf1640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf2809d3f0 0x7faf281de1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:33252/0 (socket says 172.21.15.164:33252) 2024-03-23T14:57:37.872 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.865+0000 7faf2caf1640 1 -- 172.21.15.164:0/273882375 learned_addr learned my addr 172.21.15.164:0/273882375 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:37.872 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.865+0000 7faf27fff640 1 --2- 172.21.15.164:0/273882375 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faf2809c550 0x7faf2813bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:37.872 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.865+0000 7faf277fe640 1 --2- 172.21.15.164:0/273882375 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7faf2809b9a0 0x7faf2809ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:37.872 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.865+0000 7faf2caf1640 1 -- 172.21.15.164:0/273882375 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7faf2809b9a0 msgr2=0x7faf2809ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:37.872 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.865+0000 7faf2caf1640 1 --2- 172.21.15.164:0/273882375 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7faf2809b9a0 0x7faf2809ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:37.872 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.865+0000 7faf2caf1640 1 -- 172.21.15.164:0/273882375 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faf2809c550 msgr2=0x7faf2813bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:37.872 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.865+0000 7faf2caf1640 1 --2- 172.21.15.164:0/273882375 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faf2809c550 0x7faf2813bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:37.872 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.865+0000 7faf2caf1640 1 -- 172.21.15.164:0/273882375 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf2809c930 con 0x7faf2809d3f0 2024-03-23T14:57:37.872 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.865+0000 7faf2caf1640 1 --2- 172.21.15.164:0/273882375 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf2809d3f0 0x7faf281de1e0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7faf20002240 tx=0x7faf20019cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:37.872 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.865+0000 7faf24ff9640 1 -- 172.21.15.164:0/273882375 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf200bd070 con 0x7faf2809d3f0 2024-03-23T14:57:37.872 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.865+0000 7faf24ff9640 1 -- 172.21.15.164:0/273882375 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf200aebc0 con 0x7faf2809d3f0 2024-03-23T14:57:37.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.865+0000 7faf24ff9640 1 -- 172.21.15.164:0/273882375 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf20009690 con 0x7faf2809d3f0 2024-03-23T14:57:37.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.865+0000 7faf2e554640 1 -- 172.21.15.164:0/273882375 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf2809d9a0 con 0x7faf2809d3f0 2024-03-23T14:57:37.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.865+0000 7faf2e554640 1 -- 172.21.15.164:0/273882375 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf2809be70 con 0x7faf2809d3f0 2024-03-23T14:57:37.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.865+0000 7faf2e554640 1 -- 172.21.15.164:0/273882375 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf280690a0 con 0x7faf2809d3f0 2024-03-23T14:57:37.873 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.865+0000 7faf24ff9640 1 -- 172.21.15.164:0/273882375 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7faf200b4070 con 0x7faf2809d3f0 2024-03-23T14:57:37.874 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.865+0000 7faf24ff9640 1 --2- 172.21.15.164:0/273882375 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7faf140672a0 0x7faf14069760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:37.874 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.865+0000 7faf24ff9640 1 -- 172.21.15.164:0/273882375 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 4857+0+0 (secure 0 0 0) 0x7faf2011f0b0 con 0x7faf2809d3f0 2024-03-23T14:57:37.874 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.865+0000 7faf27fff640 1 --2- 172.21.15.164:0/273882375 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7faf140672a0 0x7faf14069760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:37.874 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.865+0000 7faf27fff640 1 --2- 172.21.15.164:0/273882375 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7faf140672a0 0x7faf14069760 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7faf18001060 tx=0x7faf18009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:37.875 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:37.869+0000 7faf24ff9640 1 -- 172.21.15.164:0/273882375 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7faf200ec040 con 0x7faf2809d3f0 2024-03-23T14:57:38.073 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.069+0000 7faf2e554640 1 -- 172.21.15.164:0/273882375 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) -- 0x7faf2809b650 con 0x7faf2809d3f0 2024-03-23T14:57:38.078 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.073+0000 7faf24ff9640 1 -- 172.21.15.164:0/273882375 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v12) ==== 90+0+0 (secure 0 0 0) 0x7faf200ef700 con 0x7faf2809d3f0 2024-03-23T14:57:38.081 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.077+0000 7faf2e554640 1 -- 172.21.15.164:0/273882375 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7faf140672a0 msgr2=0x7faf14069760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:38.081 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.077+0000 7faf2e554640 1 --2- 172.21.15.164:0/273882375 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7faf140672a0 0x7faf14069760 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7faf18001060 tx=0x7faf18009040 comp rx=0 tx=0).stop 2024-03-23T14:57:38.082 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.077+0000 7faf2e554640 1 -- 172.21.15.164:0/273882375 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf2809d3f0 msgr2=0x7faf281de1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:38.082 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.077+0000 7faf2e554640 1 --2- 172.21.15.164:0/273882375 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf2809d3f0 0x7faf281de1e0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7faf20002240 tx=0x7faf20019cc0 comp rx=0 tx=0).stop 2024-03-23T14:57:38.082 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.077+0000 7faf2e554640 1 -- 172.21.15.164:0/273882375 shutdown_connections 2024-03-23T14:57:38.082 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.077+0000 7faf2e554640 1 --2- 172.21.15.164:0/273882375 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf2809d3f0 0x7faf281de1e0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:38.082 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.077+0000 7faf2e554640 1 --2- 172.21.15.164:0/273882375 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7faf2809b9a0 0x7faf2809ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:38.082 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.077+0000 7faf2e554640 1 --2- 172.21.15.164:0/273882375 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7faf140672a0 0x7faf14069760 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:38.082 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.077+0000 7faf2e554640 1 --2- 172.21.15.164:0/273882375 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faf2809c550 0x7faf2813bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:38.082 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.077+0000 7faf2e554640 1 -- 172.21.15.164:0/273882375 >> 172.21.15.164:0/273882375 conn(0x7faf280b8ca0 msgr2=0x7faf28145d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:38.082 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.077+0000 7faf2e554640 1 -- 172.21.15.164:0/273882375 shutdown_connections 2024-03-23T14:57:38.083 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.077+0000 7faf2e554640 1 -- 172.21.15.164:0/273882375 wait complete. 2024-03-23T14:57:38.097 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T14:57:38.211 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.205+0000 7fdea0f5e640 1 Processor -- start 2024-03-23T14:57:38.211 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.205+0000 7fdea0f5e640 1 -- start start 2024-03-23T14:57:38.212 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.205+0000 7fdea0f5e640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fde9c14eb80 0x7fde9c150f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:38.212 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.205+0000 7fdea0f5e640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde9c0b51d0 con 0x7fde9c0b0230 2024-03-23T14:57:38.212 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.205+0000 7fdea0f5e640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde9c0b0610 con 0x7fde9c148cf0 2024-03-23T14:57:38.212 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.205+0000 7fdea0f5e640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fde9c0b1500 con 0x7fde9c14eb80 2024-03-23T14:57:38.212 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.209+0000 7fde9ad9d640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fde9c14eb80 0x7fde9c150f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:38.212 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.209+0000 7fde9a59c640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fde9c0b0230 0x7fde9c0b34d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:58804/0 (socket says 172.21.15.164:58804) 2024-03-23T14:57:38.212 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.209+0000 7fde9a59c640 1 -- 172.21.15.164:0/2199678286 learned_addr learned my addr 172.21.15.164:0/2199678286 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-03-23T14:57:38.212 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.209+0000 7fde9ad9d640 1 -- 172.21.15.164:0/2199678286 >> v1:172.21.15.173:6789/0 conn(0x7fde9c148cf0 legacy=0x7fde9c149180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:38.212 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.209+0000 7fde9ad9d640 1 -- 172.21.15.164:0/2199678286 >> v1:172.21.15.164:6789/0 conn(0x7fde9c0b0230 legacy=0x7fde9c0b34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:38.212 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.209+0000 7fde9ad9d640 1 -- 172.21.15.164:0/2199678286 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde9c06d250 con 0x7fde9c14eb80 2024-03-23T14:57:38.212 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.209+0000 7fde9959a640 1 -- 172.21.15.164:0/2199678286 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1404938214 0 0) 0x7fde9c0b51d0 con 0x7fde9c0b0230 2024-03-23T14:57:38.214 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.209+0000 7fde9ad9d640 1 --2- 172.21.15.164:0/2199678286 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fde9c14eb80 0x7fde9c150f70 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fde9c0b0800 tx=0x7fde8c01c310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b267a57c1ba70e08 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:38.214 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.209+0000 7fde9959a640 1 -- 172.21.15.164:0/2199678286 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde8c0ac640 con 0x7fde9c14eb80 2024-03-23T14:57:38.214 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.209+0000 7fde9959a640 1 -- 172.21.15.164:0/2199678286 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde8c0ac7a0 con 0x7fde9c14eb80 2024-03-23T14:57:38.214 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.209+0000 7fde9959a640 1 -- 172.21.15.164:0/2199678286 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde8c0acb30 con 0x7fde9c14eb80 2024-03-23T14:57:38.215 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.209+0000 7fdea0f5e640 1 -- 172.21.15.164:0/2199678286 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fde9c14eb80 msgr2=0x7fde9c150f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:38.215 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.209+0000 7fdea0f5e640 1 --2- 172.21.15.164:0/2199678286 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fde9c14eb80 0x7fde9c150f70 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fde9c0b0800 tx=0x7fde8c01c310 comp rx=0 tx=0).stop 2024-03-23T14:57:38.215 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.209+0000 7fdea0f5e640 1 -- 172.21.15.164:0/2199678286 shutdown_connections 2024-03-23T14:57:38.215 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.209+0000 7fdea0f5e640 1 --2- 172.21.15.164:0/2199678286 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fde9c14eb80 0x7fde9c150f70 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:38.215 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.209+0000 7fdea0f5e640 1 -- 172.21.15.164:0/2199678286 >> 172.21.15.164:0/2199678286 conn(0x7fde9c0b8ca0 msgr2=0x7fde9c0b90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:38.215 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.209+0000 7fdea0f5e640 1 -- 172.21.15.164:0/2199678286 shutdown_connections 2024-03-23T14:57:38.215 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.209+0000 7fdea0f5e640 1 -- 172.21.15.164:0/2199678286 wait complete. 2024-03-23T14:57:38.216 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.213+0000 7fdea0f5e640 1 Processor -- start 2024-03-23T14:57:38.216 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.213+0000 7fdea0f5e640 1 -- start start 2024-03-23T14:57:38.218 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.213+0000 7fdea0f5e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fde9c0b0230 0x7fde9c09b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:38.218 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.213+0000 7fdea0f5e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fde9c148cf0 0x7fde9c06f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:38.218 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.213+0000 7fdea0f5e640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fde9c14eb80 0x7fde9c144810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:38.218 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.213+0000 7fdea0f5e640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fde9c0722d0 con 0x7fde9c0b0230 2024-03-23T14:57:38.218 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.213+0000 7fdea0f5e640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fde9c09d580 con 0x7fde9c148cf0 2024-03-23T14:57:38.218 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.213+0000 7fdea0f5e640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fde9c09ac20 con 0x7fde9c14eb80 2024-03-23T14:57:38.218 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.213+0000 7fde9a59c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fde9c0b0230 0x7fde9c09b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:38.218 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.213+0000 7fde9a59c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fde9c0b0230 0x7fde9c09b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:33266/0 (socket says 172.21.15.164:33266) 2024-03-23T14:57:38.218 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.213+0000 7fde9a59c640 1 -- 172.21.15.164:0/1614118471 learned_addr learned my addr 172.21.15.164:0/1614118471 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:38.218 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.213+0000 7fde99d9b640 1 --2- 172.21.15.164:0/1614118471 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fde9c148cf0 0x7fde9c06f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:38.218 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.213+0000 7fde9a59c640 1 -- 172.21.15.164:0/1614118471 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fde9c14eb80 msgr2=0x7fde9c144810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:38.218 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.213+0000 7fde9a59c640 1 --2- 172.21.15.164:0/1614118471 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fde9c14eb80 0x7fde9c144810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:38.218 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.213+0000 7fde9a59c640 1 -- 172.21.15.164:0/1614118471 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fde9c148cf0 msgr2=0x7fde9c06f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:38.219 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.213+0000 7fde9a59c640 1 --2- 172.21.15.164:0/1614118471 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fde9c148cf0 0x7fde9c06f1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:38.219 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.213+0000 7fde9a59c640 1 -- 172.21.15.164:0/1614118471 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde8c009420 con 0x7fde9c0b0230 2024-03-23T14:57:38.219 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.213+0000 7fde9a59c640 1 --2- 172.21.15.164:0/1614118471 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fde9c0b0230 0x7fde9c09b9a0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fde94001570 tx=0x7fde9400cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:38.219 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.213+0000 7fde92ffd640 1 -- 172.21.15.164:0/1614118471 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde9400a640 con 0x7fde9c0b0230 2024-03-23T14:57:38.219 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.213+0000 7fde92ffd640 1 -- 172.21.15.164:0/1614118471 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde9400a7c0 con 0x7fde9c0b0230 2024-03-23T14:57:38.219 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.213+0000 7fde92ffd640 1 -- 172.21.15.164:0/1614118471 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde9400ab70 con 0x7fde9c0b0230 2024-03-23T14:57:38.219 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.213+0000 7fdea0f5e640 1 -- 172.21.15.164:0/1614118471 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde9c0b0f70 con 0x7fde9c0b0230 2024-03-23T14:57:38.219 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.213+0000 7fdea0f5e640 1 -- 172.21.15.164:0/1614118471 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde9c06fd30 con 0x7fde9c0b0230 2024-03-23T14:57:38.219 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.213+0000 7fdea0f5e640 1 -- 172.21.15.164:0/1614118471 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde9c0690a0 con 0x7fde9c0b0230 2024-03-23T14:57:38.220 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.213+0000 7fde92ffd640 1 -- 172.21.15.164:0/1614118471 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7fde9400acd0 con 0x7fde9c0b0230 2024-03-23T14:57:38.220 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.213+0000 7fde92ffd640 1 --2- 172.21.15.164:0/1614118471 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fde7c0672a0 0x7fde7c069760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:38.220 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.213+0000 7fde92ffd640 1 -- 172.21.15.164:0/1614118471 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 4857+0+0 (secure 0 0 0) 0x7fde94084300 con 0x7fde9c0b0230 2024-03-23T14:57:38.220 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.213+0000 7fde99d9b640 1 --2- 172.21.15.164:0/1614118471 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fde7c0672a0 0x7fde7c069760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:38.220 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.217+0000 7fde99d9b640 1 --2- 172.21.15.164:0/1614118471 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fde7c0672a0 0x7fde7c069760 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fde9c139e60 tx=0x7fde84009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:38.223 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.217+0000 7fde92ffd640 1 -- 172.21.15.164:0/1614118471 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7fde94051210 con 0x7fde9c0b0230 2024-03-23T14:57:38.379 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.373+0000 7fdea0f5e640 1 -- 172.21.15.164:0/1614118471 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fde9c0b2fa0 con 0x7fde9c0b0230 2024-03-23T14:57:38.381 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.377+0000 7fde92ffd640 1 -- 172.21.15.164:0/1614118471 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+12915 (secure 0 0 0) 0x7fde940548d0 con 0x7fde9c0b0230 2024-03-23T14:57:38.381 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-23T14:57:38.381 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":25,"fsid":"45c7911b-b69d-4e68-907b-fbfeb780aa3b","created":"2024-03-23T14:56:34.367633+0000","modified":"2024-03-23T14:57:35.557194+0000","last_up_change":"2024-03-23T14:56:46.297403+0000","last_in_change":"2024-03-23T14:56:36.629884+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T14:56:47.186157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"80abdc62-bd0d-4e60-a554-7ed6fd67cf54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":984716690},{"type":"v1","addr":"172.21.15.164:6801","nonce":984716690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":984716690},{"type":"v1","addr":"172.21.15.164:6803","nonce":984716690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":984716690},{"type":"v1","addr":"172.21.15.164:6807","nonce":984716690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":984716690},{"type":"v1","addr":"172.21.15.164:6805","nonce":984716690}]},"public_addr":"172.21.15.164:6801/984716690","cluster_addr":"172.21.15.164:6803/984716690","heartbeat_back_addr":"172.21.15.164:6807/984716690","heartbeat_front_addr":"172.21.15.164:6805/984716690","state":["exists","up"]},{"osd":1,"uuid":"f9001b06-be4d-4f42-a0e6-bcef2afa0d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":1349657239},{"type":"v1","addr":"172.21.15.164:6817","nonce":1349657239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":1349657239},{"type":"v1","addr":"172.21.15.164:6819","nonce":1349657239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":1349657239},{"type":"v1","addr":"172.21.15.164:6823","nonce":1349657239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":1349657239},{"type":"v1","addr":"172.21.15.164:6821","nonce":1349657239}]},"public_addr":"172.21.15.164:6817/1349657239","cluster_addr":"172.21.15.164:6819/1349657239","heartbeat_back_addr":"172.21.15.164:6823/1349657239","heartbeat_front_addr":"172.21.15.164:6821/1349657239","state":["exists","up"]},{"osd":2,"uuid":"fa0aeb9f-8b54-4831-a188-3e361c7980d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":887861252},{"type":"v1","addr":"172.21.15.164:6809","nonce":887861252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":887861252},{"type":"v1","addr":"172.21.15.164:6811","nonce":887861252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":887861252},{"type":"v1","addr":"172.21.15.164:6815","nonce":887861252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":887861252},{"type":"v1","addr":"172.21.15.164:6813","nonce":887861252}]},"public_addr":"172.21.15.164:6809/887861252","cluster_addr":"172.21.15.164:6811/887861252","heartbeat_back_addr":"172.21.15.164:6815/887861252","heartbeat_front_addr":"172.21.15.164:6813/887861252","state":["exists","up"]},{"osd":3,"uuid":"7e18c957-3091-46d5-8e17-8f3354c1726c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":812540271},{"type":"v1","addr":"172.21.15.164:6825","nonce":812540271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":812540271},{"type":"v1","addr":"172.21.15.164:6827","nonce":812540271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":812540271},{"type":"v1","addr":"172.21.15.164:6831","nonce":812540271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":812540271},{"type":"v1","addr":"172.21.15.164:6829","nonce":812540271}]},"public_addr":"172.21.15.164:6825/812540271","cluster_addr":"172.21.15.164:6827/812540271","heartbeat_back_addr":"172.21.15.164:6831/812540271","heartbeat_front_addr":"172.21.15.164:6829/812540271","state":["exists","up"]},{"osd":4,"uuid":"8e83f315-42bf-4471-9ce8-d81948da0f9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":2656857421},{"type":"v1","addr":"172.21.15.173:6825","nonce":2656857421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2656857421},{"type":"v1","addr":"172.21.15.173:6827","nonce":2656857421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2656857421},{"type":"v1","addr":"172.21.15.173:6831","nonce":2656857421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":2656857421},{"type":"v1","addr":"172.21.15.173:6829","nonce":2656857421}]},"public_addr":"172.21.15.173:6825/2656857421","cluster_addr":"172.21.15.173:6827/2656857421","heartbeat_back_addr":"172.21.15.173:6831/2656857421","heartbeat_front_addr":"172.21.15.173:6829/2656857421","state":["exists","up"]},{"osd":5,"uuid":"a230c872-71f9-40ab-8999-defd0162d4ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":3676483185},{"type":"v1","addr":"172.21.15.173:6801","nonce":3676483185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":3676483185},{"type":"v1","addr":"172.21.15.173:6803","nonce":3676483185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":3676483185},{"type":"v1","addr":"172.21.15.173:6807","nonce":3676483185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":3676483185},{"type":"v1","addr":"172.21.15.173:6805","nonce":3676483185}]},"public_addr":"172.21.15.173:6801/3676483185","cluster_addr":"172.21.15.173:6803/3676483185","heartbeat_back_addr":"172.21.15.173:6807/3676483185","heartbeat_front_addr":"172.21.15.173:6805/3676483185","state":["exists","up"]},{"osd":6,"uuid":"6193bfd7-8432-4f7f-a117-a3fcee21af99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":3631684087},{"type":"v1","addr":"172.21.15.173:6809","nonce":3631684087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":3631684087},{"type":"v1","addr":"172.21.15.173:6811","nonce":3631684087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":3631684087},{"type":"v1","addr":"172.21.15.173:6815","nonce":3631684087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":3631684087},{"type":"v1","addr":"172.21.15.173:6813","nonce":3631684087}]},"public_addr":"172.21.15.173:6809/3631684087","cluster_addr":"172.21.15.173:6811/3631684087","heartbeat_back_addr":"172.21.15.173:6815/3631684087","heartbeat_front_addr":"172.21.15.173:6813/3631684087","state":["exists","up"]},{"osd":7,"uuid":"b0f1b80a-afc4-4538-83b9-f081d2dcb8cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":3261114948},{"type":"v1","addr":"172.21.15.173:6817","nonce":3261114948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":3261114948},{"type":"v1","addr":"172.21.15.173:6819","nonce":3261114948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":3261114948},{"type":"v1","addr":"172.21.15.173:6823","nonce":3261114948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":3261114948},{"type":"v1","addr":"172.21.15.173:6821","nonce":3261114948}]},"public_addr":"172.21.15.173:6817/3261114948","cluster_addr":"172.21.15.173:6819/3261114948","heartbeat_back_addr":"172.21.15.173:6823/3261114948","heartbeat_front_addr":"172.21.15.173:6821/3261114948","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.188390+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.159932+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.191101+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.181776+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.224365+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.177515+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.153631+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.139972+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T14:57:38.383 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.377+0000 7fdea0f5e640 1 -- 172.21.15.164:0/1614118471 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fde7c0672a0 msgr2=0x7fde7c069760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:38.383 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.377+0000 7fdea0f5e640 1 --2- 172.21.15.164:0/1614118471 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fde7c0672a0 0x7fde7c069760 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fde9c139e60 tx=0x7fde84009040 comp rx=0 tx=0).stop 2024-03-23T14:57:38.383 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.377+0000 7fdea0f5e640 1 -- 172.21.15.164:0/1614118471 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fde9c0b0230 msgr2=0x7fde9c09b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:38.383 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.377+0000 7fdea0f5e640 1 --2- 172.21.15.164:0/1614118471 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fde9c0b0230 0x7fde9c09b9a0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fde94001570 tx=0x7fde9400cd20 comp rx=0 tx=0).stop 2024-03-23T14:57:38.383 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.377+0000 7fdea0f5e640 1 -- 172.21.15.164:0/1614118471 shutdown_connections 2024-03-23T14:57:38.383 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.377+0000 7fdea0f5e640 1 --2- 172.21.15.164:0/1614118471 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fde9c14eb80 0x7fde9c144810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:38.383 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.377+0000 7fdea0f5e640 1 --2- 172.21.15.164:0/1614118471 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fde7c0672a0 0x7fde7c069760 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:38.384 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.377+0000 7fdea0f5e640 1 --2- 172.21.15.164:0/1614118471 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fde9c148cf0 0x7fde9c06f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:38.384 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.377+0000 7fdea0f5e640 1 --2- 172.21.15.164:0/1614118471 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fde9c0b0230 0x7fde9c09b9a0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:38.384 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.377+0000 7fdea0f5e640 1 -- 172.21.15.164:0/1614118471 >> 172.21.15.164:0/1614118471 conn(0x7fde9c0b8ca0 msgr2=0x7fde9c0b5560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:38.384 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.377+0000 7fdea0f5e640 1 -- 172.21.15.164:0/1614118471 shutdown_connections 2024-03-23T14:57:38.384 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.377+0000 7fdea0f5e640 1 -- 172.21.15.164:0/1614118471 wait complete. 2024-03-23T14:57:38.398 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T14:56:47.186157+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-23T14:57:38.399 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-23T14:57:38.505 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.501+0000 7f5f4d4d0640 1 Processor -- start 2024-03-23T14:57:38.505 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.501+0000 7f5f4d4d0640 1 -- start start 2024-03-23T14:57:38.506 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.501+0000 7f5f4d4d0640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5f480b0230 0x7f5f480b34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:38.506 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.501+0000 7f5f4d4d0640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f5f4814db70 0x7f5f4814ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:38.506 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.501+0000 7f5f4d4d0640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f480b51d0 con 0x7f5f48148cf0 2024-03-23T14:57:38.506 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.501+0000 7f5f4d4d0640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f5f480b0610 con 0x7f5f4814db70 2024-03-23T14:57:38.506 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.501+0000 7f5f4d4d0640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f5f480b1500 con 0x7f5f480b0230 2024-03-23T14:57:38.506 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.501+0000 7f5f477fe640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f5f4814db70 0x7f5f4814ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:38.506 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.501+0000 7f5f467fc640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f5f48148cf0 0x7f5f48149150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:58808/0 (socket says 172.21.15.164:58808) 2024-03-23T14:57:38.506 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.501+0000 7f5f467fc640 1 -- 172.21.15.164:0/4156824245 learned_addr learned my addr 172.21.15.164:0/4156824245 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-03-23T14:57:38.507 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.501+0000 7f5f477fe640 1 -- 172.21.15.164:0/4156824245 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5f480b0230 msgr2=0x7f5f480b34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:38.507 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.501+0000 7f5f46ffd640 1 --2- 172.21.15.164:0/4156824245 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5f480b0230 0x7f5f480b34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:38.507 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.501+0000 7f5f45ffb640 1 -- 172.21.15.164:0/4156824245 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1060507061 0 0) 0x7f5f480b51d0 con 0x7f5f48148cf0 2024-03-23T14:57:38.507 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.501+0000 7f5f477fe640 1 --2- 172.21.15.164:0/4156824245 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5f480b0230 0x7f5f480b34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:38.507 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.501+0000 7f5f477fe640 1 -- 172.21.15.164:0/4156824245 >> v1:172.21.15.164:6789/0 conn(0x7f5f48148cf0 legacy=0x7f5f48149150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:38.508 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.501+0000 7f5f477fe640 1 -- 172.21.15.164:0/4156824245 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f4806d250 con 0x7f5f4814db70 2024-03-23T14:57:38.508 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.501+0000 7f5f46ffd640 1 --2- 172.21.15.164:0/4156824245 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5f480b0230 0x7f5f480b34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T14:57:38.509 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.505+0000 7f5f477fe640 1 --2- 172.21.15.164:0/4156824245 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5f4814db70 0x7f5f4814ff60 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f5f380097c0 tx=0x7f5f3801c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0093a337b759d16 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:38.509 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.505+0000 7f5f45ffb640 1 -- 172.21.15.164:0/4156824245 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f380ac640 con 0x7f5f4814db70 2024-03-23T14:57:38.509 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.505+0000 7f5f45ffb640 1 -- 172.21.15.164:0/4156824245 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f380ac7a0 con 0x7f5f4814db70 2024-03-23T14:57:38.509 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.505+0000 7f5f45ffb640 1 -- 172.21.15.164:0/4156824245 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f380acb30 con 0x7f5f4814db70 2024-03-23T14:57:38.510 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.505+0000 7f5f4d4d0640 1 -- 172.21.15.164:0/4156824245 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5f4814db70 msgr2=0x7f5f4814ff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:38.510 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.505+0000 7f5f4d4d0640 1 --2- 172.21.15.164:0/4156824245 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5f4814db70 0x7f5f4814ff60 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f5f380097c0 tx=0x7f5f3801c3e0 comp rx=0 tx=0).stop 2024-03-23T14:57:38.510 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.505+0000 7f5f4d4d0640 1 -- 172.21.15.164:0/4156824245 shutdown_connections 2024-03-23T14:57:38.510 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.505+0000 7f5f4d4d0640 1 --2- 172.21.15.164:0/4156824245 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5f4814db70 0x7f5f4814ff60 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:38.510 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.505+0000 7f5f4d4d0640 1 --2- 172.21.15.164:0/4156824245 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5f480b0230 0x7f5f480b34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:38.510 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.505+0000 7f5f4d4d0640 1 -- 172.21.15.164:0/4156824245 >> 172.21.15.164:0/4156824245 conn(0x7f5f480b8ca0 msgr2=0x7f5f480b90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:38.510 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.505+0000 7f5f4d4d0640 1 -- 172.21.15.164:0/4156824245 shutdown_connections 2024-03-23T14:57:38.513 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.505+0000 7f5f4d4d0640 1 -- 172.21.15.164:0/4156824245 wait complete. 2024-03-23T14:57:38.516 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.505+0000 7f5f4d4d0640 1 Processor -- start 2024-03-23T14:57:38.516 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.505+0000 7f5f4d4d0640 1 -- start start 2024-03-23T14:57:38.516 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.505+0000 7f5f4d4d0640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5f480b0230 0x7f5f481445b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:38.516 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.505+0000 7f5f4d4d0640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5f48148cf0 0x7f5f4814ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:38.516 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.505+0000 7f5f4d4d0640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5f4814db70 0x7f5f481e3050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:38.516 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.505+0000 7f5f4d4d0640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5f48135990 con 0x7f5f4814db70 2024-03-23T14:57:38.516 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.505+0000 7f5f4d4d0640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f5f4813ac10 con 0x7f5f480b0230 2024-03-23T14:57:38.516 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.505+0000 7f5f4d4d0640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f5f4814f920 con 0x7f5f48148cf0 2024-03-23T14:57:38.516 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.505+0000 7f5f477fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5f4814db70 0x7f5f481e3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:38.516 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.505+0000 7f5f477fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5f4814db70 0x7f5f481e3050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:33282/0 (socket says 172.21.15.164:33282) 2024-03-23T14:57:38.516 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.505+0000 7f5f477fe640 1 -- 172.21.15.164:0/4142344450 learned_addr learned my addr 172.21.15.164:0/4142344450 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:38.516 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.509+0000 7f5f477fe640 1 -- 172.21.15.164:0/4142344450 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5f48148cf0 msgr2=0x7f5f4814ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:38.516 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.509+0000 7f5f477fe640 1 --2- 172.21.15.164:0/4142344450 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5f48148cf0 0x7f5f4814ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:38.516 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.509+0000 7f5f477fe640 1 -- 172.21.15.164:0/4142344450 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5f480b0230 msgr2=0x7f5f481445b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:38.517 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.509+0000 7f5f477fe640 1 --2- 172.21.15.164:0/4142344450 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5f480b0230 0x7f5f481445b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:38.517 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.509+0000 7f5f477fe640 1 -- 172.21.15.164:0/4142344450 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f38009420 con 0x7f5f4814db70 2024-03-23T14:57:38.517 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.509+0000 7f5f477fe640 1 --2- 172.21.15.164:0/4142344450 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5f4814db70 0x7f5f481e3050 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f5f3801aa90 tx=0x7f5f3801ac20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:38.517 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.509+0000 7f5f3f7fe640 1 -- 172.21.15.164:0/4142344450 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f38004230 con 0x7f5f4814db70 2024-03-23T14:57:38.517 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.509+0000 7f5f3f7fe640 1 -- 172.21.15.164:0/4142344450 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f38004390 con 0x7f5f4814db70 2024-03-23T14:57:38.517 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.509+0000 7f5f4d4d0640 1 -- 172.21.15.164:0/4142344450 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f4814fc00 con 0x7f5f4814db70 2024-03-23T14:57:38.517 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.509+0000 7f5f3f7fe640 1 -- 172.21.15.164:0/4142344450 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f38003820 con 0x7f5f4814db70 2024-03-23T14:57:38.517 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.509+0000 7f5f4d4d0640 1 -- 172.21.15.164:0/4142344450 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f4814e870 con 0x7f5f4814db70 2024-03-23T14:57:38.517 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.509+0000 7f5f4d4d0640 1 -- 172.21.15.164:0/4142344450 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f4814e400 con 0x7f5f4814db70 2024-03-23T14:57:38.517 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.509+0000 7f5f3f7fe640 1 -- 172.21.15.164:0/4142344450 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7f5f380b3040 con 0x7f5f4814db70 2024-03-23T14:57:38.517 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.509+0000 7f5f3f7fe640 1 --2- 172.21.15.164:0/4142344450 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f5f280672a0 0x7f5f28069760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:38.517 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.509+0000 7f5f46ffd640 1 --2- 172.21.15.164:0/4142344450 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f5f280672a0 0x7f5f28069760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:38.517 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.509+0000 7f5f3f7fe640 1 -- 172.21.15.164:0/4142344450 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 4857+0+0 (secure 0 0 0) 0x7f5f3811f620 con 0x7f5f4814db70 2024-03-23T14:57:38.517 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.509+0000 7f5f46ffd640 1 --2- 172.21.15.164:0/4142344450 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f5f280672a0 0x7f5f28069760 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f5f48144fc0 tx=0x7f5f400083c0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:38.518 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.513+0000 7f5f3f7fe640 1 -- 172.21.15.164:0/4142344450 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7f5f380ec5b0 con 0x7f5f4814db70 2024-03-23T14:57:38.665 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.661+0000 7f5f4d4d0640 1 -- 172.21.15.164:0/4142344450 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f5f481487d0 con 0x7f5f4814db70 2024-03-23T14:57:38.666 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.661+0000 7f5f3f7fe640 1 -- 172.21.15.164:0/4142344450 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) ==== 93+0+10 (secure 0 0 0) 0x7f5f380efc70 con 0x7f5f4814db70 2024-03-23T14:57:38.666 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 1 2024-03-23T14:57:38.668 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.665+0000 7f5f4d4d0640 1 -- 172.21.15.164:0/4142344450 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f5f280672a0 msgr2=0x7f5f28069760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:38.668 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.665+0000 7f5f4d4d0640 1 --2- 172.21.15.164:0/4142344450 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f5f280672a0 0x7f5f28069760 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f5f48144fc0 tx=0x7f5f400083c0 comp rx=0 tx=0).stop 2024-03-23T14:57:38.668 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.665+0000 7f5f4d4d0640 1 -- 172.21.15.164:0/4142344450 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5f4814db70 msgr2=0x7f5f481e3050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:38.669 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.665+0000 7f5f4d4d0640 1 --2- 172.21.15.164:0/4142344450 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5f4814db70 0x7f5f481e3050 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f5f3801aa90 tx=0x7f5f3801ac20 comp rx=0 tx=0).stop 2024-03-23T14:57:38.669 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.665+0000 7f5f4d4d0640 1 -- 172.21.15.164:0/4142344450 shutdown_connections 2024-03-23T14:57:38.669 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.665+0000 7f5f4d4d0640 1 --2- 172.21.15.164:0/4142344450 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5f4814db70 0x7f5f481e3050 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:38.669 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.665+0000 7f5f4d4d0640 1 --2- 172.21.15.164:0/4142344450 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5f48148cf0 0x7f5f4814ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:38.669 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.665+0000 7f5f4d4d0640 1 --2- 172.21.15.164:0/4142344450 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f5f280672a0 0x7f5f28069760 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:38.669 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.665+0000 7f5f4d4d0640 1 --2- 172.21.15.164:0/4142344450 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5f480b0230 0x7f5f481445b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:38.669 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.665+0000 7f5f4d4d0640 1 -- 172.21.15.164:0/4142344450 >> 172.21.15.164:0/4142344450 conn(0x7f5f480b8ca0 msgr2=0x7f5f480b43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:38.669 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.665+0000 7f5f4d4d0640 1 -- 172.21.15.164:0/4142344450 shutdown_connections 2024-03-23T14:57:38.669 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.665+0000 7f5f4d4d0640 1 -- 172.21.15.164:0/4142344450 wait complete. 2024-03-23T14:57:38.684 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-23T14:57:38.792 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.789+0000 7f2356e75640 1 Processor -- start 2024-03-23T14:57:38.793 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.789+0000 7f2356e75640 1 -- start start 2024-03-23T14:57:38.794 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.789+0000 7f2356e75640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f23500b55e0 0x7f23500b5a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:38.794 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.789+0000 7f2356e75640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f2350147920 0x7f2350147d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:38.794 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.789+0000 7f2356e75640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2350072d70 con 0x7f23500b5200 2024-03-23T14:57:38.794 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.789+0000 7f2356e75640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f23500af910 con 0x7f23500b55e0 2024-03-23T14:57:38.794 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.789+0000 7f2356e75640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 -- 0x7f235009f170 con 0x7f2350147920 2024-03-23T14:57:38.794 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.789+0000 7f234ffff640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f23500b55e0 0x7f23500b5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:38.794 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.789+0000 7f2354c11640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f23500b5200 0x7f235014ab30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:58822/0 (socket says 172.21.15.164:58822) 2024-03-23T14:57:38.794 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.789+0000 7f2354c11640 1 -- 172.21.15.164:0/542310805 learned_addr learned my addr 172.21.15.164:0/542310805 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-03-23T14:57:38.794 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.789+0000 7f234ffff640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f23500b55e0 0x7f23500b5a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:33290/0 (socket says 172.21.15.164:33290) 2024-03-23T14:57:38.794 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.789+0000 7f234ffff640 1 -- 172.21.15.164:0/542310805 >> v2:172.21.15.173:3300/0 conn(0x7f2350147920 msgr2=0x7f2350147d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:38.794 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.789+0000 7f234ffff640 1 --2- 172.21.15.164:0/542310805 >> v2:172.21.15.173:3300/0 conn(0x7f2350147920 0x7f2350147d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:38.794 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.789+0000 7f234ffff640 1 -- 172.21.15.164:0/542310805 >> v1:172.21.15.164:6789/0 conn(0x7f23500b5200 legacy=0x7f235014ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:38.795 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.789+0000 7f234ffff640 1 -- 172.21.15.164:0/542310805 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f235006e590 con 0x7f23500b55e0 2024-03-23T14:57:38.795 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.789+0000 7f234f7fe640 1 -- 172.21.15.164:0/542310805 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3526982093 0 0) 0x7f2350072d70 con 0x7f23500b5200 2024-03-23T14:57:38.796 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.793+0000 7f234ffff640 1 --2- 172.21.15.164:0/542310805 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f23500b55e0 0x7f23500b5a70 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f235006c260 tx=0x7f234401c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5eb4fa21fc5328d server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:38.796 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.793+0000 7f234f7fe640 1 -- 172.21.15.164:0/542310805 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23440ac410 con 0x7f23500b55e0 2024-03-23T14:57:38.796 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.793+0000 7f234f7fe640 1 -- 172.21.15.164:0/542310805 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23440ac570 con 0x7f23500b55e0 2024-03-23T14:57:38.796 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.793+0000 7f234f7fe640 1 -- 172.21.15.164:0/542310805 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23440ac900 con 0x7f23500b55e0 2024-03-23T14:57:38.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.793+0000 7f2356e75640 1 -- 172.21.15.164:0/542310805 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f23500b55e0 msgr2=0x7f23500b5a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:38.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.793+0000 7f2356e75640 1 --2- 172.21.15.164:0/542310805 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f23500b55e0 0x7f23500b5a70 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f235006c260 tx=0x7f234401c3f0 comp rx=0 tx=0).stop 2024-03-23T14:57:38.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.793+0000 7f2356e75640 1 -- 172.21.15.164:0/542310805 shutdown_connections 2024-03-23T14:57:38.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.793+0000 7f2356e75640 1 --2- 172.21.15.164:0/542310805 >> v2:172.21.15.173:3300/0 conn(0x7f2350147920 0x7f2350147d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:38.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.793+0000 7f2356e75640 1 --2- 172.21.15.164:0/542310805 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f23500b55e0 0x7f23500b5a70 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:38.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.793+0000 7f2356e75640 1 -- 172.21.15.164:0/542310805 >> 172.21.15.164:0/542310805 conn(0x7f23500b9ae0 msgr2=0x7f23500b9ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:38.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.793+0000 7f2356e75640 1 -- 172.21.15.164:0/542310805 shutdown_connections 2024-03-23T14:57:38.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.793+0000 7f2356e75640 1 -- 172.21.15.164:0/542310805 wait complete. 2024-03-23T14:57:38.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.793+0000 7f2356e75640 1 Processor -- start 2024-03-23T14:57:38.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.793+0000 7f2356e75640 1 -- start start 2024-03-23T14:57:38.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.793+0000 7f2356e75640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f23500b5200 0x7f235013fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:38.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.793+0000 7f2356e75640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f23500b55e0 0x7f23501e34f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:38.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.793+0000 7f2356e75640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2350147920 0x7f23500b3110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:38.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.793+0000 7f2356e75640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f23501406e0 con 0x7f2350147920 2024-03-23T14:57:38.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.793+0000 7f2356e75640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f2350147420 con 0x7f23500b55e0 2024-03-23T14:57:38.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.793+0000 7f2356e75640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f23501439a0 con 0x7f23500b5200 2024-03-23T14:57:38.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.793+0000 7f2355412640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2350147920 0x7f23500b3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:38.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.793+0000 7f2355412640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2350147920 0x7f23500b3110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:33304/0 (socket says 172.21.15.164:33304) 2024-03-23T14:57:38.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.793+0000 7f2355412640 1 -- 172.21.15.164:0/3835980292 learned_addr learned my addr 172.21.15.164:0/3835980292 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:38.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.793+0000 7f2355412640 1 -- 172.21.15.164:0/3835980292 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f23500b5200 msgr2=0x7f235013fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:38.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.793+0000 7f234ffff640 1 --2- 172.21.15.164:0/3835980292 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f23500b55e0 0x7f23501e34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:38.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.793+0000 7f2355412640 1 --2- 172.21.15.164:0/3835980292 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f23500b5200 0x7f235013fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:38.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.793+0000 7f2355412640 1 -- 172.21.15.164:0/3835980292 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f23500b55e0 msgr2=0x7f23501e34f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:38.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.793+0000 7f2355412640 1 --2- 172.21.15.164:0/3835980292 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f23500b55e0 0x7f23501e34f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:38.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.793+0000 7f2355412640 1 -- 172.21.15.164:0/3835980292 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2344009420 con 0x7f2350147920 2024-03-23T14:57:38.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.793+0000 7f2355412640 1 --2- 172.21.15.164:0/3835980292 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2350147920 0x7f23500b3110 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f234800b970 tx=0x7f234800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:38.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.793+0000 7f234d7fa640 1 -- 172.21.15.164:0/3835980292 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f234800cb40 con 0x7f2350147920 2024-03-23T14:57:38.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.793+0000 7f234d7fa640 1 -- 172.21.15.164:0/3835980292 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f234800ccc0 con 0x7f2350147920 2024-03-23T14:57:38.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.793+0000 7f2356e75640 1 -- 172.21.15.164:0/3835980292 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2350146b40 con 0x7f2350147920 2024-03-23T14:57:38.801 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.793+0000 7f234d7fa640 1 -- 172.21.15.164:0/3835980292 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2348014680 con 0x7f2350147920 2024-03-23T14:57:38.801 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.793+0000 7f2356e75640 1 -- 172.21.15.164:0/3835980292 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f235013f7f0 con 0x7f2350147920 2024-03-23T14:57:38.801 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.797+0000 7f2356e75640 1 -- 172.21.15.164:0/3835980292 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f235013f650 con 0x7f2350147920 2024-03-23T14:57:38.801 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.797+0000 7f234d7fa640 1 -- 172.21.15.164:0/3835980292 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7f234801d3b0 con 0x7f2350147920 2024-03-23T14:57:38.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.797+0000 7f234d7fa640 1 --2- 172.21.15.164:0/3835980292 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f233c0672a0 0x7f233c069760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:38.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.797+0000 7f234d7fa640 1 -- 172.21.15.164:0/3835980292 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 4857+0+0 (secure 0 0 0) 0x7f2348084c50 con 0x7f2350147920 2024-03-23T14:57:38.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.797+0000 7f2354c11640 1 --2- 172.21.15.164:0/3835980292 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f233c0672a0 0x7f233c069760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:38.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.797+0000 7f2354c11640 1 --2- 172.21.15.164:0/3835980292 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f233c0672a0 0x7f233c069760 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f23400015a0 tx=0x7f2340007970 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:38.805 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.801+0000 7f234d7fa640 1 -- 172.21.15.164:0/3835980292 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7f235013f650 con 0x7f2350147920 2024-03-23T14:57:38.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.945+0000 7f2356e75640 1 -- 172.21.15.164:0/3835980292 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f235013f650 con 0x7f2350147920 2024-03-23T14:57:38.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.949+0000 7f234d7fa640 1 -- 172.21.15.164:0/3835980292 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v25) ==== 84+0+65 (secure 0 0 0) 0x7f235013f650 con 0x7f2350147920 2024-03-23T14:57:38.952 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-23T14:57:38.952 INFO:teuthology.orchestra.run.smithi164.stdout:[ 2024-03-23T14:57:38.952 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-03-23T14:57:38.952 INFO:teuthology.orchestra.run.smithi164.stdout: "poolnum": 1, 2024-03-23T14:57:38.952 INFO:teuthology.orchestra.run.smithi164.stdout: "poolname": ".mgr" 2024-03-23T14:57:38.952 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-03-23T14:57:38.952 INFO:teuthology.orchestra.run.smithi164.stdout:] 2024-03-23T14:57:38.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.949+0000 7f2356e75640 1 -- 172.21.15.164:0/3835980292 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f233c0672a0 msgr2=0x7f233c069760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:38.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.949+0000 7f2356e75640 1 --2- 172.21.15.164:0/3835980292 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f233c0672a0 0x7f233c069760 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f23400015a0 tx=0x7f2340007970 comp rx=0 tx=0).stop 2024-03-23T14:57:38.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.949+0000 7f2356e75640 1 -- 172.21.15.164:0/3835980292 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2350147920 msgr2=0x7f23500b3110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:38.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.949+0000 7f2356e75640 1 --2- 172.21.15.164:0/3835980292 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2350147920 0x7f23500b3110 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f234800b970 tx=0x7f234800be30 comp rx=0 tx=0).stop 2024-03-23T14:57:38.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.949+0000 7f2356e75640 1 -- 172.21.15.164:0/3835980292 shutdown_connections 2024-03-23T14:57:38.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.949+0000 7f2356e75640 1 --2- 172.21.15.164:0/3835980292 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2350147920 0x7f23500b3110 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:38.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.949+0000 7f2356e75640 1 --2- 172.21.15.164:0/3835980292 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f233c0672a0 0x7f233c069760 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:38.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.949+0000 7f2356e75640 1 --2- 172.21.15.164:0/3835980292 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f23500b55e0 0x7f23501e34f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:38.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.949+0000 7f2356e75640 1 --2- 172.21.15.164:0/3835980292 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f23500b5200 0x7f235013fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:38.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.949+0000 7f2356e75640 1 -- 172.21.15.164:0/3835980292 >> 172.21.15.164:0/3835980292 conn(0x7f23500b9ae0 msgr2=0x7f23500b9190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:38.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.949+0000 7f2356e75640 1 -- 172.21.15.164:0/3835980292 shutdown_connections 2024-03-23T14:57:38.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:38.949+0000 7f2356e75640 1 -- 172.21.15.164:0/3835980292 wait complete. 2024-03-23T14:57:38.969 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-23T14:57:38.970 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-23T14:57:39.083 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.077+0000 7ff831687640 1 Processor -- start 2024-03-23T14:57:39.083 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.077+0000 7ff831687640 1 -- start start 2024-03-23T14:57:39.083 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.077+0000 7ff831687640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff82c0b0230 0x7ff82c0b34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:39.083 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.077+0000 7ff831687640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7ff82c14db70 0x7ff82c14ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:39.083 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.077+0000 7ff831687640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff82c0b51d0 con 0x7ff82c148cf0 2024-03-23T14:57:39.083 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.077+0000 7ff831687640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 -- 0x7ff82c0b0610 con 0x7ff82c14db70 2024-03-23T14:57:39.083 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.077+0000 7ff831687640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7ff82c0b1500 con 0x7ff82c0b0230 2024-03-23T14:57:39.083 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.077+0000 7ff82a7fc640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7ff82c148cf0 0x7ff82c149150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:58832/0 (socket says 172.21.15.164:58832) 2024-03-23T14:57:39.084 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.077+0000 7ff82a7fc640 1 -- 172.21.15.164:0/2084877879 learned_addr learned my addr 172.21.15.164:0/2084877879 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-03-23T14:57:39.084 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.077+0000 7ff82b7fe640 1 --2- 172.21.15.164:0/2084877879 >> v2:172.21.15.173:3300/0 conn(0x7ff82c14db70 0x7ff82c14ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:39.084 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.077+0000 7ff82affd640 1 --2- 172.21.15.164:0/2084877879 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff82c0b0230 0x7ff82c0b34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:39.084 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.077+0000 7ff829ffb640 1 -- 172.21.15.164:0/2084877879 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 770914172 0 0) 0x7ff82c0b51d0 con 0x7ff82c148cf0 2024-03-23T14:57:39.084 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.077+0000 7ff829ffb640 1 -- 172.21.15.164:0/2084877879 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff814003530 con 0x7ff82c148cf0 2024-03-23T14:57:39.084 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.077+0000 7ff829ffb640 1 -- 172.21.15.164:0/2084877879 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 214493261 0 0) 0x7ff814003530 con 0x7ff82c148cf0 2024-03-23T14:57:39.084 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.077+0000 7ff829ffb640 1 -- 172.21.15.164:0/2084877879 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff82c0b0230 msgr2=0x7ff82c0b34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:39.084 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.077+0000 7ff829ffb640 1 --2- 172.21.15.164:0/2084877879 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff82c0b0230 0x7ff82c0b34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:39.084 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.077+0000 7ff829ffb640 1 -- 172.21.15.164:0/2084877879 >> v2:172.21.15.173:3300/0 conn(0x7ff82c14db70 msgr2=0x7ff82c14ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:39.084 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.077+0000 7ff829ffb640 1 --2- 172.21.15.164:0/2084877879 >> v2:172.21.15.173:3300/0 conn(0x7ff82c14db70 0x7ff82c14ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:39.085 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.077+0000 7ff829ffb640 1 -- 172.21.15.164:0/2084877879 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff82c06d250 con 0x7ff82c148cf0 2024-03-23T14:57:39.085 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.077+0000 7ff829ffb640 1 -- 172.21.15.164:0/2084877879 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2526390548 0 0) 0x7ff820001680 con 0x7ff82c148cf0 2024-03-23T14:57:39.085 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.077+0000 7ff829ffb640 1 -- 172.21.15.164:0/2084877879 >> v1:172.21.15.164:6789/0 conn(0x7ff82c148cf0 legacy=0x7ff82c149150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:39.085 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.077+0000 7ff829ffb640 1 --2- 172.21.15.164:0/2084877879 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff82c0b0230 0x7ff814003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:39.085 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.077+0000 7ff829ffb640 1 --2- 172.21.15.164:0/2084877879 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff814003f00 0x7ff8140047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:39.085 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.077+0000 7ff829ffb640 1 --2- 172.21.15.164:0/2084877879 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff814004d10 0x7ff814005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:39.085 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.077+0000 7ff829ffb640 1 -- 172.21.15.164:0/2084877879 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff82c0b0610 con 0x7ff814003f00 2024-03-23T14:57:39.085 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.077+0000 7ff829ffb640 1 -- 172.21.15.164:0/2084877879 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ff82c0b1500 con 0x7ff814004d10 2024-03-23T14:57:39.085 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.077+0000 7ff829ffb640 1 -- 172.21.15.164:0/2084877879 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7ff81400b9e0 con 0x7ff82c0b0230 2024-03-23T14:57:39.085 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.077+0000 7ff82a7fc640 1 --2- 172.21.15.164:0/2084877879 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff814003f00 0x7ff8140047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:39.085 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.077+0000 7ff82affd640 1 --2- 172.21.15.164:0/2084877879 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff82c0b0230 0x7ff814003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:39.085 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.077+0000 7ff82b7fe640 1 --2- 172.21.15.164:0/2084877879 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff814004d10 0x7ff814005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:39.085 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.077+0000 7ff82a7fc640 1 -- 172.21.15.164:0/2084877879 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff82c0b0230 msgr2=0x7ff814003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:39.086 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.077+0000 7ff82a7fc640 1 --2- 172.21.15.164:0/2084877879 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff82c0b0230 0x7ff814003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:39.086 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.077+0000 7ff82a7fc640 1 -- 172.21.15.164:0/2084877879 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff814004d10 msgr2=0x7ff814005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:39.086 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.077+0000 7ff82a7fc640 1 --2- 172.21.15.164:0/2084877879 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff814004d10 0x7ff814005190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:39.086 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.077+0000 7ff82a7fc640 1 -- 172.21.15.164:0/2084877879 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff814003530 con 0x7ff814003f00 2024-03-23T14:57:39.086 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.077+0000 7ff82affd640 1 --2- 172.21.15.164:0/2084877879 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff82c0b0230 0x7ff814003980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T14:57:39.090 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.081+0000 7ff82a7fc640 1 --2- 172.21.15.164:0/2084877879 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff814003f00 0x7ff8140047d0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7ff82000c270 tx=0x7ff82001cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b64c6c55cd9d986 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:39.090 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.081+0000 7ff829ffb640 1 -- 172.21.15.164:0/2084877879 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8200a5930 con 0x7ff814003f00 2024-03-23T14:57:39.090 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.081+0000 7ff829ffb640 1 -- 172.21.15.164:0/2084877879 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8200a5a90 con 0x7ff814003f00 2024-03-23T14:57:39.091 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.081+0000 7ff829ffb640 1 -- 172.21.15.164:0/2084877879 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8200a5ea0 con 0x7ff814003f00 2024-03-23T14:57:39.091 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.081+0000 7ff831687640 1 -- 172.21.15.164:0/2084877879 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff814003f00 msgr2=0x7ff8140047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:39.091 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.081+0000 7ff831687640 1 --2- 172.21.15.164:0/2084877879 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff814003f00 0x7ff8140047d0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7ff82000c270 tx=0x7ff82001cc50 comp rx=0 tx=0).stop 2024-03-23T14:57:39.091 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.081+0000 7ff82b7fe640 1 -- 172.21.15.164:0/2084877879 reap_dead start 2024-03-23T14:57:39.091 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.081+0000 7ff831687640 1 -- 172.21.15.164:0/2084877879 shutdown_connections 2024-03-23T14:57:39.091 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.081+0000 7ff831687640 1 -- 172.21.15.164:0/2084877879 >> 172.21.15.164:0/2084877879 conn(0x7ff82c0b8ca0 msgr2=0x7ff82c0b90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:39.091 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.081+0000 7ff831687640 1 -- 172.21.15.164:0/2084877879 shutdown_connections 2024-03-23T14:57:39.091 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.081+0000 7ff831687640 1 -- 172.21.15.164:0/2084877879 wait complete. 2024-03-23T14:57:39.091 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.081+0000 7ff831687640 1 Processor -- start 2024-03-23T14:57:39.091 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.081+0000 7ff831687640 1 -- start start 2024-03-23T14:57:39.092 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.081+0000 7ff831687640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff82c147b70 0x7ff82c1e3140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:39.092 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.081+0000 7ff831687640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff82c09b500 0x7ff82c1e3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:39.092 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.081+0000 7ff831687640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff82c148630 0x7ff82c1e3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:39.092 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.081+0000 7ff831687640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff82c09d720 con 0x7ff82c09b500 2024-03-23T14:57:39.092 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.081+0000 7ff831687640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ff82c09d540 con 0x7ff82c147b70 2024-03-23T14:57:39.092 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.081+0000 7ff831687640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7ff82c072530 con 0x7ff82c148630 2024-03-23T14:57:39.092 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.081+0000 7ff82a7fc640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff82c09b500 0x7ff82c1e3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:39.092 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.081+0000 7ff82a7fc640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff82c09b500 0x7ff82c1e3680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:33314/0 (socket says 172.21.15.164:33314) 2024-03-23T14:57:39.092 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.081+0000 7ff82a7fc640 1 -- 172.21.15.164:0/2549995010 learned_addr learned my addr 172.21.15.164:0/2549995010 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:39.092 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.081+0000 7ff82affd640 1 --2- 172.21.15.164:0/2549995010 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff82c147b70 0x7ff82c1e3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:39.092 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.081+0000 7ff82a7fc640 1 -- 172.21.15.164:0/2549995010 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff82c148630 msgr2=0x7ff82c1e3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:39.092 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.081+0000 7ff82a7fc640 1 --2- 172.21.15.164:0/2549995010 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff82c148630 0x7ff82c1e3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:39.092 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.081+0000 7ff82a7fc640 1 -- 172.21.15.164:0/2549995010 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff82c147b70 msgr2=0x7ff82c1e3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:39.093 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.081+0000 7ff82a7fc640 1 --2- 172.21.15.164:0/2549995010 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff82c147b70 0x7ff82c1e3140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:39.093 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.081+0000 7ff82a7fc640 1 -- 172.21.15.164:0/2549995010 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff82c0b0f70 con 0x7ff82c09b500 2024-03-23T14:57:39.093 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.081+0000 7ff82a7fc640 1 --2- 172.21.15.164:0/2549995010 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff82c09b500 0x7ff82c1e3680 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7ff82000c210 tx=0x7ff814005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:39.093 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.081+0000 7ff8137fe640 1 -- 172.21.15.164:0/2549995010 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff820004ab0 con 0x7ff82c09b500 2024-03-23T14:57:39.093 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.081+0000 7ff8137fe640 1 -- 172.21.15.164:0/2549995010 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff820004c10 con 0x7ff82c09b500 2024-03-23T14:57:39.093 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.081+0000 7ff831687640 1 -- 172.21.15.164:0/2549995010 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff82c147f50 con 0x7ff82c09b500 2024-03-23T14:57:39.093 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.081+0000 7ff8137fe640 1 -- 172.21.15.164:0/2549995010 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff820001c60 con 0x7ff82c09b500 2024-03-23T14:57:39.093 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.081+0000 7ff831687640 1 -- 172.21.15.164:0/2549995010 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff82c09bd30 con 0x7ff82c09b500 2024-03-23T14:57:39.093 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.085+0000 7ff831687640 1 -- 172.21.15.164:0/2549995010 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff82c072390 con 0x7ff82c09b500 2024-03-23T14:57:39.093 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.085+0000 7ff8137fe640 1 -- 172.21.15.164:0/2549995010 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7ff8200b9430 con 0x7ff82c09b500 2024-03-23T14:57:39.093 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.085+0000 7ff8137fe640 1 --2- 172.21.15.164:0/2549995010 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7ff8180672d0 0x7ff818069790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:39.093 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.085+0000 7ff82affd640 1 --2- 172.21.15.164:0/2549995010 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7ff8180672d0 0x7ff818069790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:39.093 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.085+0000 7ff8137fe640 1 -- 172.21.15.164:0/2549995010 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 4857+0+0 (secure 0 0 0) 0x7ff8200b4030 con 0x7ff82c09b500 2024-03-23T14:57:39.094 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.085+0000 7ff82affd640 1 --2- 172.21.15.164:0/2549995010 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7ff8180672d0 0x7ff818069790 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff82c0af3f0 tx=0x7ff81c00d040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:39.094 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.089+0000 7ff8137fe640 1 -- 172.21.15.164:0/2549995010 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7ff82c072390 con 0x7ff82c09b500 2024-03-23T14:57:39.253 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.249+0000 7ff831687640 1 -- 172.21.15.164:0/2549995010 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7ff82c072390 con 0x7ff82c09b500 2024-03-23T14:57:39.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.573+0000 7ff8137fe640 1 -- 172.21.15.164:0/2549995010 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v26) ==== 138+0+0 (secure 0 0 0) 0x7ff82c072390 con 0x7ff82c09b500 2024-03-23T14:57:39.577 INFO:teuthology.orchestra.run.smithi164.stderr:pool 'cephfs_metadata' created 2024-03-23T14:57:39.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.577+0000 7ff831687640 1 -- 172.21.15.164:0/2549995010 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7ff8180672d0 msgr2=0x7ff818069790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:39.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.577+0000 7ff831687640 1 --2- 172.21.15.164:0/2549995010 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7ff8180672d0 0x7ff818069790 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff82c0af3f0 tx=0x7ff81c00d040 comp rx=0 tx=0).stop 2024-03-23T14:57:39.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.577+0000 7ff831687640 1 -- 172.21.15.164:0/2549995010 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff82c09b500 msgr2=0x7ff82c1e3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:39.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.577+0000 7ff831687640 1 --2- 172.21.15.164:0/2549995010 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff82c09b500 0x7ff82c1e3680 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7ff82000c210 tx=0x7ff814005800 comp rx=0 tx=0).stop 2024-03-23T14:57:39.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.577+0000 7ff831687640 1 -- 172.21.15.164:0/2549995010 shutdown_connections 2024-03-23T14:57:39.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.577+0000 7ff831687640 1 --2- 172.21.15.164:0/2549995010 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff82c148630 0x7ff82c1e3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:39.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.577+0000 7ff831687640 1 --2- 172.21.15.164:0/2549995010 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff82c09b500 0x7ff82c1e3680 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:39.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.577+0000 7ff831687640 1 --2- 172.21.15.164:0/2549995010 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7ff8180672d0 0x7ff818069790 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:39.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.577+0000 7ff831687640 1 --2- 172.21.15.164:0/2549995010 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff82c147b70 0x7ff82c1e3140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:39.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.577+0000 7ff831687640 1 -- 172.21.15.164:0/2549995010 >> 172.21.15.164:0/2549995010 conn(0x7ff82c0b8ca0 msgr2=0x7ff82c0b2310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:39.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.577+0000 7ff831687640 1 -- 172.21.15.164:0/2549995010 shutdown_connections 2024-03-23T14:57:39.582 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.577+0000 7ff831687640 1 -- 172.21.15.164:0/2549995010 wait complete. 2024-03-23T14:57:39.607 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-23T14:57:39.735 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.729+0000 7f13161f5640 1 Processor -- start 2024-03-23T14:57:39.736 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.729+0000 7f13161f5640 1 -- start start 2024-03-23T14:57:39.738 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.733+0000 7f13161f5640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f13100b56f0 0x7f131014a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:39.738 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.733+0000 7f13161f5640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13100b4300 con 0x7f1310141640 2024-03-23T14:57:39.738 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.733+0000 7f13161f5640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13100b45e0 con 0x7f13100b4a70 2024-03-23T14:57:39.738 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.733+0000 7f13161f5640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 -- 0x7f13100b5fa0 con 0x7f13100b56f0 2024-03-23T14:57:39.738 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.733+0000 7f130ffff640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f1310141640 0x7f1310151c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:58840/0 (socket says 172.21.15.164:58840) 2024-03-23T14:57:39.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.733+0000 7f130ffff640 1 -- 172.21.15.164:0/3901524732 learned_addr learned my addr 172.21.15.164:0/3901524732 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-03-23T14:57:39.741 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.733+0000 7f130f7fe640 1 --2- 172.21.15.164:0/3901524732 >> v2:172.21.15.173:3300/0 conn(0x7f13100b56f0 0x7f131014a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:39.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.733+0000 7f130e7fc640 1 -- 172.21.15.164:0/3901524732 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4152660289 0 0) 0x7f13100b4300 con 0x7f1310141640 2024-03-23T14:57:39.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.733+0000 7f130e7fc640 1 -- 172.21.15.164:0/3901524732 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f12f8003530 con 0x7f1310141640 2024-03-23T14:57:39.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.733+0000 7f130e7fc640 1 -- 172.21.15.164:0/3901524732 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 815671773 0 0) 0x7f12f8003530 con 0x7f1310141640 2024-03-23T14:57:39.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.733+0000 7f130e7fc640 1 -- 172.21.15.164:0/3901524732 >> v2:172.21.15.173:3300/0 conn(0x7f13100b56f0 msgr2=0x7f131014a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:39.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.733+0000 7f130e7fc640 1 --2- 172.21.15.164:0/3901524732 >> v2:172.21.15.173:3300/0 conn(0x7f13100b56f0 0x7f131014a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:39.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.733+0000 7f130e7fc640 1 -- 172.21.15.164:0/3901524732 >> v1:172.21.15.173:6789/0 conn(0x7f13100b4a70 legacy=0x7f1310140f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:39.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.733+0000 7f130e7fc640 1 -- 172.21.15.164:0/3901524732 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f131014ade0 con 0x7f1310141640 2024-03-23T14:57:39.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.733+0000 7f130e7fc640 1 -- 172.21.15.164:0/3901524732 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2526390548 0 0) 0x7f1308001a00 con 0x7f1310141640 2024-03-23T14:57:39.742 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.733+0000 7f130e7fc640 1 -- 172.21.15.164:0/3901524732 >> v1:172.21.15.164:6789/0 conn(0x7f1310141640 legacy=0x7f1310151c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:39.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.733+0000 7f130e7fc640 1 --2- 172.21.15.164:0/3901524732 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f12f80039d0 0x7f12f8003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:39.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.733+0000 7f130e7fc640 1 --2- 172.21.15.164:0/3901524732 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f12f80047d0 0x7f12f8004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:39.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.733+0000 7f130e7fc640 1 --2- 172.21.15.164:0/3901524732 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f12f8005190 0x7f12f800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:39.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.733+0000 7f130e7fc640 1 -- 172.21.15.164:0/3901524732 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f12f800c060 con 0x7f12f8005190 2024-03-23T14:57:39.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.733+0000 7f130e7fc640 1 -- 172.21.15.164:0/3901524732 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f12f800c1d0 con 0x7f12f80039d0 2024-03-23T14:57:39.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.733+0000 7f130e7fc640 1 -- 172.21.15.164:0/3901524732 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f12f800c390 con 0x7f12f80047d0 2024-03-23T14:57:39.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.733+0000 7f130ffff640 1 --2- 172.21.15.164:0/3901524732 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f12f8005190 0x7f12f800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:39.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.733+0000 7f130f7fe640 1 --2- 172.21.15.164:0/3901524732 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f12f80039d0 0x7f12f8003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:39.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.733+0000 7f130effd640 1 --2- 172.21.15.164:0/3901524732 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f12f80047d0 0x7f12f8004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:39.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.733+0000 7f130ffff640 1 -- 172.21.15.164:0/3901524732 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f12f80047d0 msgr2=0x7f12f8004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:39.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.733+0000 7f130ffff640 1 --2- 172.21.15.164:0/3901524732 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f12f80047d0 0x7f12f8004c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:39.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.733+0000 7f130ffff640 1 -- 172.21.15.164:0/3901524732 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f12f80039d0 msgr2=0x7f12f8003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:39.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.733+0000 7f130ffff640 1 --2- 172.21.15.164:0/3901524732 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f12f80039d0 0x7f12f8003df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:39.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.733+0000 7f130ffff640 1 -- 172.21.15.164:0/3901524732 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12f8003530 con 0x7f12f8005190 2024-03-23T14:57:39.747 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.733+0000 7f130ffff640 1 -- 172.21.15.164:0/3901524732 reap_dead start 2024-03-23T14:57:39.747 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.733+0000 7f130ffff640 1 --2- 172.21.15.164:0/3901524732 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f12f8005190 0x7f12f800b990 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f130800a760 tx=0x7f130801d280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8e4341b78a44fd server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:39.748 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.737+0000 7f130e7fc640 1 -- 172.21.15.164:0/3901524732 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13080ad630 con 0x7f12f8005190 2024-03-23T14:57:39.748 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.737+0000 7f130e7fc640 1 -- 172.21.15.164:0/3901524732 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13080a8070 con 0x7f12f8005190 2024-03-23T14:57:39.748 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.737+0000 7f130e7fc640 1 -- 172.21.15.164:0/3901524732 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13080ada10 con 0x7f12f8005190 2024-03-23T14:57:39.748 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.737+0000 7f13161f5640 1 -- 172.21.15.164:0/3901524732 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f12f8005190 msgr2=0x7f12f800b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:39.748 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.737+0000 7f13161f5640 1 --2- 172.21.15.164:0/3901524732 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f12f8005190 0x7f12f800b990 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f130800a760 tx=0x7f130801d280 comp rx=0 tx=0).stop 2024-03-23T14:57:39.748 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.737+0000 7f13161f5640 1 -- 172.21.15.164:0/3901524732 shutdown_connections 2024-03-23T14:57:39.748 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.737+0000 7f13161f5640 1 --2- 172.21.15.164:0/3901524732 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f12f8005190 0x7f12f800b990 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:39.748 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.737+0000 7f13161f5640 1 -- 172.21.15.164:0/3901524732 >> 172.21.15.164:0/3901524732 conn(0x7f131002c3c0 msgr2=0x7f131002c7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:39.748 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.737+0000 7f13161f5640 1 -- 172.21.15.164:0/3901524732 shutdown_connections 2024-03-23T14:57:39.748 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.737+0000 7f13161f5640 1 -- 172.21.15.164:0/3901524732 wait complete. 2024-03-23T14:57:39.748 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.737+0000 7f13161f5640 1 Processor -- start 2024-03-23T14:57:39.748 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.737+0000 7f13161f5640 1 -- start start 2024-03-23T14:57:39.748 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.737+0000 7f13161f5640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f12f8005190 0x7f1310147760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:39.748 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.737+0000 7f13161f5640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f13101435c0 0x7f13101df850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:39.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.737+0000 7f13161f5640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f13101dfd90 0x7f1310141de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:39.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.737+0000 7f13161f5640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f13101dcc10 con 0x7f12f8005190 2024-03-23T14:57:39.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.737+0000 7f13161f5640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f13101e0220 con 0x7f13101dfd90 2024-03-23T14:57:39.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.737+0000 7f13161f5640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f1310143d40 con 0x7f13101435c0 2024-03-23T14:57:39.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.737+0000 7f130f7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f12f8005190 0x7f1310147760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:39.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.737+0000 7f130f7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f12f8005190 0x7f1310147760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:33328/0 (socket says 172.21.15.164:33328) 2024-03-23T14:57:39.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.737+0000 7f130f7fe640 1 -- 172.21.15.164:0/1949774916 learned_addr learned my addr 172.21.15.164:0/1949774916 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:39.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.737+0000 7f130ffff640 1 --2- 172.21.15.164:0/1949774916 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f13101dfd90 0x7f1310141de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:39.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.737+0000 7f130effd640 1 --2- 172.21.15.164:0/1949774916 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f13101435c0 0x7f13101df850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:39.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.737+0000 7f130f7fe640 1 -- 172.21.15.164:0/1949774916 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f13101435c0 msgr2=0x7f13101df850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:39.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.737+0000 7f130f7fe640 1 --2- 172.21.15.164:0/1949774916 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f13101435c0 0x7f13101df850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:39.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.737+0000 7f130f7fe640 1 -- 172.21.15.164:0/1949774916 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f13101dfd90 msgr2=0x7f1310141de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:39.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.737+0000 7f130f7fe640 1 --2- 172.21.15.164:0/1949774916 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f13101dfd90 0x7f1310141de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:39.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.737+0000 7f130f7fe640 1 -- 172.21.15.164:0/1949774916 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13100b3770 con 0x7f12f8005190 2024-03-23T14:57:39.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.737+0000 7f130f7fe640 1 --2- 172.21.15.164:0/1949774916 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f12f8005190 0x7f1310147760 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f13000108c0 tx=0x7f1300010d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:39.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.737+0000 7f12f7fff640 1 -- 172.21.15.164:0/1949774916 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1300009630 con 0x7f12f8005190 2024-03-23T14:57:39.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.737+0000 7f12f7fff640 1 -- 172.21.15.164:0/1949774916 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f130000eb60 con 0x7f12f8005190 2024-03-23T14:57:39.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.737+0000 7f13161f5640 1 -- 172.21.15.164:0/1949774916 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13101dc8a0 con 0x7f12f8005190 2024-03-23T14:57:39.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.737+0000 7f12f7fff640 1 -- 172.21.15.164:0/1949774916 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f130001c400 con 0x7f12f8005190 2024-03-23T14:57:39.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.737+0000 7f13161f5640 1 -- 172.21.15.164:0/1949774916 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1310147ca0 con 0x7f12f8005190 2024-03-23T14:57:39.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.737+0000 7f13161f5640 1 -- 172.21.15.164:0/1949774916 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13100690a0 con 0x7f12f8005190 2024-03-23T14:57:39.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.741+0000 7f12f7fff640 1 -- 172.21.15.164:0/1949774916 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7f130001c560 con 0x7f12f8005190 2024-03-23T14:57:39.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.741+0000 7f12f7fff640 1 --2- 172.21.15.164:0/1949774916 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f12fc0672a0 0x7f12fc069760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:39.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.741+0000 7f12f7fff640 1 -- 172.21.15.164:0/1949774916 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 5227+0+0 (secure 0 0 0) 0x7f1300085630 con 0x7f12f8005190 2024-03-23T14:57:39.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.741+0000 7f130effd640 1 --2- 172.21.15.164:0/1949774916 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f12fc0672a0 0x7f12fc069760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:39.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.741+0000 7f130effd640 1 --2- 172.21.15.164:0/1949774916 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f12fc0672a0 0x7f12fc069760 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f1304000f70 tx=0x7f1304009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:39.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.745+0000 7f12f7fff640 1 -- 172.21.15.164:0/1949774916 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7f13000525e0 con 0x7f12f8005190 2024-03-23T14:57:39.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:39.893+0000 7f13161f5640 1 -- 172.21.15.164:0/1949774916 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f1310148250 con 0x7f12f8005190 2024-03-23T14:57:40.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:40.581+0000 7f12f7fff640 1 -- 172.21.15.164:0/1949774916 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v27) ==== 170+0+0 (secure 0 0 0) 0x7f1300055ca0 con 0x7f12f8005190 2024-03-23T14:57:40.587 INFO:teuthology.orchestra.run.smithi164.stderr:pool 'cephfs_data' created 2024-03-23T14:57:40.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:40.585+0000 7f13161f5640 1 -- 172.21.15.164:0/1949774916 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f12fc0672a0 msgr2=0x7f12fc069760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:40.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:40.585+0000 7f13161f5640 1 --2- 172.21.15.164:0/1949774916 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f12fc0672a0 0x7f12fc069760 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f1304000f70 tx=0x7f1304009040 comp rx=0 tx=0).stop 2024-03-23T14:57:40.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:40.585+0000 7f13161f5640 1 -- 172.21.15.164:0/1949774916 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f12f8005190 msgr2=0x7f1310147760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:40.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:40.585+0000 7f13161f5640 1 --2- 172.21.15.164:0/1949774916 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f12f8005190 0x7f1310147760 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f13000108c0 tx=0x7f1300010d80 comp rx=0 tx=0).stop 2024-03-23T14:57:40.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:40.585+0000 7f13161f5640 1 -- 172.21.15.164:0/1949774916 shutdown_connections 2024-03-23T14:57:40.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:40.585+0000 7f13161f5640 1 --2- 172.21.15.164:0/1949774916 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f12fc0672a0 0x7f12fc069760 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:40.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:40.585+0000 7f13161f5640 1 --2- 172.21.15.164:0/1949774916 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f13101dfd90 0x7f1310141de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:40.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:40.585+0000 7f13161f5640 1 --2- 172.21.15.164:0/1949774916 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f13101435c0 0x7f13101df850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:40.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:40.585+0000 7f13161f5640 1 --2- 172.21.15.164:0/1949774916 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f12f8005190 0x7f1310147760 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:40.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:40.585+0000 7f13161f5640 1 -- 172.21.15.164:0/1949774916 >> 172.21.15.164:0/1949774916 conn(0x7f131002c3c0 msgr2=0x7f131013f280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:40.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:40.585+0000 7f13161f5640 1 -- 172.21.15.164:0/1949774916 shutdown_connections 2024-03-23T14:57:40.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:40.585+0000 7f13161f5640 1 -- 172.21.15.164:0/1949774916 wait complete. 2024-03-23T14:57:40.630 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-23T14:57:40.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:40.745+0000 7f4859eef640 1 Processor -- start 2024-03-23T14:57:40.751 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:40.745+0000 7f4859eef640 1 -- start start 2024-03-23T14:57:40.752 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:40.749+0000 7f4859eef640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f48540b4e90 0x7f485414f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:40.752 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:40.749+0000 7f4859eef640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f485409e2d0 con 0x7f485409def0 2024-03-23T14:57:40.752 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:40.749+0000 7f4859eef640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f485406cb70 con 0x7f48540b4a00 2024-03-23T14:57:40.752 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:40.749+0000 7f4859eef640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f4854146de0 con 0x7f48540b4e90 2024-03-23T14:57:40.752 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:40.749+0000 7f48537fe640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f485409def0 0x7f48540ae620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:58848/0 (socket says 172.21.15.164:58848) 2024-03-23T14:57:40.752 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:40.749+0000 7f48537fe640 1 -- 172.21.15.164:0/3931864062 learned_addr learned my addr 172.21.15.164:0/3931864062 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-03-23T14:57:40.752 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:40.749+0000 7f4853fff640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f48540b4e90 0x7f485414f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:40.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:40.749+0000 7f4853fff640 1 -- 172.21.15.164:0/3931864062 >> v1:172.21.15.173:6789/0 conn(0x7f48540b4a00 legacy=0x7f4854147420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:40.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:40.749+0000 7f4853fff640 1 -- 172.21.15.164:0/3931864062 >> v1:172.21.15.164:6789/0 conn(0x7f485409def0 legacy=0x7f48540ae620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:40.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:40.749+0000 7f48527fc640 1 -- 172.21.15.164:0/3931864062 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3522032831 0 0) 0x7f485409e2d0 con 0x7f485409def0 2024-03-23T14:57:40.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:40.749+0000 7f4853fff640 1 -- 172.21.15.164:0/3931864062 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f485406ce50 con 0x7f48540b4e90 2024-03-23T14:57:40.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:40.749+0000 7f4853fff640 1 --2- 172.21.15.164:0/3931864062 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f48540b4e90 0x7f485414f120 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f484c009d90 tx=0x7f484c01c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=606cf868229eb137 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:40.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:40.749+0000 7f48527fc640 1 -- 172.21.15.164:0/3931864062 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f484c0a8630 con 0x7f48540b4e90 2024-03-23T14:57:40.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:40.749+0000 7f48527fc640 1 -- 172.21.15.164:0/3931864062 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f484c0a8790 con 0x7f48540b4e90 2024-03-23T14:57:40.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:40.749+0000 7f48527fc640 1 -- 172.21.15.164:0/3931864062 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f484c0a8b20 con 0x7f48540b4e90 2024-03-23T14:57:40.756 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:40.749+0000 7f4859eef640 1 -- 172.21.15.164:0/3931864062 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f48540b4e90 msgr2=0x7f485414f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:40.756 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:40.749+0000 7f4859eef640 1 --2- 172.21.15.164:0/3931864062 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f48540b4e90 0x7f485414f120 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f484c009d90 tx=0x7f484c01c810 comp rx=0 tx=0).stop 2024-03-23T14:57:40.756 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:40.753+0000 7f4859eef640 1 -- 172.21.15.164:0/3931864062 shutdown_connections 2024-03-23T14:57:40.756 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:40.753+0000 7f4859eef640 1 --2- 172.21.15.164:0/3931864062 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f48540b4e90 0x7f485414f120 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:40.756 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:40.753+0000 7f4859eef640 1 -- 172.21.15.164:0/3931864062 >> 172.21.15.164:0/3931864062 conn(0x7f48540b8ca0 msgr2=0x7f48540b90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:40.756 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:40.753+0000 7f4859eef640 1 -- 172.21.15.164:0/3931864062 shutdown_connections 2024-03-23T14:57:40.756 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:40.753+0000 7f4859eef640 1 -- 172.21.15.164:0/3931864062 wait complete. 2024-03-23T14:57:40.757 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:40.753+0000 7f4859eef640 1 Processor -- start 2024-03-23T14:57:40.757 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:40.753+0000 7f4859eef640 1 -- start start 2024-03-23T14:57:40.758 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:40.753+0000 7f4859eef640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f485409def0 0x7f4854146560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:40.759 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:40.753+0000 7f4859eef640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f48540b4a00 0x7f4854138780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:40.759 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:40.753+0000 7f4859eef640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f48540b4e90 0x7f4854133c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:40.759 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:40.753+0000 7f4859eef640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f4854135440 con 0x7f48540b4a00 2024-03-23T14:57:40.759 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:40.753+0000 7f4859eef640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f4854131e40 con 0x7f485409def0 2024-03-23T14:57:40.759 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:40.753+0000 7f4859eef640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f4854133500 con 0x7f48540b4e90 2024-03-23T14:57:40.759 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:40.753+0000 7f4852ffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f48540b4a00 0x7f4854138780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:40.759 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:40.753+0000 7f4852ffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f48540b4a00 0x7f4854138780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:33342/0 (socket says 172.21.15.164:33342) 2024-03-23T14:57:40.759 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:40.753+0000 7f4852ffd640 1 -- 172.21.15.164:0/3470638561 learned_addr learned my addr 172.21.15.164:0/3470638561 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:40.759 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:40.753+0000 7f48537fe640 1 --2- 172.21.15.164:0/3470638561 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f485409def0 0x7f4854146560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:40.759 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:40.753+0000 7f4852ffd640 1 -- 172.21.15.164:0/3470638561 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f48540b4e90 msgr2=0x7f4854133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:40.759 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:40.753+0000 7f4852ffd640 1 --2- 172.21.15.164:0/3470638561 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f48540b4e90 0x7f4854133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:40.759 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:40.753+0000 7f4852ffd640 1 -- 172.21.15.164:0/3470638561 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f485409def0 msgr2=0x7f4854146560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:40.759 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:40.753+0000 7f4852ffd640 1 --2- 172.21.15.164:0/3470638561 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f485409def0 0x7f4854146560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:40.759 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:40.753+0000 7f4852ffd640 1 -- 172.21.15.164:0/3470638561 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f484c009a70 con 0x7f48540b4a00 2024-03-23T14:57:40.760 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:40.753+0000 7f4852ffd640 1 --2- 172.21.15.164:0/3470638561 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f48540b4a00 0x7f4854138780 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f484400c450 tx=0x7f484400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:40.760 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:40.753+0000 7f483bfff640 1 -- 172.21.15.164:0/3470638561 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4844012830 con 0x7f48540b4a00 2024-03-23T14:57:40.760 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:40.753+0000 7f483bfff640 1 -- 172.21.15.164:0/3470638561 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48440129b0 con 0x7f48540b4a00 2024-03-23T14:57:40.760 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:40.753+0000 7f4859eef640 1 -- 172.21.15.164:0/3470638561 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4854147d40 con 0x7f48540b4a00 2024-03-23T14:57:40.760 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:40.753+0000 7f483bfff640 1 -- 172.21.15.164:0/3470638561 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4844012d60 con 0x7f48540b4a00 2024-03-23T14:57:40.760 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:40.753+0000 7f4859eef640 1 -- 172.21.15.164:0/3470638561 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48541320e0 con 0x7f48540b4a00 2024-03-23T14:57:40.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:40.753+0000 7f4859eef640 1 -- 172.21.15.164:0/3470638561 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4854132300 con 0x7f48540b4a00 2024-03-23T14:57:40.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:40.757+0000 7f483bfff640 1 -- 172.21.15.164:0/3470638561 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7f484401c460 con 0x7f48540b4a00 2024-03-23T14:57:40.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:40.757+0000 7f483bfff640 1 --2- 172.21.15.164:0/3470638561 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f48400672a0 0x7f4840069760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:40.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:40.757+0000 7f483bfff640 1 -- 172.21.15.164:0/3470638561 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5609+0+0 (secure 0 0 0) 0x7f4844085830 con 0x7f48540b4a00 2024-03-23T14:57:40.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:40.757+0000 7f48537fe640 1 --2- 172.21.15.164:0/3470638561 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f48400672a0 0x7f4840069760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:40.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:40.757+0000 7f48537fe640 1 --2- 172.21.15.164:0/3470638561 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f48400672a0 0x7f4840069760 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f4848000f70 tx=0x7f4848007c10 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:40.764 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:40.761+0000 7f483bfff640 1 -- 172.21.15.164:0/3470638561 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7f48440525b0 con 0x7f48540b4a00 2024-03-23T14:57:40.960 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:40.953+0000 7f4859eef640 1 -- 172.21.15.164:0/3470638561 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f48541356f0 con 0x7f48540b4a00 2024-03-23T14:57:41.616 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:41.605+0000 7f483bfff640 1 -- 172.21.15.164:0/3470638561 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v11) ==== 172+0+0 (secure 0 0 0) 0x7f4844055c70 con 0x7f48540b4a00 2024-03-23T14:57:41.616 INFO:teuthology.orchestra.run.smithi164.stderr:new fs with metadata pool 4 and data pool 5 2024-03-23T14:57:41.629 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:41.625+0000 7f4859eef640 1 -- 172.21.15.164:0/3470638561 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f48400672a0 msgr2=0x7f4840069760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:41.629 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:41.625+0000 7f4859eef640 1 --2- 172.21.15.164:0/3470638561 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f48400672a0 0x7f4840069760 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f4848000f70 tx=0x7f4848007c10 comp rx=0 tx=0).stop 2024-03-23T14:57:41.629 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:41.625+0000 7f4859eef640 1 -- 172.21.15.164:0/3470638561 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f48540b4a00 msgr2=0x7f4854138780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:41.630 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:41.625+0000 7f4859eef640 1 --2- 172.21.15.164:0/3470638561 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f48540b4a00 0x7f4854138780 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f484400c450 tx=0x7f484400c910 comp rx=0 tx=0).stop 2024-03-23T14:57:41.630 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:41.625+0000 7f4859eef640 1 -- 172.21.15.164:0/3470638561 shutdown_connections 2024-03-23T14:57:41.630 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:41.625+0000 7f4859eef640 1 --2- 172.21.15.164:0/3470638561 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f48540b4e90 0x7f4854133c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:41.630 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:41.625+0000 7f4859eef640 1 --2- 172.21.15.164:0/3470638561 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f48540b4a00 0x7f4854138780 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:41.630 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:41.625+0000 7f4859eef640 1 --2- 172.21.15.164:0/3470638561 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f48400672a0 0x7f4840069760 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:41.630 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:41.625+0000 7f4859eef640 1 --2- 172.21.15.164:0/3470638561 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f485409def0 0x7f4854146560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:41.630 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:41.625+0000 7f4859eef640 1 -- 172.21.15.164:0/3470638561 >> 172.21.15.164:0/3470638561 conn(0x7f48540b8ca0 msgr2=0x7f48540b33d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:41.630 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:41.625+0000 7f4859eef640 1 -- 172.21.15.164:0/3470638561 shutdown_connections 2024-03-23T14:57:41.630 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:41.625+0000 7f4859eef640 1 -- 172.21.15.164:0/3470638561 wait complete. 2024-03-23T14:57:41.660 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T14:57:41.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:41.769+0000 7fd0aef72640 1 Processor -- start 2024-03-23T14:57:41.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:41.769+0000 7fd0aef72640 1 -- start start 2024-03-23T14:57:41.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:41.773+0000 7fd0aef72640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fd0a814eb80 0x7fd0a8150f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:41.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:41.773+0000 7fd0aef72640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0a80b51d0 con 0x7fd0a80b0230 2024-03-23T14:57:41.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:41.773+0000 7fd0aef72640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0a80b0610 con 0x7fd0a8148cf0 2024-03-23T14:57:41.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:41.773+0000 7fd0aef72640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 -- 0x7fd0a80b1500 con 0x7fd0a814eb80 2024-03-23T14:57:41.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:41.773+0000 7fd0acd0e640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fd0a80b0230 0x7fd0a80b34d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:58854/0 (socket says 172.21.15.164:58854) 2024-03-23T14:57:41.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:41.773+0000 7fd0acd0e640 1 -- 172.21.15.164:0/3484025684 learned_addr learned my addr 172.21.15.164:0/3484025684 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-03-23T14:57:41.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:41.773+0000 7fd0ad50f640 1 --2- 172.21.15.164:0/3484025684 >> v2:172.21.15.173:3300/0 conn(0x7fd0a814eb80 0x7fd0a8150f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:41.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:41.773+0000 7fd0a77fe640 1 -- 172.21.15.164:0/3484025684 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 72537941 0 0) 0x7fd0a80b51d0 con 0x7fd0a80b0230 2024-03-23T14:57:41.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:41.773+0000 7fd0a77fe640 1 -- 172.21.15.164:0/3484025684 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd094003530 con 0x7fd0a80b0230 2024-03-23T14:57:41.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:41.773+0000 7fd0a77fe640 1 -- 172.21.15.164:0/3484025684 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3033189380 0 0) 0x7fd094003530 con 0x7fd0a80b0230 2024-03-23T14:57:41.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:41.773+0000 7fd0a77fe640 1 -- 172.21.15.164:0/3484025684 >> v2:172.21.15.173:3300/0 conn(0x7fd0a814eb80 msgr2=0x7fd0a8150f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:41.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:41.773+0000 7fd0a77fe640 1 --2- 172.21.15.164:0/3484025684 >> v2:172.21.15.173:3300/0 conn(0x7fd0a814eb80 0x7fd0a8150f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:41.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:41.773+0000 7fd0a77fe640 1 -- 172.21.15.164:0/3484025684 >> v1:172.21.15.173:6789/0 conn(0x7fd0a8148cf0 legacy=0x7fd0a8149180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:41.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:41.773+0000 7fd0a77fe640 1 -- 172.21.15.164:0/3484025684 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0a806d250 con 0x7fd0a80b0230 2024-03-23T14:57:41.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:41.773+0000 7fd0a77fe640 1 -- 172.21.15.164:0/3484025684 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2526390548 0 0) 0x7fd0a0001a00 con 0x7fd0a80b0230 2024-03-23T14:57:41.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:41.773+0000 7fd0a77fe640 1 -- 172.21.15.164:0/3484025684 >> v1:172.21.15.164:6789/0 conn(0x7fd0a80b0230 legacy=0x7fd0a80b34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:41.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:41.773+0000 7fd0a77fe640 1 --2- 172.21.15.164:0/3484025684 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd094003980 0x7fd094003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:41.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:41.773+0000 7fd0a77fe640 1 --2- 172.21.15.164:0/3484025684 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd0940047d0 0x7fd094004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:41.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:41.773+0000 7fd0a77fe640 1 --2- 172.21.15.164:0/3484025684 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd094005390 0x7fd09400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:41.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:41.773+0000 7fd0a77fe640 1 -- 172.21.15.164:0/3484025684 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a80b1500 con 0x7fd094005390 2024-03-23T14:57:41.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:41.773+0000 7fd0a77fe640 1 -- 172.21.15.164:0/3484025684 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd09400c230 con 0x7fd0940047d0 2024-03-23T14:57:41.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:41.773+0000 7fd0a77fe640 1 -- 172.21.15.164:0/3484025684 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fd09400c3f0 con 0x7fd094003980 2024-03-23T14:57:41.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:41.773+0000 7fd0ad50f640 1 --2- 172.21.15.164:0/3484025684 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd094005390 0x7fd09400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:41.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:41.773+0000 7fd0acd0e640 1 --2- 172.21.15.164:0/3484025684 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd094003980 0x7fd094003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:41.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:41.773+0000 7fd0a7fff640 1 --2- 172.21.15.164:0/3484025684 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd0940047d0 0x7fd094004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:41.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:41.773+0000 7fd0ad50f640 1 -- 172.21.15.164:0/3484025684 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd094003980 msgr2=0x7fd094003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:41.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:41.773+0000 7fd0ad50f640 1 --2- 172.21.15.164:0/3484025684 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd094003980 0x7fd094003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:41.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:41.773+0000 7fd0ad50f640 1 -- 172.21.15.164:0/3484025684 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd0940047d0 msgr2=0x7fd094004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:41.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:41.773+0000 7fd0ad50f640 1 --2- 172.21.15.164:0/3484025684 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd0940047d0 0x7fd094004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:41.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:41.773+0000 7fd0ad50f640 1 -- 172.21.15.164:0/3484025684 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd094003530 con 0x7fd094005390 2024-03-23T14:57:41.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:41.773+0000 7fd0acd0e640 1 --2- 172.21.15.164:0/3484025684 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd094003980 0x7fd094003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T14:57:41.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:41.773+0000 7fd0ad50f640 1 -- 172.21.15.164:0/3484025684 reap_dead start 2024-03-23T14:57:41.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:41.773+0000 7fd0ad50f640 1 --2- 172.21.15.164:0/3484025684 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd094005390 0x7fd09400b990 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fd09800ba30 tx=0x7fd09801c6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=681b3cfc08170ce server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:41.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:41.773+0000 7fd0a77fe640 1 -- 172.21.15.164:0/3484025684 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd098004220 con 0x7fd094005390 2024-03-23T14:57:41.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:41.773+0000 7fd0a77fe640 1 -- 172.21.15.164:0/3484025684 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd098004380 con 0x7fd094005390 2024-03-23T14:57:41.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:41.773+0000 7fd0a77fe640 1 -- 172.21.15.164:0/3484025684 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd098003860 con 0x7fd094005390 2024-03-23T14:57:41.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:41.777+0000 7fd0aef72640 1 -- 172.21.15.164:0/3484025684 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd094005390 msgr2=0x7fd09400b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:41.781 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:41.777+0000 7fd0aef72640 1 --2- 172.21.15.164:0/3484025684 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd094005390 0x7fd09400b990 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fd09800ba30 tx=0x7fd09801c6b0 comp rx=0 tx=0).stop 2024-03-23T14:57:41.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:41.777+0000 7fd0aef72640 1 -- 172.21.15.164:0/3484025684 shutdown_connections 2024-03-23T14:57:41.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:41.777+0000 7fd0aef72640 1 --2- 172.21.15.164:0/3484025684 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd094005390 0x7fd09400b990 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:41.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:41.777+0000 7fd0aef72640 1 -- 172.21.15.164:0/3484025684 >> 172.21.15.164:0/3484025684 conn(0x7fd0a80b8ca0 msgr2=0x7fd0a80b90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:41.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:41.777+0000 7fd0aef72640 1 -- 172.21.15.164:0/3484025684 shutdown_connections 2024-03-23T14:57:41.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:41.777+0000 7fd0aef72640 1 -- 172.21.15.164:0/3484025684 wait complete. 2024-03-23T14:57:41.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:41.777+0000 7fd0aef72640 1 Processor -- start 2024-03-23T14:57:41.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:41.777+0000 7fd0aef72640 1 -- start start 2024-03-23T14:57:41.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:41.777+0000 7fd0aef72640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd094005390 0x7fd0a8144da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:41.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:41.777+0000 7fd0aef72640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd0a809c000 0x7fd0a809c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:41.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:41.777+0000 7fd0aef72640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd0a8135800 0x7fd0a8135cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:41.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:41.777+0000 7fd0aef72640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a81e1860 con 0x7fd094005390 2024-03-23T14:57:41.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:41.777+0000 7fd0aef72640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a809b770 con 0x7fd0a8135800 2024-03-23T14:57:41.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:41.777+0000 7fd0aef72640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fd0a809c9c0 con 0x7fd0a809c000 2024-03-23T14:57:41.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:41.777+0000 7fd0acd0e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd094005390 0x7fd0a8144da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:41.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:41.777+0000 7fd0acd0e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd094005390 0x7fd0a8144da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:33352/0 (socket says 172.21.15.164:33352) 2024-03-23T14:57:41.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:41.777+0000 7fd0acd0e640 1 -- 172.21.15.164:0/392844512 learned_addr learned my addr 172.21.15.164:0/392844512 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:41.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:41.777+0000 7fd0a7fff640 1 --2- 172.21.15.164:0/392844512 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd0a809c000 0x7fd0a809c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:41.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:41.777+0000 7fd0ad50f640 1 --2- 172.21.15.164:0/392844512 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd0a8135800 0x7fd0a8135cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:41.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:41.777+0000 7fd0acd0e640 1 -- 172.21.15.164:0/392844512 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd0a809c000 msgr2=0x7fd0a809c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:41.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:41.777+0000 7fd0acd0e640 1 --2- 172.21.15.164:0/392844512 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd0a809c000 0x7fd0a809c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:41.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:41.777+0000 7fd0acd0e640 1 -- 172.21.15.164:0/392844512 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd0a8135800 msgr2=0x7fd0a8135cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:41.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:41.777+0000 7fd0acd0e640 1 --2- 172.21.15.164:0/392844512 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd0a8135800 0x7fd0a8135cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:41.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:41.777+0000 7fd0acd0e640 1 -- 172.21.15.164:0/392844512 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0a809b490 con 0x7fd094005390 2024-03-23T14:57:41.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:41.777+0000 7fd0acd0e640 1 --2- 172.21.15.164:0/392844512 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd094005390 0x7fd0a8144da0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fd0a0002130 tx=0x7fd0a000b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:41.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:41.777+0000 7fd0a57fa640 1 -- 172.21.15.164:0/392844512 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0a0004d70 con 0x7fd094005390 2024-03-23T14:57:41.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:41.777+0000 7fd0a57fa640 1 -- 172.21.15.164:0/392844512 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0a00070d0 con 0x7fd094005390 2024-03-23T14:57:41.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:41.777+0000 7fd0aef72640 1 -- 172.21.15.164:0/392844512 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0a809ba50 con 0x7fd094005390 2024-03-23T14:57:41.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:41.777+0000 7fd0a57fa640 1 -- 172.21.15.164:0/392844512 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0a0007440 con 0x7fd094005390 2024-03-23T14:57:41.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:41.777+0000 7fd0aef72640 1 -- 172.21.15.164:0/392844512 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0a81e1210 con 0x7fd094005390 2024-03-23T14:57:41.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:41.781+0000 7fd0aef72640 1 -- 172.21.15.164:0/392844512 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0a809e390 con 0x7fd094005390 2024-03-23T14:57:41.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:41.781+0000 7fd0a57fa640 1 -- 172.21.15.164:0/392844512 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7fd0a00075a0 con 0x7fd094005390 2024-03-23T14:57:41.786 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:41.781+0000 7fd0a57fa640 1 --2- 172.21.15.164:0/392844512 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fd0880672d0 0x7fd088069790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:41.786 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:41.781+0000 7fd0a7fff640 1 --2- 172.21.15.164:0/392844512 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fd0880672d0 0x7fd088069790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:41.786 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:41.781+0000 7fd0a57fa640 1 -- 172.21.15.164:0/392844512 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 5709+0+0 (secure 0 0 0) 0x7fd0a00853c0 con 0x7fd094005390 2024-03-23T14:57:41.786 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:41.781+0000 7fd0a7fff640 1 --2- 172.21.15.164:0/392844512 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fd0880672d0 0x7fd088069790 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fd0a81457c0 tx=0x7fd090009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:41.789 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:41.785+0000 7fd0a57fa640 1 -- 172.21.15.164:0/392844512 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7fd0a0051f60 con 0x7fd094005390 2024-03-23T14:57:41.949 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:41.945+0000 7fd0aef72640 1 -- 172.21.15.164:0/392844512 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd0a80690a0 con 0x7fd094005390 2024-03-23T14:57:41.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:41.945+0000 7fd0a57fa640 1 -- 172.21.15.164:0/392844512 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) ==== 74+0+16706 (secure 0 0 0) 0x7fd0a0007850 con 0x7fd094005390 2024-03-23T14:57:41.951 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-23T14:57:41.951 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":28,"fsid":"45c7911b-b69d-4e68-907b-fbfeb780aa3b","created":"2024-03-23T14:56:34.367633+0000","modified":"2024-03-23T14:57:40.960951+0000","last_up_change":"2024-03-23T14:56:46.297403+0000","last_in_change":"2024-03-23T14:56:36.629884+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T14:56:47.186157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-03-23T14:57:39.290666+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-03-23T14:57:39.934337+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"80abdc62-bd0d-4e60-a554-7ed6fd67cf54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":984716690},{"type":"v1","addr":"172.21.15.164:6801","nonce":984716690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":984716690},{"type":"v1","addr":"172.21.15.164:6803","nonce":984716690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":984716690},{"type":"v1","addr":"172.21.15.164:6807","nonce":984716690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":984716690},{"type":"v1","addr":"172.21.15.164:6805","nonce":984716690}]},"public_addr":"172.21.15.164:6801/984716690","cluster_addr":"172.21.15.164:6803/984716690","heartbeat_back_addr":"172.21.15.164:6807/984716690","heartbeat_front_addr":"172.21.15.164:6805/984716690","state":["exists","up"]},{"osd":1,"uuid":"f9001b06-be4d-4f42-a0e6-bcef2afa0d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":1349657239},{"type":"v1","addr":"172.21.15.164:6817","nonce":1349657239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":1349657239},{"type":"v1","addr":"172.21.15.164:6819","nonce":1349657239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":1349657239},{"type":"v1","addr":"172.21.15.164:6823","nonce":1349657239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":1349657239},{"type":"v1","addr":"172.21.15.164:6821","nonce":1349657239}]},"public_addr":"172.21.15.164:6817/1349657239","cluster_addr":"172.21.15.164:6819/1349657239","heartbeat_back_addr":"172.21.15.164:6823/1349657239","heartbeat_front_addr":"172.21.15.164:6821/1349657239","state":["exists","up"]},{"osd":2,"uuid":"fa0aeb9f-8b54-4831-a188-3e361c7980d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":887861252},{"type":"v1","addr":"172.21.15.164:6809","nonce":887861252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":887861252},{"type":"v1","addr":"172.21.15.164:6811","nonce":887861252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":887861252},{"type":"v1","addr":"172.21.15.164:6815","nonce":887861252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":887861252},{"type":"v1","addr":"172.21.15.164:6813","nonce":887861252}]},"public_addr":"172.21.15.164:6809/887861252","cluster_addr":"172.21.15.164:6811/887861252","heartbeat_back_addr":"172.21.15.164:6815/887861252","heartbeat_front_addr":"172.21.15.164:6813/887861252","state":["exists","up"]},{"osd":3,"uuid":"7e18c957-3091-46d5-8e17-8f3354c1726c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":812540271},{"type":"v1","addr":"172.21.15.164:6825","nonce":812540271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":812540271},{"type":"v1","addr":"172.21.15.164:6827","nonce":812540271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":812540271},{"type":"v1","addr":"172.21.15.164:6831","nonce":812540271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":812540271},{"type":"v1","addr":"172.21.15.164:6829","nonce":812540271}]},"public_addr":"172.21.15.164:6825/812540271","cluster_addr":"172.21.15.164:6827/812540271","heartbeat_back_addr":"172.21.15.164:6831/812540271","heartbeat_front_addr":"172.21.15.164:6829/812540271","state":["exists","up"]},{"osd":4,"uuid":"8e83f315-42bf-4471-9ce8-d81948da0f9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":2656857421},{"type":"v1","addr":"172.21.15.173:6825","nonce":2656857421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2656857421},{"type":"v1","addr":"172.21.15.173:6827","nonce":2656857421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2656857421},{"type":"v1","addr":"172.21.15.173:6831","nonce":2656857421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":2656857421},{"type":"v1","addr":"172.21.15.173:6829","nonce":2656857421}]},"public_addr":"172.21.15.173:6825/2656857421","cluster_addr":"172.21.15.173:6827/2656857421","heartbeat_back_addr":"172.21.15.173:6831/2656857421","heartbeat_front_addr":"172.21.15.173:6829/2656857421","state":["exists","up"]},{"osd":5,"uuid":"a230c872-71f9-40ab-8999-defd0162d4ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":3676483185},{"type":"v1","addr":"172.21.15.173:6801","nonce":3676483185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":3676483185},{"type":"v1","addr":"172.21.15.173:6803","nonce":3676483185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":3676483185},{"type":"v1","addr":"172.21.15.173:6807","nonce":3676483185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":3676483185},{"type":"v1","addr":"172.21.15.173:6805","nonce":3676483185}]},"public_addr":"172.21.15.173:6801/3676483185","cluster_addr":"172.21.15.173:6803/3676483185","heartbeat_back_addr":"172.21.15.173:6807/3676483185","heartbeat_front_addr":"172.21.15.173:6805/3676483185","state":["exists","up"]},{"osd":6,"uuid":"6193bfd7-8432-4f7f-a117-a3fcee21af99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":3631684087},{"type":"v1","addr":"172.21.15.173:6809","nonce":3631684087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":3631684087},{"type":"v1","addr":"172.21.15.173:6811","nonce":3631684087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":3631684087},{"type":"v1","addr":"172.21.15.173:6815","nonce":3631684087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":3631684087},{"type":"v1","addr":"172.21.15.173:6813","nonce":3631684087}]},"public_addr":"172.21.15.173:6809/3631684087","cluster_addr":"172.21.15.173:6811/3631684087","heartbeat_back_addr":"172.21.15.173:6815/3631684087","heartbeat_front_addr":"172.21.15.173:6813/3631684087","state":["exists","up"]},{"osd":7,"uuid":"b0f1b80a-afc4-4538-83b9-f081d2dcb8cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":3261114948},{"type":"v1","addr":"172.21.15.173:6817","nonce":3261114948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":3261114948},{"type":"v1","addr":"172.21.15.173:6819","nonce":3261114948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":3261114948},{"type":"v1","addr":"172.21.15.173:6823","nonce":3261114948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":3261114948},{"type":"v1","addr":"172.21.15.173:6821","nonce":3261114948}]},"public_addr":"172.21.15.173:6817/3261114948","cluster_addr":"172.21.15.173:6819/3261114948","heartbeat_back_addr":"172.21.15.173:6823/3261114948","heartbeat_front_addr":"172.21.15.173:6821/3261114948","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.188390+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.159932+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.191101+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.181776+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.224365+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.177515+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.153631+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.139972+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T14:57:41.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:41.949+0000 7fd0aef72640 1 -- 172.21.15.164:0/392844512 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fd0880672d0 msgr2=0x7fd088069790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:41.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:41.949+0000 7fd0aef72640 1 --2- 172.21.15.164:0/392844512 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fd0880672d0 0x7fd088069790 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fd0a81457c0 tx=0x7fd090009040 comp rx=0 tx=0).stop 2024-03-23T14:57:41.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:41.949+0000 7fd0aef72640 1 -- 172.21.15.164:0/392844512 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd094005390 msgr2=0x7fd0a8144da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:41.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:41.949+0000 7fd0aef72640 1 --2- 172.21.15.164:0/392844512 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd094005390 0x7fd0a8144da0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fd0a0002130 tx=0x7fd0a000b4a0 comp rx=0 tx=0).stop 2024-03-23T14:57:41.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:41.949+0000 7fd0aef72640 1 -- 172.21.15.164:0/392844512 shutdown_connections 2024-03-23T14:57:41.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:41.949+0000 7fd0aef72640 1 --2- 172.21.15.164:0/392844512 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fd0880672d0 0x7fd088069790 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:41.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:41.949+0000 7fd0aef72640 1 --2- 172.21.15.164:0/392844512 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd0a8135800 0x7fd0a8135cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:41.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:41.949+0000 7fd0aef72640 1 --2- 172.21.15.164:0/392844512 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd0a809c000 0x7fd0a809c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:41.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:41.949+0000 7fd0aef72640 1 --2- 172.21.15.164:0/392844512 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd094005390 0x7fd0a8144da0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:41.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:41.949+0000 7fd0aef72640 1 -- 172.21.15.164:0/392844512 >> 172.21.15.164:0/392844512 conn(0x7fd0a80b8ca0 msgr2=0x7fd0a813f290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:41.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:41.949+0000 7fd0aef72640 1 -- 172.21.15.164:0/392844512 shutdown_connections 2024-03-23T14:57:41.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:41.949+0000 7fd0aef72640 1 -- 172.21.15.164:0/392844512 wait complete. 2024-03-23T14:57:41.970 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T14:57:42.087 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.081+0000 7f5d699c9640 1 Processor -- start 2024-03-23T14:57:42.087 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.081+0000 7f5d699c9640 1 -- start start 2024-03-23T14:57:42.088 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.085+0000 7f5d699c9640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f5d640adc90 0x7f5d640ae070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:42.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.085+0000 7f5d699c9640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f5d6414ad20 0x7f5d6414b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:42.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.085+0000 7f5d699c9640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d6406c9c0 con 0x7f5d6406bf70 2024-03-23T14:57:42.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.085+0000 7f5d699c9640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f5d640ae810 con 0x7f5d640adc90 2024-03-23T14:57:42.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.085+0000 7f5d699c9640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 -- 0x7f5d640b2ce0 con 0x7f5d6414ad20 2024-03-23T14:57:42.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.085+0000 7f5d62ffd640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f5d640adc90 0x7f5d640ae070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:42.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.085+0000 7f5d62ffd640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f5d640adc90 0x7f5d640ae070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:33364/0 (socket says 172.21.15.164:33364) 2024-03-23T14:57:42.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.085+0000 7f5d627fc640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f5d6406bf70 0x7f5d640b4a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:58862/0 (socket says 172.21.15.164:58862) 2024-03-23T14:57:42.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.085+0000 7f5d62ffd640 1 -- 172.21.15.164:0/1917796475 learned_addr learned my addr 172.21.15.164:0/1917796475 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:42.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.085+0000 7f5d637fe640 1 --2- 172.21.15.164:0/1917796475 >> v2:172.21.15.173:3300/0 conn(0x7f5d6414ad20 0x7f5d6414b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:42.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.085+0000 7f5d62ffd640 1 -- 172.21.15.164:0/1917796475 >> v2:172.21.15.173:3300/0 conn(0x7f5d6414ad20 msgr2=0x7f5d6414b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:42.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.085+0000 7f5d62ffd640 1 --2- 172.21.15.164:0/1917796475 >> v2:172.21.15.173:3300/0 conn(0x7f5d6414ad20 0x7f5d6414b190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:42.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.085+0000 7f5d62ffd640 1 -- 172.21.15.164:0/1917796475 >> v1:172.21.15.164:6789/0 conn(0x7f5d6406bf70 legacy=0x7f5d640b4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:42.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.085+0000 7f5d61ffb640 1 -- 172.21.15.164:0/1917796475 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2433768816 0 0) 0x7f5d6406c9c0 con 0x7f5d6406bf70 2024-03-23T14:57:42.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.085+0000 7f5d62ffd640 1 -- 172.21.15.164:0/1917796475 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d640af460 con 0x7f5d640adc90 2024-03-23T14:57:42.091 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.085+0000 7f5d62ffd640 1 --2- 172.21.15.164:0/1917796475 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5d640adc90 0x7f5d640ae070 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f5d640af370 tx=0x7f5d5801c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4c24b512cb6ff04 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:42.091 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.085+0000 7f5d61ffb640 1 -- 172.21.15.164:0/1917796475 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d580ac640 con 0x7f5d640adc90 2024-03-23T14:57:42.091 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.085+0000 7f5d61ffb640 1 -- 172.21.15.164:0/1917796475 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d580ac7a0 con 0x7f5d640adc90 2024-03-23T14:57:42.091 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.085+0000 7f5d61ffb640 1 -- 172.21.15.164:0/1917796475 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d580acb30 con 0x7f5d640adc90 2024-03-23T14:57:42.092 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.089+0000 7f5d699c9640 1 -- 172.21.15.164:0/1917796475 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5d640adc90 msgr2=0x7f5d640ae070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:42.092 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.089+0000 7f5d699c9640 1 --2- 172.21.15.164:0/1917796475 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5d640adc90 0x7f5d640ae070 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f5d640af370 tx=0x7f5d5801c420 comp rx=0 tx=0).stop 2024-03-23T14:57:42.092 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.089+0000 7f5d699c9640 1 -- 172.21.15.164:0/1917796475 shutdown_connections 2024-03-23T14:57:42.092 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.089+0000 7f5d699c9640 1 --2- 172.21.15.164:0/1917796475 >> v2:172.21.15.173:3300/0 conn(0x7f5d6414ad20 0x7f5d6414b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:42.092 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.089+0000 7f5d699c9640 1 --2- 172.21.15.164:0/1917796475 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5d640adc90 0x7f5d640ae070 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:42.092 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.089+0000 7f5d699c9640 1 -- 172.21.15.164:0/1917796475 >> 172.21.15.164:0/1917796475 conn(0x7f5d640b8ca0 msgr2=0x7f5d640b90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:42.092 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.089+0000 7f5d699c9640 1 -- 172.21.15.164:0/1917796475 shutdown_connections 2024-03-23T14:57:42.093 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.089+0000 7f5d699c9640 1 -- 172.21.15.164:0/1917796475 wait complete. 2024-03-23T14:57:42.095 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.089+0000 7f5d699c9640 1 Processor -- start 2024-03-23T14:57:42.095 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.089+0000 7f5d699c9640 1 -- start start 2024-03-23T14:57:42.095 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.089+0000 7f5d699c9640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5d640adc90 0x7f5d64145b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:42.095 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.089+0000 7f5d699c9640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5d6414ad20 0x7f5d641e3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:42.095 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.089+0000 7f5d699c9640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5d641e3c90 0x7f5d641e0150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:42.095 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.089+0000 7f5d699c9640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5d641e32c0 con 0x7f5d6414ad20 2024-03-23T14:57:42.095 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.089+0000 7f5d699c9640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f5d641dfb40 con 0x7f5d641e3c90 2024-03-23T14:57:42.095 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.089+0000 7f5d699c9640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f5d6406fd10 con 0x7f5d640adc90 2024-03-23T14:57:42.095 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.089+0000 7f5d627fc640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5d6414ad20 0x7f5d641e3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:42.095 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.089+0000 7f5d627fc640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5d6414ad20 0x7f5d641e3750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:33370/0 (socket says 172.21.15.164:33370) 2024-03-23T14:57:42.095 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.089+0000 7f5d627fc640 1 -- 172.21.15.164:0/4049431205 learned_addr learned my addr 172.21.15.164:0/4049431205 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:42.095 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.089+0000 7f5d637fe640 1 --2- 172.21.15.164:0/4049431205 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5d641e3c90 0x7f5d641e0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:42.095 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.089+0000 7f5d62ffd640 1 --2- 172.21.15.164:0/4049431205 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5d640adc90 0x7f5d64145b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:42.095 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.089+0000 7f5d627fc640 1 -- 172.21.15.164:0/4049431205 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5d640adc90 msgr2=0x7f5d64145b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:42.096 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.089+0000 7f5d627fc640 1 --2- 172.21.15.164:0/4049431205 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5d640adc90 0x7f5d64145b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:42.096 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.089+0000 7f5d627fc640 1 -- 172.21.15.164:0/4049431205 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5d641e3c90 msgr2=0x7f5d641e0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:42.096 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.089+0000 7f5d627fc640 1 --2- 172.21.15.164:0/4049431205 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5d641e3c90 0x7f5d641e0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:42.096 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.089+0000 7f5d627fc640 1 -- 172.21.15.164:0/4049431205 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d580094d0 con 0x7f5d6414ad20 2024-03-23T14:57:42.096 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.089+0000 7f5d62ffd640 1 --2- 172.21.15.164:0/4049431205 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5d640adc90 0x7f5d64145b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T14:57:42.096 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.089+0000 7f5d627fc640 1 --2- 172.21.15.164:0/4049431205 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5d6414ad20 0x7f5d641e3750 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f5d5400c450 tx=0x7f5d5400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:42.096 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.089+0000 7f5d4bfff640 1 -- 172.21.15.164:0/4049431205 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d540074e0 con 0x7f5d6414ad20 2024-03-23T14:57:42.096 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.089+0000 7f5d4bfff640 1 -- 172.21.15.164:0/4049431205 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d54007660 con 0x7f5d6414ad20 2024-03-23T14:57:42.096 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.089+0000 7f5d699c9640 1 -- 172.21.15.164:0/4049431205 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d64070d10 con 0x7f5d6414ad20 2024-03-23T14:57:42.096 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.089+0000 7f5d4bfff640 1 -- 172.21.15.164:0/4049431205 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d54004a30 con 0x7f5d6414ad20 2024-03-23T14:57:42.096 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.089+0000 7f5d699c9640 1 -- 172.21.15.164:0/4049431205 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d6406fa30 con 0x7f5d6414ad20 2024-03-23T14:57:42.097 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.089+0000 7f5d699c9640 1 -- 172.21.15.164:0/4049431205 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d641e3450 con 0x7f5d6414ad20 2024-03-23T14:57:42.097 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.093+0000 7f5d4bfff640 1 -- 172.21.15.164:0/4049431205 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7f5d54004b90 con 0x7f5d6414ad20 2024-03-23T14:57:42.097 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.093+0000 7f5d4bfff640 1 --2- 172.21.15.164:0/4049431205 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f5d500672a0 0x7f5d50069760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:42.097 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.093+0000 7f5d62ffd640 1 --2- 172.21.15.164:0/4049431205 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f5d500672a0 0x7f5d50069760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:42.097 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.093+0000 7f5d4bfff640 1 -- 172.21.15.164:0/4049431205 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 5709+0+0 (secure 0 0 0) 0x7f5d54084430 con 0x7f5d6414ad20 2024-03-23T14:57:42.097 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.093+0000 7f5d62ffd640 1 --2- 172.21.15.164:0/4049431205 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f5d500672a0 0x7f5d50069760 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5d580ae000 tx=0x7f5d58008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:42.101 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.097+0000 7f5d4bfff640 1 -- 172.21.15.164:0/4049431205 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7f5d54051130 con 0x7f5d6414ad20 2024-03-23T14:57:42.264 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.261+0000 7f5d699c9640 1 -- 172.21.15.164:0/4049431205 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5d641e30b0 con 0x7f5d6414ad20 2024-03-23T14:57:42.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.261+0000 7f5d4bfff640 1 -- 172.21.15.164:0/4049431205 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) ==== 74+0+16706 (secure 0 0 0) 0x7f5d540547f0 con 0x7f5d6414ad20 2024-03-23T14:57:42.266 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-23T14:57:42.267 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":28,"fsid":"45c7911b-b69d-4e68-907b-fbfeb780aa3b","created":"2024-03-23T14:56:34.367633+0000","modified":"2024-03-23T14:57:40.960951+0000","last_up_change":"2024-03-23T14:56:46.297403+0000","last_in_change":"2024-03-23T14:56:36.629884+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T14:56:47.186157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-03-23T14:57:39.290666+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-03-23T14:57:39.934337+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"80abdc62-bd0d-4e60-a554-7ed6fd67cf54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":984716690},{"type":"v1","addr":"172.21.15.164:6801","nonce":984716690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":984716690},{"type":"v1","addr":"172.21.15.164:6803","nonce":984716690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":984716690},{"type":"v1","addr":"172.21.15.164:6807","nonce":984716690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":984716690},{"type":"v1","addr":"172.21.15.164:6805","nonce":984716690}]},"public_addr":"172.21.15.164:6801/984716690","cluster_addr":"172.21.15.164:6803/984716690","heartbeat_back_addr":"172.21.15.164:6807/984716690","heartbeat_front_addr":"172.21.15.164:6805/984716690","state":["exists","up"]},{"osd":1,"uuid":"f9001b06-be4d-4f42-a0e6-bcef2afa0d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":1349657239},{"type":"v1","addr":"172.21.15.164:6817","nonce":1349657239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":1349657239},{"type":"v1","addr":"172.21.15.164:6819","nonce":1349657239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":1349657239},{"type":"v1","addr":"172.21.15.164:6823","nonce":1349657239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":1349657239},{"type":"v1","addr":"172.21.15.164:6821","nonce":1349657239}]},"public_addr":"172.21.15.164:6817/1349657239","cluster_addr":"172.21.15.164:6819/1349657239","heartbeat_back_addr":"172.21.15.164:6823/1349657239","heartbeat_front_addr":"172.21.15.164:6821/1349657239","state":["exists","up"]},{"osd":2,"uuid":"fa0aeb9f-8b54-4831-a188-3e361c7980d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":887861252},{"type":"v1","addr":"172.21.15.164:6809","nonce":887861252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":887861252},{"type":"v1","addr":"172.21.15.164:6811","nonce":887861252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":887861252},{"type":"v1","addr":"172.21.15.164:6815","nonce":887861252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":887861252},{"type":"v1","addr":"172.21.15.164:6813","nonce":887861252}]},"public_addr":"172.21.15.164:6809/887861252","cluster_addr":"172.21.15.164:6811/887861252","heartbeat_back_addr":"172.21.15.164:6815/887861252","heartbeat_front_addr":"172.21.15.164:6813/887861252","state":["exists","up"]},{"osd":3,"uuid":"7e18c957-3091-46d5-8e17-8f3354c1726c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":812540271},{"type":"v1","addr":"172.21.15.164:6825","nonce":812540271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":812540271},{"type":"v1","addr":"172.21.15.164:6827","nonce":812540271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":812540271},{"type":"v1","addr":"172.21.15.164:6831","nonce":812540271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":812540271},{"type":"v1","addr":"172.21.15.164:6829","nonce":812540271}]},"public_addr":"172.21.15.164:6825/812540271","cluster_addr":"172.21.15.164:6827/812540271","heartbeat_back_addr":"172.21.15.164:6831/812540271","heartbeat_front_addr":"172.21.15.164:6829/812540271","state":["exists","up"]},{"osd":4,"uuid":"8e83f315-42bf-4471-9ce8-d81948da0f9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":2656857421},{"type":"v1","addr":"172.21.15.173:6825","nonce":2656857421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2656857421},{"type":"v1","addr":"172.21.15.173:6827","nonce":2656857421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2656857421},{"type":"v1","addr":"172.21.15.173:6831","nonce":2656857421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":2656857421},{"type":"v1","addr":"172.21.15.173:6829","nonce":2656857421}]},"public_addr":"172.21.15.173:6825/2656857421","cluster_addr":"172.21.15.173:6827/2656857421","heartbeat_back_addr":"172.21.15.173:6831/2656857421","heartbeat_front_addr":"172.21.15.173:6829/2656857421","state":["exists","up"]},{"osd":5,"uuid":"a230c872-71f9-40ab-8999-defd0162d4ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":3676483185},{"type":"v1","addr":"172.21.15.173:6801","nonce":3676483185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":3676483185},{"type":"v1","addr":"172.21.15.173:6803","nonce":3676483185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":3676483185},{"type":"v1","addr":"172.21.15.173:6807","nonce":3676483185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":3676483185},{"type":"v1","addr":"172.21.15.173:6805","nonce":3676483185}]},"public_addr":"172.21.15.173:6801/3676483185","cluster_addr":"172.21.15.173:6803/3676483185","heartbeat_back_addr":"172.21.15.173:6807/3676483185","heartbeat_front_addr":"172.21.15.173:6805/3676483185","state":["exists","up"]},{"osd":6,"uuid":"6193bfd7-8432-4f7f-a117-a3fcee21af99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":3631684087},{"type":"v1","addr":"172.21.15.173:6809","nonce":3631684087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":3631684087},{"type":"v1","addr":"172.21.15.173:6811","nonce":3631684087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":3631684087},{"type":"v1","addr":"172.21.15.173:6815","nonce":3631684087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":3631684087},{"type":"v1","addr":"172.21.15.173:6813","nonce":3631684087}]},"public_addr":"172.21.15.173:6809/3631684087","cluster_addr":"172.21.15.173:6811/3631684087","heartbeat_back_addr":"172.21.15.173:6815/3631684087","heartbeat_front_addr":"172.21.15.173:6813/3631684087","state":["exists","up"]},{"osd":7,"uuid":"b0f1b80a-afc4-4538-83b9-f081d2dcb8cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":3261114948},{"type":"v1","addr":"172.21.15.173:6817","nonce":3261114948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":3261114948},{"type":"v1","addr":"172.21.15.173:6819","nonce":3261114948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":3261114948},{"type":"v1","addr":"172.21.15.173:6823","nonce":3261114948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":3261114948},{"type":"v1","addr":"172.21.15.173:6821","nonce":3261114948}]},"public_addr":"172.21.15.173:6817/3261114948","cluster_addr":"172.21.15.173:6819/3261114948","heartbeat_back_addr":"172.21.15.173:6823/3261114948","heartbeat_front_addr":"172.21.15.173:6821/3261114948","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.188390+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.159932+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.191101+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.181776+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.224365+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.177515+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.153631+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.139972+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T14:57:42.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.265+0000 7f5d699c9640 1 -- 172.21.15.164:0/4049431205 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f5d500672a0 msgr2=0x7f5d50069760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:42.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.265+0000 7f5d699c9640 1 --2- 172.21.15.164:0/4049431205 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f5d500672a0 0x7f5d50069760 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5d580ae000 tx=0x7f5d58008040 comp rx=0 tx=0).stop 2024-03-23T14:57:42.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.265+0000 7f5d699c9640 1 -- 172.21.15.164:0/4049431205 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5d6414ad20 msgr2=0x7f5d641e3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:42.269 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.265+0000 7f5d699c9640 1 --2- 172.21.15.164:0/4049431205 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5d6414ad20 0x7f5d641e3750 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f5d5400c450 tx=0x7f5d5400c910 comp rx=0 tx=0).stop 2024-03-23T14:57:42.269 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.265+0000 7f5d699c9640 1 -- 172.21.15.164:0/4049431205 shutdown_connections 2024-03-23T14:57:42.269 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.265+0000 7f5d699c9640 1 --2- 172.21.15.164:0/4049431205 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f5d500672a0 0x7f5d50069760 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:42.269 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.265+0000 7f5d699c9640 1 --2- 172.21.15.164:0/4049431205 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5d641e3c90 0x7f5d641e0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:42.269 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.265+0000 7f5d699c9640 1 --2- 172.21.15.164:0/4049431205 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5d6414ad20 0x7f5d641e3750 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:42.269 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.265+0000 7f5d699c9640 1 --2- 172.21.15.164:0/4049431205 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5d640adc90 0x7f5d64145b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:42.269 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.265+0000 7f5d699c9640 1 -- 172.21.15.164:0/4049431205 >> 172.21.15.164:0/4049431205 conn(0x7f5d640b8ca0 msgr2=0x7f5d640b5cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:42.269 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.265+0000 7f5d699c9640 1 -- 172.21.15.164:0/4049431205 shutdown_connections 2024-03-23T14:57:42.269 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.265+0000 7f5d699c9640 1 -- 172.21.15.164:0/4049431205 wait complete. 2024-03-23T14:57:42.285 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-23T14:57:42.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.397+0000 7f6c6d068640 1 Processor -- start 2024-03-23T14:57:42.403 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.397+0000 7f6c6d068640 1 -- start start 2024-03-23T14:57:42.404 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.401+0000 7f6c6d068640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6c680725b0 0x7f6c68148890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:42.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.401+0000 7f6c6d068640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f6c6806e730 0x7f6c680b3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:42.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.401+0000 7f6c6d068640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c680b0ca0 con 0x7f6c6806d770 2024-03-23T14:57:42.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.401+0000 7f6c6d068640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 -- 0x7f6c68072990 con 0x7f6c6806e730 2024-03-23T14:57:42.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.401+0000 7f6c6d068640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f6c680b95f0 con 0x7f6c680725b0 2024-03-23T14:57:42.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.401+0000 7f6c66d9d640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6c680725b0 0x7f6c68148890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:42.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.401+0000 7f6c6659c640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f6c6806e730 0x7f6c680b3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:42.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.401+0000 7f6c66d9d640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6c680725b0 0x7f6c68148890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.164:56972/0 (socket says 172.21.15.164:56972) 2024-03-23T14:57:42.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.401+0000 7f6c66d9d640 1 -- 172.21.15.164:0/2475658514 learned_addr learned my addr 172.21.15.164:0/2475658514 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:42.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.401+0000 7f6c6659c640 1 -- 172.21.15.164:0/2475658514 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6c680725b0 msgr2=0x7f6c68148890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:42.405 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.401+0000 7f6c6659c640 1 --2- 172.21.15.164:0/2475658514 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6c680725b0 0x7f6c68148890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:42.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.401+0000 7f6c6659c640 1 -- 172.21.15.164:0/2475658514 >> v1:172.21.15.173:6789/0 conn(0x7f6c6806d770 legacy=0x7f6c6802bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:42.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.401+0000 7f6c6659c640 1 -- 172.21.15.164:0/2475658514 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c68148dd0 con 0x7f6c6806e730 2024-03-23T14:57:42.406 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.401+0000 7f6c66d9d640 1 --2- 172.21.15.164:0/2475658514 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6c680725b0 0x7f6c68148890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T14:57:42.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.401+0000 7f6c6659c640 1 --2- 172.21.15.164:0/2475658514 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6c6806e730 0x7f6c680b3280 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f6c5c0097c0 tx=0x7f6c5c01c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e162384a693c45f server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:42.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.405+0000 7f6c65d9b640 1 -- 172.21.15.164:0/2475658514 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c5c0ab600 con 0x7f6c6806e730 2024-03-23T14:57:42.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.405+0000 7f6c65d9b640 1 -- 172.21.15.164:0/2475658514 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c5c0a7050 con 0x7f6c6806e730 2024-03-23T14:57:42.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.405+0000 7f6c65d9b640 1 -- 172.21.15.164:0/2475658514 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c5c0ab980 con 0x7f6c6806e730 2024-03-23T14:57:42.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.405+0000 7f6c6d068640 1 -- 172.21.15.164:0/2475658514 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6c6806e730 msgr2=0x7f6c680b3280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:42.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.405+0000 7f6c6d068640 1 --2- 172.21.15.164:0/2475658514 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6c6806e730 0x7f6c680b3280 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f6c5c0097c0 tx=0x7f6c5c01c410 comp rx=0 tx=0).stop 2024-03-23T14:57:42.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.405+0000 7f6c6d068640 1 -- 172.21.15.164:0/2475658514 shutdown_connections 2024-03-23T14:57:42.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.405+0000 7f6c6d068640 1 --2- 172.21.15.164:0/2475658514 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6c6806e730 0x7f6c680b3280 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:42.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.405+0000 7f6c6d068640 1 --2- 172.21.15.164:0/2475658514 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6c680725b0 0x7f6c68148890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:42.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.405+0000 7f6c6d068640 1 -- 172.21.15.164:0/2475658514 >> 172.21.15.164:0/2475658514 conn(0x7f6c680b7970 msgr2=0x7f6c680b7d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:42.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.405+0000 7f6c6d068640 1 -- 172.21.15.164:0/2475658514 shutdown_connections 2024-03-23T14:57:42.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.405+0000 7f6c6d068640 1 -- 172.21.15.164:0/2475658514 wait complete. 2024-03-23T14:57:42.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.405+0000 7f6c6d068640 1 Processor -- start 2024-03-23T14:57:42.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.405+0000 7f6c6d068640 1 -- start start 2024-03-23T14:57:42.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.405+0000 7f6c6d068640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6c6806d770 0x7f6c681420c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:42.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.405+0000 7f6c6d068640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6c6806e730 0x7f6c6813c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:42.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.405+0000 7f6c6d068640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6c680725b0 0x7f6c681e0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:42.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.405+0000 7f6c6d068640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6c681e0b80 con 0x7f6c680725b0 2024-03-23T14:57:42.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.405+0000 7f6c6d068640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f6c6813f000 con 0x7f6c6806d770 2024-03-23T14:57:42.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.405+0000 7f6c6d068640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f6c6813ca30 con 0x7f6c6806e730 2024-03-23T14:57:42.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.405+0000 7f6c6759e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6c680725b0 0x7f6c681e0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:42.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.405+0000 7f6c6759e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6c680725b0 0x7f6c681e0fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:33378/0 (socket says 172.21.15.164:33378) 2024-03-23T14:57:42.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.405+0000 7f6c6759e640 1 -- 172.21.15.164:0/104331880 learned_addr learned my addr 172.21.15.164:0/104331880 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:42.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.405+0000 7f6c6659c640 1 --2- 172.21.15.164:0/104331880 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6c6806e730 0x7f6c6813c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:42.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.405+0000 7f6c66d9d640 1 --2- 172.21.15.164:0/104331880 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6c6806d770 0x7f6c681420c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:42.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.405+0000 7f6c6759e640 1 -- 172.21.15.164:0/104331880 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6c6806e730 msgr2=0x7f6c6813c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:42.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.405+0000 7f6c6759e640 1 --2- 172.21.15.164:0/104331880 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6c6806e730 0x7f6c6813c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:42.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.405+0000 7f6c6759e640 1 -- 172.21.15.164:0/104331880 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6c6806d770 msgr2=0x7f6c681420c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:42.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.405+0000 7f6c6759e640 1 --2- 172.21.15.164:0/104331880 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6c6806d770 0x7f6c681420c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:42.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.405+0000 7f6c6759e640 1 -- 172.21.15.164:0/104331880 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c580094d0 con 0x7f6c680725b0 2024-03-23T14:57:42.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.405+0000 7f6c66d9d640 1 --2- 172.21.15.164:0/104331880 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6c6806d770 0x7f6c681420c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T14:57:42.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.409+0000 7f6c6759e640 1 --2- 172.21.15.164:0/104331880 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6c680725b0 0x7f6c681e0fd0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f6c6000c730 tx=0x7f6c6000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:42.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.409+0000 7f6c4f7fe640 1 -- 172.21.15.164:0/104331880 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c60003b80 con 0x7f6c680725b0 2024-03-23T14:57:42.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.409+0000 7f6c4f7fe640 1 -- 172.21.15.164:0/104331880 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c60003ce0 con 0x7f6c680725b0 2024-03-23T14:57:42.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.409+0000 7f6c6d068640 1 -- 172.21.15.164:0/104331880 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c5c009420 con 0x7f6c680725b0 2024-03-23T14:57:42.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.409+0000 7f6c4f7fe640 1 -- 172.21.15.164:0/104331880 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c6000aa30 con 0x7f6c680725b0 2024-03-23T14:57:42.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.409+0000 7f6c6d068640 1 -- 172.21.15.164:0/104331880 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c680b0740 con 0x7f6c680725b0 2024-03-23T14:57:42.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.409+0000 7f6c6d068640 1 -- 172.21.15.164:0/104331880 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c681e0cf0 con 0x7f6c680725b0 2024-03-23T14:57:42.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.409+0000 7f6c4f7fe640 1 -- 172.21.15.164:0/104331880 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7f6c6000ab90 con 0x7f6c680725b0 2024-03-23T14:57:42.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.409+0000 7f6c4f7fe640 1 --2- 172.21.15.164:0/104331880 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f6c540672a0 0x7f6c54069760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:42.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.409+0000 7f6c4f7fe640 1 -- 172.21.15.164:0/104331880 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 5709+0+0 (secure 0 0 0) 0x7f6c60084b00 con 0x7f6c680725b0 2024-03-23T14:57:42.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.409+0000 7f6c66d9d640 1 --2- 172.21.15.164:0/104331880 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f6c540672a0 0x7f6c54069760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:42.415 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.409+0000 7f6c66d9d640 1 --2- 172.21.15.164:0/104331880 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f6c540672a0 0x7f6c54069760 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f6c58009d30 tx=0x7f6c58008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:42.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.413+0000 7f6c4f7fe640 1 -- 172.21.15.164:0/104331880 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7f6c681e0cf0 con 0x7f6c680725b0 2024-03-23T14:57:42.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.593+0000 7f6c6d068640 1 -- 172.21.15.164:0/104331880 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f6c6813b690 con 0x7f6c680725b0 2024-03-23T14:57:42.622 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.617+0000 7f6c4f7fe640 1 -- 172.21.15.164:0/104331880 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v12) ==== 118+0+0 (secure 0 0 0) 0x7f6c60051740 con 0x7f6c680725b0 2024-03-23T14:57:42.624 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.621+0000 7f6c6d068640 1 -- 172.21.15.164:0/104331880 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f6c540672a0 msgr2=0x7f6c54069760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:42.624 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.621+0000 7f6c6d068640 1 --2- 172.21.15.164:0/104331880 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f6c540672a0 0x7f6c54069760 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f6c58009d30 tx=0x7f6c58008040 comp rx=0 tx=0).stop 2024-03-23T14:57:42.624 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.621+0000 7f6c6d068640 1 -- 172.21.15.164:0/104331880 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6c680725b0 msgr2=0x7f6c681e0fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:42.624 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.621+0000 7f6c6d068640 1 --2- 172.21.15.164:0/104331880 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6c680725b0 0x7f6c681e0fd0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f6c6000c730 tx=0x7f6c6000cbf0 comp rx=0 tx=0).stop 2024-03-23T14:57:42.624 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.621+0000 7f6c6d068640 1 -- 172.21.15.164:0/104331880 shutdown_connections 2024-03-23T14:57:42.624 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.621+0000 7f6c6d068640 1 --2- 172.21.15.164:0/104331880 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6c680725b0 0x7f6c681e0fd0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:42.624 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.621+0000 7f6c6d068640 1 --2- 172.21.15.164:0/104331880 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6c6806e730 0x7f6c6813c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:42.625 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.621+0000 7f6c6d068640 1 --2- 172.21.15.164:0/104331880 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f6c540672a0 0x7f6c54069760 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:42.625 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.621+0000 7f6c6d068640 1 --2- 172.21.15.164:0/104331880 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6c6806d770 0x7f6c681420c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:42.625 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.621+0000 7f6c6d068640 1 -- 172.21.15.164:0/104331880 >> 172.21.15.164:0/104331880 conn(0x7f6c680b7970 msgr2=0x7f6c68145ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:42.625 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.621+0000 7f6c6d068640 1 -- 172.21.15.164:0/104331880 shutdown_connections 2024-03-23T14:57:42.625 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.621+0000 7f6c6d068640 1 -- 172.21.15.164:0/104331880 wait complete. 2024-03-23T14:57:42.647 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-03-23T14:57:42.647 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-03-23T14:57:42.764 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.761+0000 7fc4bab4e640 1 Processor -- start 2024-03-23T14:57:42.764 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.761+0000 7fc4bab4e640 1 -- start start 2024-03-23T14:57:42.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.761+0000 7fc4bab4e640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc4b40b9ac0 0x7fc4b413f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:42.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.761+0000 7fc4bab4e640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4b4145240 con 0x7fc4b406f8e0 2024-03-23T14:57:42.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.761+0000 7fc4bab4e640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4b4070050 con 0x7fc4b414db90 2024-03-23T14:57:42.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.761+0000 7fc4bab4e640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fc4b40b3b50 con 0x7fc4b40b9ac0 2024-03-23T14:57:42.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.761+0000 7fc4b3fff640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fc4b406f8e0 0x7fc4b41459b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:46750/0 (socket says 172.21.15.164:46750) 2024-03-23T14:57:42.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.761+0000 7fc4b3fff640 1 -- 172.21.15.164:0/1277188168 learned_addr learned my addr 172.21.15.164:0/1277188168 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-03-23T14:57:42.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.761+0000 7fc4b88ea640 1 --2- 172.21.15.164:0/1277188168 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc4b40b9ac0 0x7fc4b413f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:42.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.761+0000 7fc4b37fe640 1 -- 172.21.15.164:0/1277188168 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 439076374 0 0) 0x7fc4b4145240 con 0x7fc4b406f8e0 2024-03-23T14:57:42.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.761+0000 7fc4b37fe640 1 -- 172.21.15.164:0/1277188168 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc4a4003530 con 0x7fc4b406f8e0 2024-03-23T14:57:42.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.761+0000 7fc4b37fe640 1 -- 172.21.15.164:0/1277188168 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4029616357 0 0) 0x7fc4a4003530 con 0x7fc4b406f8e0 2024-03-23T14:57:42.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.761+0000 7fc4b37fe640 1 -- 172.21.15.164:0/1277188168 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc4b40b9ac0 msgr2=0x7fc4b413f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:42.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.761+0000 7fc4b37fe640 1 --2- 172.21.15.164:0/1277188168 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc4b40b9ac0 0x7fc4b413f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:42.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.761+0000 7fc4b37fe640 1 -- 172.21.15.164:0/1277188168 >> v1:172.21.15.173:6789/0 conn(0x7fc4b414db90 legacy=0x7fc4b414ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:42.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.761+0000 7fc4b37fe640 1 -- 172.21.15.164:0/1277188168 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4b40b9ea0 con 0x7fc4b406f8e0 2024-03-23T14:57:42.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.761+0000 7fc4b37fe640 1 -- 172.21.15.164:0/1277188168 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2526390548 0 0) 0x7fc4a0001680 con 0x7fc4b406f8e0 2024-03-23T14:57:42.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.761+0000 7fc4b37fe640 1 -- 172.21.15.164:0/1277188168 >> v1:172.21.15.164:6789/0 conn(0x7fc4b406f8e0 legacy=0x7fc4b41459b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:42.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.761+0000 7fc4b37fe640 1 --2- 172.21.15.164:0/1277188168 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc4a40039b0 0x7fc4a4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:42.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.761+0000 7fc4b37fe640 1 --2- 172.21.15.164:0/1277188168 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc4a40047d0 0x7fc4a4004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:42.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.761+0000 7fc4b37fe640 1 --2- 172.21.15.164:0/1277188168 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc4b40b9ac0 0x7fc4a4005260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:42.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.761+0000 7fc4b37fe640 1 -- 172.21.15.164:0/1277188168 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a400bb20 con 0x7fc4a40047d0 2024-03-23T14:57:42.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.761+0000 7fc4b37fe640 1 -- 172.21.15.164:0/1277188168 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a400bc90 con 0x7fc4a40039b0 2024-03-23T14:57:42.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.761+0000 7fc4b37fe640 1 -- 172.21.15.164:0/1277188168 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fc4a400be50 con 0x7fc4b40b9ac0 2024-03-23T14:57:42.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.761+0000 7fc4b3fff640 1 --2- 172.21.15.164:0/1277188168 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc4a40047d0 0x7fc4a4004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:42.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.761+0000 7fc4b88ea640 1 --2- 172.21.15.164:0/1277188168 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc4a40039b0 0x7fc4a4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:42.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.761+0000 7fc4b90eb640 1 --2- 172.21.15.164:0/1277188168 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc4b40b9ac0 0x7fc4a4005260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:42.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.761+0000 7fc4b3fff640 1 -- 172.21.15.164:0/1277188168 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc4b40b9ac0 msgr2=0x7fc4a4005260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:42.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.761+0000 7fc4b3fff640 1 --2- 172.21.15.164:0/1277188168 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc4b40b9ac0 0x7fc4a4005260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:42.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.761+0000 7fc4b3fff640 1 -- 172.21.15.164:0/1277188168 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc4a40039b0 msgr2=0x7fc4a4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:42.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.761+0000 7fc4b3fff640 1 --2- 172.21.15.164:0/1277188168 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc4a40039b0 0x7fc4a4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:42.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.761+0000 7fc4b3fff640 1 -- 172.21.15.164:0/1277188168 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4a4003530 con 0x7fc4a40047d0 2024-03-23T14:57:42.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.761+0000 7fc4b90eb640 1 --2- 172.21.15.164:0/1277188168 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc4b40b9ac0 0x7fc4a4005260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T14:57:42.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.765+0000 7fc4b3fff640 1 --2- 172.21.15.164:0/1277188168 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc4a40047d0 0x7fc4a4004c50 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fc4a000c240 tx=0x7fc4a001cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1046bb56e64ffe7f server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:42.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.765+0000 7fc4b37fe640 1 -- 172.21.15.164:0/1277188168 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4a0005ae0 con 0x7fc4a40047d0 2024-03-23T14:57:42.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.765+0000 7fc4b37fe640 1 -- 172.21.15.164:0/1277188168 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4a0005c60 con 0x7fc4a40047d0 2024-03-23T14:57:42.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.765+0000 7fc4b37fe640 1 -- 172.21.15.164:0/1277188168 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4a00a5830 con 0x7fc4a40047d0 2024-03-23T14:57:42.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.765+0000 7fc4bab4e640 1 -- 172.21.15.164:0/1277188168 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc4a40047d0 msgr2=0x7fc4a4004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:42.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.765+0000 7fc4bab4e640 1 --2- 172.21.15.164:0/1277188168 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc4a40047d0 0x7fc4a4004c50 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fc4a000c240 tx=0x7fc4a001cc60 comp rx=0 tx=0).stop 2024-03-23T14:57:42.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.765+0000 7fc4b90eb640 1 -- 172.21.15.164:0/1277188168 reap_dead start 2024-03-23T14:57:42.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.765+0000 7fc4bab4e640 1 -- 172.21.15.164:0/1277188168 shutdown_connections 2024-03-23T14:57:42.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.765+0000 7fc4bab4e640 1 -- 172.21.15.164:0/1277188168 >> 172.21.15.164:0/1277188168 conn(0x7fc4b406b930 msgr2=0x7fc4b40b9150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:42.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.765+0000 7fc4bab4e640 1 -- 172.21.15.164:0/1277188168 shutdown_connections 2024-03-23T14:57:42.770 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.765+0000 7fc4bab4e640 1 -- 172.21.15.164:0/1277188168 wait complete. 2024-03-23T14:57:42.772 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.765+0000 7fc4bab4e640 1 Processor -- start 2024-03-23T14:57:42.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.765+0000 7fc4bab4e640 1 -- start start 2024-03-23T14:57:42.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.765+0000 7fc4bab4e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc4a40047d0 0x7fc4b4142710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:42.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.765+0000 7fc4bab4e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc4b40725f0 0x7fc4b4144420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:42.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.765+0000 7fc4bab4e640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc4b41e09e0 0x7fc4b41e0e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:42.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.765+0000 7fc4bab4e640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc4b41416e0 con 0x7fc4a40047d0 2024-03-23T14:57:42.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.765+0000 7fc4bab4e640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fc4b413fe90 con 0x7fc4b40725f0 2024-03-23T14:57:42.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.765+0000 7fc4bab4e640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fc4b4072d50 con 0x7fc4b41e09e0 2024-03-23T14:57:42.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.765+0000 7fc4b88ea640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc4a40047d0 0x7fc4b4142710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:42.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.765+0000 7fc4b88ea640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc4a40047d0 0x7fc4b4142710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:44682/0 (socket says 172.21.15.164:44682) 2024-03-23T14:57:42.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.765+0000 7fc4b88ea640 1 -- 172.21.15.164:0/3022148625 learned_addr learned my addr 172.21.15.164:0/3022148625 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:42.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.765+0000 7fc4b3fff640 1 --2- 172.21.15.164:0/3022148625 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc4b40725f0 0x7fc4b4144420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:42.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.765+0000 7fc4b90eb640 1 --2- 172.21.15.164:0/3022148625 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc4b41e09e0 0x7fc4b41e0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:42.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.765+0000 7fc4b88ea640 1 -- 172.21.15.164:0/3022148625 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc4b41e09e0 msgr2=0x7fc4b41e0e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:42.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.765+0000 7fc4b88ea640 1 --2- 172.21.15.164:0/3022148625 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc4b41e09e0 0x7fc4b41e0e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:42.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.765+0000 7fc4b88ea640 1 -- 172.21.15.164:0/3022148625 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc4b40725f0 msgr2=0x7fc4b4144420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:42.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.765+0000 7fc4b88ea640 1 --2- 172.21.15.164:0/3022148625 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc4b40725f0 0x7fc4b4144420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:42.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.765+0000 7fc4b88ea640 1 -- 172.21.15.164:0/3022148625 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4b4141f10 con 0x7fc4a40047d0 2024-03-23T14:57:42.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.765+0000 7fc4b88ea640 1 --2- 172.21.15.164:0/3022148625 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc4a40047d0 0x7fc4b4142710 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fc4a80115e0 tx=0x7fc4a8011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:42.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.769+0000 7fc4b17fa640 1 -- 172.21.15.164:0/3022148625 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4a800f8f0 con 0x7fc4a40047d0 2024-03-23T14:57:42.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.769+0000 7fc4b17fa640 1 -- 172.21.15.164:0/3022148625 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4a800fa50 con 0x7fc4a40047d0 2024-03-23T14:57:42.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.769+0000 7fc4bab4e640 1 -- 172.21.15.164:0/3022148625 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4b41421f0 con 0x7fc4a40047d0 2024-03-23T14:57:42.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.769+0000 7fc4b17fa640 1 -- 172.21.15.164:0/3022148625 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4a800fdc0 con 0x7fc4a40047d0 2024-03-23T14:57:42.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.769+0000 7fc4bab4e640 1 -- 172.21.15.164:0/3022148625 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4b413fb30 con 0x7fc4a40047d0 2024-03-23T14:57:42.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.769+0000 7fc4bab4e640 1 -- 172.21.15.164:0/3022148625 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4b4069080 con 0x7fc4a40047d0 2024-03-23T14:57:42.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.769+0000 7fc4b17fa640 1 -- 172.21.15.164:0/3022148625 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7fc4a8015460 con 0x7fc4a40047d0 2024-03-23T14:57:42.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.769+0000 7fc4b17fa640 1 --2- 172.21.15.164:0/3022148625 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fc4980672a0 0x7fc498069760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:42.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.769+0000 7fc4b17fa640 1 -- 172.21.15.164:0/3022148625 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5709+0+0 (secure 0 0 0) 0x7fc4a80858d0 con 0x7fc4a40047d0 2024-03-23T14:57:42.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.769+0000 7fc4b3fff640 1 --2- 172.21.15.164:0/3022148625 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fc4980672a0 0x7fc498069760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:42.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.769+0000 7fc4b3fff640 1 --2- 172.21.15.164:0/3022148625 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fc4980672a0 0x7fc498069760 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc4a00ab000 tx=0x7fc4a00a7070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:42.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:42.773+0000 7fc4b17fa640 1 -- 172.21.15.164:0/3022148625 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7fc4a80524f0 con 0x7fc4a40047d0 2024-03-23T14:57:43.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:43.041+0000 7fc4bab4e640 1 -- 172.21.15.164:0/3022148625 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fc4b413cf50 con 0x7fc4a40047d0 2024-03-23T14:57:43.634 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:43.629+0000 7fc4b17fa640 1 -- 172.21.15.164:0/3022148625 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v13) ==== 122+0+0 (secure 0 0 0) 0x7fc4a8055bb0 con 0x7fc4a40047d0 2024-03-23T14:57:43.637 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:43.633+0000 7fc4bab4e640 1 -- 172.21.15.164:0/3022148625 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fc4980672a0 msgr2=0x7fc498069760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:43.637 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:43.633+0000 7fc4bab4e640 1 --2- 172.21.15.164:0/3022148625 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fc4980672a0 0x7fc498069760 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc4a00ab000 tx=0x7fc4a00a7070 comp rx=0 tx=0).stop 2024-03-23T14:57:43.637 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:43.633+0000 7fc4bab4e640 1 -- 172.21.15.164:0/3022148625 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc4a40047d0 msgr2=0x7fc4b4142710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:43.637 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:43.633+0000 7fc4bab4e640 1 --2- 172.21.15.164:0/3022148625 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc4a40047d0 0x7fc4b4142710 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fc4a80115e0 tx=0x7fc4a8011aa0 comp rx=0 tx=0).stop 2024-03-23T14:57:43.637 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:43.633+0000 7fc4bab4e640 1 -- 172.21.15.164:0/3022148625 shutdown_connections 2024-03-23T14:57:43.637 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:43.633+0000 7fc4bab4e640 1 --2- 172.21.15.164:0/3022148625 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc4b41e09e0 0x7fc4b41e0e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:43.638 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:43.633+0000 7fc4bab4e640 1 --2- 172.21.15.164:0/3022148625 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fc4980672a0 0x7fc498069760 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:43.638 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:43.633+0000 7fc4bab4e640 1 --2- 172.21.15.164:0/3022148625 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc4b40725f0 0x7fc4b4144420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:43.638 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:43.633+0000 7fc4bab4e640 1 --2- 172.21.15.164:0/3022148625 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc4a40047d0 0x7fc4b4142710 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:43.638 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:43.633+0000 7fc4bab4e640 1 -- 172.21.15.164:0/3022148625 >> 172.21.15.164:0/3022148625 conn(0x7fc4b406b930 msgr2=0x7fc4b40b6380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:43.638 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:43.633+0000 7fc4bab4e640 1 -- 172.21.15.164:0/3022148625 shutdown_connections 2024-03-23T14:57:43.638 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:43.633+0000 7fc4bab4e640 1 -- 172.21.15.164:0/3022148625 wait complete. 2024-03-23T14:57:43.656 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-03-23T14:57:43.657 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T14:57:43.804 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:43.801+0000 7fddbee50640 1 Processor -- start 2024-03-23T14:57:43.804 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:43.801+0000 7fddbee50640 1 -- start start 2024-03-23T14:57:43.805 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:43.801+0000 7fddbee50640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fddb80b9e80 0x7fddb80b5590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:43.805 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:43.801+0000 7fddbee50640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fddb8140f30 0x7fddb8141380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:43.805 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:43.801+0000 7fddbee50640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fddb81418c0 con 0x7fddb80b9190 2024-03-23T14:57:43.805 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:43.801+0000 7fddbee50640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fddb80b48b0 con 0x7fddb8140f30 2024-03-23T14:57:43.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:43.801+0000 7fddbee50640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 -- 0x7fddb80b5f20 con 0x7fddb80b9e80 2024-03-23T14:57:43.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:43.801+0000 7fddbd3ed640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fddb8140f30 0x7fddb8141380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:43.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:43.801+0000 7fddbd3ed640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fddb8140f30 0x7fddb8141380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:44684/0 (socket says 172.21.15.164:44684) 2024-03-23T14:57:43.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:43.801+0000 7fddbd3ed640 1 -- 172.21.15.164:0/3905560643 learned_addr learned my addr 172.21.15.164:0/3905560643 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:43.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:43.801+0000 7fddbcbec640 1 --2- 172.21.15.164:0/3905560643 >> v2:172.21.15.173:3300/0 conn(0x7fddb80b9e80 0x7fddb80b5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:43.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:43.801+0000 7fddbd3ed640 1 -- 172.21.15.164:0/3905560643 >> v2:172.21.15.173:3300/0 conn(0x7fddb80b9e80 msgr2=0x7fddb80b5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:43.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:43.801+0000 7fddbd3ed640 1 --2- 172.21.15.164:0/3905560643 >> v2:172.21.15.173:3300/0 conn(0x7fddb80b9e80 0x7fddb80b5590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:43.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:43.801+0000 7fddbd3ed640 1 -- 172.21.15.164:0/3905560643 >> v1:172.21.15.173:6789/0 conn(0x7fddb80b9190 legacy=0x7fddb814a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:43.806 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:43.801+0000 7fddbd3ed640 1 -- 172.21.15.164:0/3905560643 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddb80b4280 con 0x7fddb8140f30 2024-03-23T14:57:43.808 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:43.801+0000 7fddbd3ed640 1 --2- 172.21.15.164:0/3905560643 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fddb8140f30 0x7fddb8141380 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fddb80bdc80 tx=0x7fddb001c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dd4939f06fdbb45 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:43.808 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:43.801+0000 7fddb77fe640 1 -- 172.21.15.164:0/3905560643 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddb00a5770 con 0x7fddb8140f30 2024-03-23T14:57:43.808 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:43.801+0000 7fddb77fe640 1 -- 172.21.15.164:0/3905560643 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fddb00a58d0 con 0x7fddb8140f30 2024-03-23T14:57:43.808 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:43.801+0000 7fddb77fe640 1 -- 172.21.15.164:0/3905560643 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddb00a5c60 con 0x7fddb8140f30 2024-03-23T14:57:43.808 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:43.805+0000 7fddbee50640 1 -- 172.21.15.164:0/3905560643 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fddb8140f30 msgr2=0x7fddb8141380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:43.808 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:43.805+0000 7fddbee50640 1 --2- 172.21.15.164:0/3905560643 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fddb8140f30 0x7fddb8141380 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fddb80bdc80 tx=0x7fddb001c890 comp rx=0 tx=0).stop 2024-03-23T14:57:43.808 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:43.805+0000 7fddbee50640 1 -- 172.21.15.164:0/3905560643 shutdown_connections 2024-03-23T14:57:43.808 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:43.805+0000 7fddbee50640 1 --2- 172.21.15.164:0/3905560643 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fddb8140f30 0x7fddb8141380 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:43.808 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:43.805+0000 7fddbee50640 1 --2- 172.21.15.164:0/3905560643 >> v2:172.21.15.173:3300/0 conn(0x7fddb80b9e80 0x7fddb80b5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:43.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:43.805+0000 7fddbee50640 1 -- 172.21.15.164:0/3905560643 >> 172.21.15.164:0/3905560643 conn(0x7fddb802c3c0 msgr2=0x7fddb802c7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:43.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:43.805+0000 7fddbee50640 1 -- 172.21.15.164:0/3905560643 shutdown_connections 2024-03-23T14:57:43.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:43.805+0000 7fddbee50640 1 -- 172.21.15.164:0/3905560643 wait complete. 2024-03-23T14:57:43.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:43.805+0000 7fddbee50640 1 Processor -- start 2024-03-23T14:57:43.809 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:43.805+0000 7fddbee50640 1 -- start start 2024-03-23T14:57:43.811 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:43.805+0000 7fddbee50640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fddb80b9190 0x7fddb813f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:43.811 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:43.805+0000 7fddbee50640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fddb80b9e80 0x7fddb81420c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:43.811 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:43.805+0000 7fddbee50640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fddb8140f30 0x7fddb8139de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:43.811 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:43.805+0000 7fddbee50640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fddb813cd30 con 0x7fddb80b9190 2024-03-23T14:57:43.811 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:43.805+0000 7fddbee50640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fddb813c5c0 con 0x7fddb80b9e80 2024-03-23T14:57:43.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:43.805+0000 7fddbee50640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fddb8144060 con 0x7fddb8140f30 2024-03-23T14:57:43.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:43.805+0000 7fddbcbec640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fddb80b9190 0x7fddb813f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:43.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:43.805+0000 7fddbcbec640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fddb80b9190 0x7fddb813f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:44692/0 (socket says 172.21.15.164:44692) 2024-03-23T14:57:43.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:43.805+0000 7fddbcbec640 1 -- 172.21.15.164:0/2846053032 learned_addr learned my addr 172.21.15.164:0/2846053032 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:43.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:43.805+0000 7fddb7fff640 1 --2- 172.21.15.164:0/2846053032 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fddb80b9e80 0x7fddb81420c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:43.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:43.805+0000 7fddbd3ed640 1 --2- 172.21.15.164:0/2846053032 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fddb8140f30 0x7fddb8139de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:43.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:43.805+0000 7fddbcbec640 1 -- 172.21.15.164:0/2846053032 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fddb8140f30 msgr2=0x7fddb8139de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:43.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:43.805+0000 7fddbcbec640 1 --2- 172.21.15.164:0/2846053032 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fddb8140f30 0x7fddb8139de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:43.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:43.805+0000 7fddbcbec640 1 -- 172.21.15.164:0/2846053032 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fddb80b9e80 msgr2=0x7fddb81420c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:43.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:43.805+0000 7fddbcbec640 1 --2- 172.21.15.164:0/2846053032 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fddb80b9e80 0x7fddb81420c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:43.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:43.805+0000 7fddbcbec640 1 -- 172.21.15.164:0/2846053032 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddb0009b30 con 0x7fddb80b9190 2024-03-23T14:57:43.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:43.805+0000 7fddb7fff640 1 --2- 172.21.15.164:0/2846053032 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fddb80b9e80 0x7fddb81420c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T14:57:43.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:43.805+0000 7fddbcbec640 1 --2- 172.21.15.164:0/2846053032 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fddb80b9190 0x7fddb813f470 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fdda400d800 tx=0x7fdda400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:43.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:43.805+0000 7fddb57fa640 1 -- 172.21.15.164:0/2846053032 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdda400b6c0 con 0x7fddb80b9190 2024-03-23T14:57:43.812 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:43.805+0000 7fddb57fa640 1 -- 172.21.15.164:0/2846053032 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdda400b820 con 0x7fddb80b9190 2024-03-23T14:57:43.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:43.805+0000 7fddb57fa640 1 -- 172.21.15.164:0/2846053032 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdda400bb90 con 0x7fddb80b9190 2024-03-23T14:57:43.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:43.805+0000 7fddbee50640 1 -- 172.21.15.164:0/2846053032 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddb813fd40 con 0x7fddb80b9190 2024-03-23T14:57:43.813 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:43.805+0000 7fddbee50640 1 -- 172.21.15.164:0/2846053032 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddb813c810 con 0x7fddb80b9190 2024-03-23T14:57:43.814 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:43.809+0000 7fddbee50640 1 -- 172.21.15.164:0/2846053032 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fddb813c9d0 con 0x7fddb80b9190 2024-03-23T14:57:43.814 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:43.809+0000 7fddb57fa640 1 -- 172.21.15.164:0/2846053032 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7fdda400bcf0 con 0x7fddb80b9190 2024-03-23T14:57:43.814 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:43.809+0000 7fddb57fa640 1 --2- 172.21.15.164:0/2846053032 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fdd9c0672a0 0x7fdd9c069760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:43.814 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:43.809+0000 7fddb57fa640 1 -- 172.21.15.164:0/2846053032 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5709+0+0 (secure 0 0 0) 0x7fdda4085f30 con 0x7fddb80b9190 2024-03-23T14:57:43.814 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:43.809+0000 7fddb7fff640 1 --2- 172.21.15.164:0/2846053032 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fdd9c0672a0 0x7fdd9c069760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:43.814 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:43.809+0000 7fddb7fff640 1 --2- 172.21.15.164:0/2846053032 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fdd9c0672a0 0x7fdd9c069760 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fddb813de10 tx=0x7fddac009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:43.817 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:43.813+0000 7fddb57fa640 1 -- 172.21.15.164:0/2846053032 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7fdda4052c50 con 0x7fddb80b9190 2024-03-23T14:57:44.218 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.213+0000 7fddbee50640 1 -- 172.21.15.164:0/2846053032 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fddb80690a0 con 0x7fddb80b9190 2024-03-23T14:57:44.219 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.213+0000 7fddb57fa640 1 -- 172.21.15.164:0/2846053032 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+3537 (secure 0 0 0) 0x7fdda4056310 con 0x7fddb80b9190 2024-03-23T14:57:44.219 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-23T14:57:44.219 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":13,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4619,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/1401585102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":1401585102},{"type":"v1","addr":"172.21.15.173:6833","nonce":1401585102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-23T14:57:40.961416+0000","modified":"2024-03-23T14:57:43.045871+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4935},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4935":{"gid":4935,"name":"a","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.164:6835/3261092607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":3261092607},{"type":"v1","addr":"172.21.15.164:6835","nonce":3261092607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4935,"qdb_cluster":[4935]},"id":2}]} 2024-03-23T14:57:44.220 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 13 2024-03-23T14:57:44.222 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.217+0000 7fddbee50640 1 -- 172.21.15.164:0/2846053032 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fdd9c0672a0 msgr2=0x7fdd9c069760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:44.222 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.217+0000 7fddbee50640 1 --2- 172.21.15.164:0/2846053032 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fdd9c0672a0 0x7fdd9c069760 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fddb813de10 tx=0x7fddac009040 comp rx=0 tx=0).stop 2024-03-23T14:57:44.222 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.217+0000 7fddbee50640 1 -- 172.21.15.164:0/2846053032 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fddb80b9190 msgr2=0x7fddb813f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:44.222 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.217+0000 7fddbee50640 1 --2- 172.21.15.164:0/2846053032 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fddb80b9190 0x7fddb813f470 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fdda400d800 tx=0x7fdda400dcc0 comp rx=0 tx=0).stop 2024-03-23T14:57:44.222 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.217+0000 7fddbee50640 1 -- 172.21.15.164:0/2846053032 shutdown_connections 2024-03-23T14:57:44.222 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.217+0000 7fddbee50640 1 --2- 172.21.15.164:0/2846053032 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fddb8140f30 0x7fddb8139de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:44.222 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.217+0000 7fddbee50640 1 --2- 172.21.15.164:0/2846053032 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fdd9c0672a0 0x7fdd9c069760 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:44.222 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.217+0000 7fddbee50640 1 --2- 172.21.15.164:0/2846053032 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fddb80b9e80 0x7fddb81420c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:44.222 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.217+0000 7fddbee50640 1 --2- 172.21.15.164:0/2846053032 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fddb80b9190 0x7fddb813f470 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:44.222 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.217+0000 7fddbee50640 1 -- 172.21.15.164:0/2846053032 >> 172.21.15.164:0/2846053032 conn(0x7fddb802c3c0 msgr2=0x7fddb80bcbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:44.222 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.217+0000 7fddbee50640 1 -- 172.21.15.164:0/2846053032 shutdown_connections 2024-03-23T14:57:44.222 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.217+0000 7fddbee50640 1 -- 172.21.15.164:0/2846053032 wait complete. 2024-03-23T14:57:44.238 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T14:57:44.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.401+0000 7eff4c1cd640 1 Processor -- start 2024-03-23T14:57:44.407 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.401+0000 7eff4c1cd640 1 -- start start 2024-03-23T14:57:44.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.405+0000 7eff4c1cd640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7eff4409def0 0x7eff440b45e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:44.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.405+0000 7eff4c1cd640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7eff440b4b20 0x7eff4406c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:44.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.405+0000 7eff4c1cd640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7eff4406d650 con 0x7eff440add50 2024-03-23T14:57:44.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.405+0000 7eff4c1cd640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 -- 0x7eff440af4a0 con 0x7eff440b4b20 2024-03-23T14:57:44.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.405+0000 7eff4c1cd640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7eff4409e2d0 con 0x7eff4409def0 2024-03-23T14:57:44.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.405+0000 7eff49f69640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7eff4409def0 0x7eff440b45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:44.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.405+0000 7eff4a76a640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7eff440b4b20 0x7eff4406c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:44.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.405+0000 7eff49f69640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7eff4409def0 0x7eff440b45e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.164:41240/0 (socket says 172.21.15.164:41240) 2024-03-23T14:57:44.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.405+0000 7eff49768640 1 --1- >> v1:172.21.15.173:6789/0 conn(0x7eff440add50 0x7eff4406bf70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.173:6789/0 says I am v1:172.21.15.164:39446/0 (socket says 172.21.15.164:39446) 2024-03-23T14:57:44.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.405+0000 7eff4a76a640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7eff440b4b20 0x7eff4406c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.164:41034/0 (socket says 172.21.15.164:41034) 2024-03-23T14:57:44.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.405+0000 7eff49f69640 1 -- 172.21.15.164:0/158055930 learned_addr learned my addr 172.21.15.164:0/158055930 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:44.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.405+0000 7eff4a76a640 1 -- 172.21.15.164:0/158055930 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7eff4409def0 msgr2=0x7eff440b45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:44.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.405+0000 7eff4a76a640 1 --2- 172.21.15.164:0/158055930 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7eff4409def0 0x7eff440b45e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:44.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.405+0000 7eff4a76a640 1 -- 172.21.15.164:0/158055930 >> v1:172.21.15.173:6789/0 conn(0x7eff440add50 legacy=0x7eff4406bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:44.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.405+0000 7eff4a76a640 1 -- 172.21.15.164:0/158055930 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff440aeca0 con 0x7eff440b4b20 2024-03-23T14:57:44.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.405+0000 7eff48f67640 1 -- 172.21.15.164:0/158055930 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2382017443 0 0) 0x7eff4406d650 con 0x7eff440add50 2024-03-23T14:57:44.410 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.405+0000 7eff49f69640 1 --2- 172.21.15.164:0/158055930 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7eff4409def0 0x7eff440b45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T14:57:44.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.409+0000 7eff4a76a640 1 --2- 172.21.15.164:0/158055930 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7eff440b4b20 0x7eff4406c680 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7eff3c009d90 tx=0x7eff3c01c850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b43cbe762f08362d server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:44.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.409+0000 7eff48f67640 1 -- 172.21.15.164:0/158055930 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff3c0a5870 con 0x7eff440b4b20 2024-03-23T14:57:44.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.409+0000 7eff48f67640 1 -- 172.21.15.164:0/158055930 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff3c0a59d0 con 0x7eff440b4b20 2024-03-23T14:57:44.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.409+0000 7eff48f67640 1 -- 172.21.15.164:0/158055930 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff3c0a5d60 con 0x7eff440b4b20 2024-03-23T14:57:44.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.409+0000 7eff4c1cd640 1 -- 172.21.15.164:0/158055930 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7eff440b4b20 msgr2=0x7eff4406c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:44.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.409+0000 7eff4c1cd640 1 --2- 172.21.15.164:0/158055930 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7eff440b4b20 0x7eff4406c680 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7eff3c009d90 tx=0x7eff3c01c850 comp rx=0 tx=0).stop 2024-03-23T14:57:44.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.409+0000 7eff4c1cd640 1 -- 172.21.15.164:0/158055930 shutdown_connections 2024-03-23T14:57:44.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.409+0000 7eff4c1cd640 1 --2- 172.21.15.164:0/158055930 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7eff440b4b20 0x7eff4406c680 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:44.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.409+0000 7eff4c1cd640 1 --2- 172.21.15.164:0/158055930 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7eff4409def0 0x7eff440b45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:44.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.409+0000 7eff4c1cd640 1 -- 172.21.15.164:0/158055930 >> 172.21.15.164:0/158055930 conn(0x7eff440b8ca0 msgr2=0x7eff440b90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:44.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.409+0000 7eff4c1cd640 1 -- 172.21.15.164:0/158055930 shutdown_connections 2024-03-23T14:57:44.414 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.409+0000 7eff4c1cd640 1 -- 172.21.15.164:0/158055930 wait complete. 2024-03-23T14:57:44.416 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.409+0000 7eff4c1cd640 1 Processor -- start 2024-03-23T14:57:44.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.409+0000 7eff4c1cd640 1 -- start start 2024-03-23T14:57:44.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.409+0000 7eff4c1cd640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7eff4409def0 0x7eff4409cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:44.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.409+0000 7eff4c1cd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7eff440add50 0x7eff44071740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:44.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.409+0000 7eff4c1cd640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7eff440b4b20 0x7eff4406fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:44.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.409+0000 7eff4c1cd640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7eff4406f870 con 0x7eff440add50 2024-03-23T14:57:44.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.409+0000 7eff4c1cd640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7eff440b1f50 con 0x7eff440b4b20 2024-03-23T14:57:44.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.409+0000 7eff4c1cd640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7eff44070c10 con 0x7eff4409def0 2024-03-23T14:57:44.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.409+0000 7eff49768640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7eff440add50 0x7eff44071740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:44.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.409+0000 7eff49768640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7eff440add50 0x7eff44071740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:44696/0 (socket says 172.21.15.164:44696) 2024-03-23T14:57:44.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.409+0000 7eff49768640 1 -- 172.21.15.164:0/312593569 learned_addr learned my addr 172.21.15.164:0/312593569 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:44.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.409+0000 7eff4a76a640 1 --2- 172.21.15.164:0/312593569 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7eff440b4b20 0x7eff4406fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:44.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.409+0000 7eff49f69640 1 --2- 172.21.15.164:0/312593569 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7eff4409def0 0x7eff4409cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:44.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.413+0000 7eff49768640 1 -- 172.21.15.164:0/312593569 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7eff4409def0 msgr2=0x7eff4409cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:44.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.413+0000 7eff49768640 1 --2- 172.21.15.164:0/312593569 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7eff4409def0 0x7eff4409cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:44.417 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.413+0000 7eff49768640 1 -- 172.21.15.164:0/312593569 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7eff440b4b20 msgr2=0x7eff4406fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:44.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.413+0000 7eff49768640 1 --2- 172.21.15.164:0/312593569 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7eff440b4b20 0x7eff4406fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:44.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.413+0000 7eff49768640 1 -- 172.21.15.164:0/312593569 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff340094d0 con 0x7eff440add50 2024-03-23T14:57:44.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.413+0000 7eff49f69640 1 --2- 172.21.15.164:0/312593569 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7eff4409def0 0x7eff4409cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T14:57:44.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.413+0000 7eff4a76a640 1 --2- 172.21.15.164:0/312593569 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7eff440b4b20 0x7eff4406fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T14:57:44.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.413+0000 7eff49768640 1 --2- 172.21.15.164:0/312593569 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7eff440add50 0x7eff44071740 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7eff400011f0 tx=0x7eff4000c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:44.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.413+0000 7eff3a7fc640 1 -- 172.21.15.164:0/312593569 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff400074e0 con 0x7eff440add50 2024-03-23T14:57:44.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.413+0000 7eff3a7fc640 1 -- 172.21.15.164:0/312593569 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff40007640 con 0x7eff440add50 2024-03-23T14:57:44.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.413+0000 7eff4c1cd640 1 -- 172.21.15.164:0/312593569 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff3c009a70 con 0x7eff440add50 2024-03-23T14:57:44.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.413+0000 7eff3a7fc640 1 -- 172.21.15.164:0/312593569 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff40004a10 con 0x7eff440add50 2024-03-23T14:57:44.418 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.413+0000 7eff4c1cd640 1 -- 172.21.15.164:0/312593569 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff44070890 con 0x7eff440add50 2024-03-23T14:57:44.419 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.413+0000 7eff4c1cd640 1 -- 172.21.15.164:0/312593569 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff44070ef0 con 0x7eff440add50 2024-03-23T14:57:44.419 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.413+0000 7eff3a7fc640 1 -- 172.21.15.164:0/312593569 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7eff40004b70 con 0x7eff440add50 2024-03-23T14:57:44.419 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.413+0000 7eff3a7fc640 1 --2- 172.21.15.164:0/312593569 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7eff280672a0 0x7eff28069760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:44.419 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.413+0000 7eff3a7fc640 1 -- 172.21.15.164:0/312593569 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5709+0+0 (secure 0 0 0) 0x7eff40084550 con 0x7eff440add50 2024-03-23T14:57:44.419 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.413+0000 7eff49f69640 1 --2- 172.21.15.164:0/312593569 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7eff280672a0 0x7eff28069760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:44.419 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.413+0000 7eff49f69640 1 --2- 172.21.15.164:0/312593569 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7eff280672a0 0x7eff28069760 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7eff4409db30 tx=0x7eff34008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:44.423 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.417+0000 7eff3a7fc640 1 -- 172.21.15.164:0/312593569 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7eff44070ef0 con 0x7eff440add50 2024-03-23T14:57:44.629 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.625+0000 7eff4c1cd640 1 -- 172.21.15.164:0/312593569 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7eff44070ef0 con 0x7eff440add50 2024-03-23T14:57:44.634 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.629+0000 7eff3a7fc640 1 -- 172.21.15.164:0/312593569 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) ==== 74+0+16693 (secure 0 0 0) 0x7eff44070ef0 con 0x7eff440add50 2024-03-23T14:57:44.634 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-23T14:57:44.634 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":29,"fsid":"45c7911b-b69d-4e68-907b-fbfeb780aa3b","created":"2024-03-23T14:56:34.367633+0000","modified":"2024-03-23T14:57:42.610096+0000","last_up_change":"2024-03-23T14:56:46.297403+0000","last_in_change":"2024-03-23T14:56:36.629884+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T14:56:47.186157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-03-23T14:57:39.290666+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-03-23T14:57:39.934337+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"29","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"80abdc62-bd0d-4e60-a554-7ed6fd67cf54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":984716690},{"type":"v1","addr":"172.21.15.164:6801","nonce":984716690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":984716690},{"type":"v1","addr":"172.21.15.164:6803","nonce":984716690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":984716690},{"type":"v1","addr":"172.21.15.164:6807","nonce":984716690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":984716690},{"type":"v1","addr":"172.21.15.164:6805","nonce":984716690}]},"public_addr":"172.21.15.164:6801/984716690","cluster_addr":"172.21.15.164:6803/984716690","heartbeat_back_addr":"172.21.15.164:6807/984716690","heartbeat_front_addr":"172.21.15.164:6805/984716690","state":["exists","up"]},{"osd":1,"uuid":"f9001b06-be4d-4f42-a0e6-bcef2afa0d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":1349657239},{"type":"v1","addr":"172.21.15.164:6817","nonce":1349657239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":1349657239},{"type":"v1","addr":"172.21.15.164:6819","nonce":1349657239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":1349657239},{"type":"v1","addr":"172.21.15.164:6823","nonce":1349657239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":1349657239},{"type":"v1","addr":"172.21.15.164:6821","nonce":1349657239}]},"public_addr":"172.21.15.164:6817/1349657239","cluster_addr":"172.21.15.164:6819/1349657239","heartbeat_back_addr":"172.21.15.164:6823/1349657239","heartbeat_front_addr":"172.21.15.164:6821/1349657239","state":["exists","up"]},{"osd":2,"uuid":"fa0aeb9f-8b54-4831-a188-3e361c7980d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":887861252},{"type":"v1","addr":"172.21.15.164:6809","nonce":887861252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":887861252},{"type":"v1","addr":"172.21.15.164:6811","nonce":887861252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":887861252},{"type":"v1","addr":"172.21.15.164:6815","nonce":887861252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":887861252},{"type":"v1","addr":"172.21.15.164:6813","nonce":887861252}]},"public_addr":"172.21.15.164:6809/887861252","cluster_addr":"172.21.15.164:6811/887861252","heartbeat_back_addr":"172.21.15.164:6815/887861252","heartbeat_front_addr":"172.21.15.164:6813/887861252","state":["exists","up"]},{"osd":3,"uuid":"7e18c957-3091-46d5-8e17-8f3354c1726c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":812540271},{"type":"v1","addr":"172.21.15.164:6825","nonce":812540271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":812540271},{"type":"v1","addr":"172.21.15.164:6827","nonce":812540271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":812540271},{"type":"v1","addr":"172.21.15.164:6831","nonce":812540271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":812540271},{"type":"v1","addr":"172.21.15.164:6829","nonce":812540271}]},"public_addr":"172.21.15.164:6825/812540271","cluster_addr":"172.21.15.164:6827/812540271","heartbeat_back_addr":"172.21.15.164:6831/812540271","heartbeat_front_addr":"172.21.15.164:6829/812540271","state":["exists","up"]},{"osd":4,"uuid":"8e83f315-42bf-4471-9ce8-d81948da0f9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":2656857421},{"type":"v1","addr":"172.21.15.173:6825","nonce":2656857421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2656857421},{"type":"v1","addr":"172.21.15.173:6827","nonce":2656857421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2656857421},{"type":"v1","addr":"172.21.15.173:6831","nonce":2656857421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":2656857421},{"type":"v1","addr":"172.21.15.173:6829","nonce":2656857421}]},"public_addr":"172.21.15.173:6825/2656857421","cluster_addr":"172.21.15.173:6827/2656857421","heartbeat_back_addr":"172.21.15.173:6831/2656857421","heartbeat_front_addr":"172.21.15.173:6829/2656857421","state":["exists","up"]},{"osd":5,"uuid":"a230c872-71f9-40ab-8999-defd0162d4ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":3676483185},{"type":"v1","addr":"172.21.15.173:6801","nonce":3676483185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":3676483185},{"type":"v1","addr":"172.21.15.173:6803","nonce":3676483185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":3676483185},{"type":"v1","addr":"172.21.15.173:6807","nonce":3676483185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":3676483185},{"type":"v1","addr":"172.21.15.173:6805","nonce":3676483185}]},"public_addr":"172.21.15.173:6801/3676483185","cluster_addr":"172.21.15.173:6803/3676483185","heartbeat_back_addr":"172.21.15.173:6807/3676483185","heartbeat_front_addr":"172.21.15.173:6805/3676483185","state":["exists","up"]},{"osd":6,"uuid":"6193bfd7-8432-4f7f-a117-a3fcee21af99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":3631684087},{"type":"v1","addr":"172.21.15.173:6809","nonce":3631684087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":3631684087},{"type":"v1","addr":"172.21.15.173:6811","nonce":3631684087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":3631684087},{"type":"v1","addr":"172.21.15.173:6815","nonce":3631684087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":3631684087},{"type":"v1","addr":"172.21.15.173:6813","nonce":3631684087}]},"public_addr":"172.21.15.173:6809/3631684087","cluster_addr":"172.21.15.173:6811/3631684087","heartbeat_back_addr":"172.21.15.173:6815/3631684087","heartbeat_front_addr":"172.21.15.173:6813/3631684087","state":["exists","up"]},{"osd":7,"uuid":"b0f1b80a-afc4-4538-83b9-f081d2dcb8cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":3261114948},{"type":"v1","addr":"172.21.15.173:6817","nonce":3261114948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":3261114948},{"type":"v1","addr":"172.21.15.173:6819","nonce":3261114948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":3261114948},{"type":"v1","addr":"172.21.15.173:6823","nonce":3261114948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":3261114948},{"type":"v1","addr":"172.21.15.173:6821","nonce":3261114948}]},"public_addr":"172.21.15.173:6817/3261114948","cluster_addr":"172.21.15.173:6819/3261114948","heartbeat_back_addr":"172.21.15.173:6823/3261114948","heartbeat_front_addr":"172.21.15.173:6821/3261114948","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.188390+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.159932+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.191101+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.181776+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.224365+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.177515+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.153631+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.139972+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T14:57:44.637 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.633+0000 7eff4c1cd640 1 -- 172.21.15.164:0/312593569 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7eff280672a0 msgr2=0x7eff28069760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:44.637 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.633+0000 7eff4c1cd640 1 --2- 172.21.15.164:0/312593569 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7eff280672a0 0x7eff28069760 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7eff4409db30 tx=0x7eff34008040 comp rx=0 tx=0).stop 2024-03-23T14:57:44.637 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.633+0000 7eff4c1cd640 1 -- 172.21.15.164:0/312593569 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7eff440add50 msgr2=0x7eff44071740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:44.637 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.633+0000 7eff4c1cd640 1 --2- 172.21.15.164:0/312593569 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7eff440add50 0x7eff44071740 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7eff400011f0 tx=0x7eff4000c9b0 comp rx=0 tx=0).stop 2024-03-23T14:57:44.637 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.633+0000 7eff4c1cd640 1 -- 172.21.15.164:0/312593569 shutdown_connections 2024-03-23T14:57:44.637 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.633+0000 7eff4c1cd640 1 --2- 172.21.15.164:0/312593569 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7eff280672a0 0x7eff28069760 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:44.637 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.633+0000 7eff4c1cd640 1 --2- 172.21.15.164:0/312593569 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7eff440b4b20 0x7eff4406fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:44.637 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.633+0000 7eff4c1cd640 1 --2- 172.21.15.164:0/312593569 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7eff440add50 0x7eff44071740 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:44.637 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.633+0000 7eff4c1cd640 1 --2- 172.21.15.164:0/312593569 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7eff4409def0 0x7eff4409cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:44.637 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.633+0000 7eff4c1cd640 1 -- 172.21.15.164:0/312593569 >> 172.21.15.164:0/312593569 conn(0x7eff440b8ca0 msgr2=0x7eff441428d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:44.637 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.633+0000 7eff4c1cd640 1 -- 172.21.15.164:0/312593569 shutdown_connections 2024-03-23T14:57:44.638 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.633+0000 7eff4c1cd640 1 -- 172.21.15.164:0/312593569 wait complete. 2024-03-23T14:57:44.653 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-23T14:57:44.653 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-23T14:57:44.764 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.761+0000 7f5507930640 1 Processor -- start 2024-03-23T14:57:44.764 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.761+0000 7f5507930640 1 -- start start 2024-03-23T14:57:44.765 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.761+0000 7f5507930640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f55000adc70 0x7f55001445e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:44.765 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.761+0000 7f5507930640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5500144b20 0x7f55001477b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:44.765 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.761+0000 7f5507930640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55000b9ea0 con 0x7f55000b9ac0 2024-03-23T14:57:44.765 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.761+0000 7f5507930640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f55000af400 con 0x7f55000adc70 2024-03-23T14:57:44.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.761+0000 7f5507930640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f5500144fd0 con 0x7f5500144b20 2024-03-23T14:57:44.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.761+0000 7f5504ecb640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f55000adc70 0x7f55001445e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:44.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.761+0000 7f55056cc640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f55000b9ac0 0x7f55000ae920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:46760/0 (socket says 172.21.15.164:46760) 2024-03-23T14:57:44.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.761+0000 7f55056cc640 1 -- 172.21.15.164:0/4274648282 learned_addr learned my addr 172.21.15.164:0/4274648282 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-03-23T14:57:44.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.761+0000 7f54f7fff640 1 -- 172.21.15.164:0/4274648282 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4151602321 0 0) 0x7f55000b9ea0 con 0x7f55000b9ac0 2024-03-23T14:57:44.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.761+0000 7f54f7fff640 1 -- 172.21.15.164:0/4274648282 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f54ec003530 con 0x7f55000b9ac0 2024-03-23T14:57:44.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.761+0000 7f5505ecd640 1 --2- 172.21.15.164:0/4274648282 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5500144b20 0x7f55001477b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:44.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.761+0000 7f5504ecb640 1 -- 172.21.15.164:0/4274648282 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5500144b20 msgr2=0x7f55001477b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:44.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.761+0000 7f5504ecb640 1 --2- 172.21.15.164:0/4274648282 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5500144b20 0x7f55001477b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:44.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.761+0000 7f54f7fff640 1 -- 172.21.15.164:0/4274648282 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 858526481 0 0) 0x7f54ec003530 con 0x7f55000b9ac0 2024-03-23T14:57:44.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.761+0000 7f5504ecb640 1 -- 172.21.15.164:0/4274648282 >> v1:172.21.15.164:6789/0 conn(0x7f55000b9ac0 legacy=0x7f55000ae920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:44.766 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.761+0000 7f5504ecb640 1 -- 172.21.15.164:0/4274648282 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55000b53e0 con 0x7f55000adc70 2024-03-23T14:57:44.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.761+0000 7f5504ecb640 1 --2- 172.21.15.164:0/4274648282 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f55000adc70 0x7f55001445e0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f54f00097c0 tx=0x7f54f001c300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc11b98e0f16aedc server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:44.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.761+0000 7f54f7fff640 1 -- 172.21.15.164:0/4274648282 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54fc001680 con 0x7f55000adc70 2024-03-23T14:57:44.767 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.765+0000 7f54f7fff640 1 -- 172.21.15.164:0/4274648282 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54f00a7050 con 0x7f55000adc70 2024-03-23T14:57:44.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.765+0000 7f54f7fff640 1 -- 172.21.15.164:0/4274648282 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f00ab620 con 0x7f55000adc70 2024-03-23T14:57:44.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.765+0000 7f5507930640 1 -- 172.21.15.164:0/4274648282 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f55000adc70 msgr2=0x7f55001445e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:44.768 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.765+0000 7f5507930640 1 --2- 172.21.15.164:0/4274648282 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f55000adc70 0x7f55001445e0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f54f00097c0 tx=0x7f54f001c300 comp rx=0 tx=0).stop 2024-03-23T14:57:44.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.765+0000 7f5507930640 1 -- 172.21.15.164:0/4274648282 shutdown_connections 2024-03-23T14:57:44.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.765+0000 7f5507930640 1 --2- 172.21.15.164:0/4274648282 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5500144b20 0x7f55001477b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:44.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.765+0000 7f5507930640 1 --2- 172.21.15.164:0/4274648282 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f55000adc70 0x7f55001445e0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:44.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.765+0000 7f5507930640 1 -- 172.21.15.164:0/4274648282 >> 172.21.15.164:0/4274648282 conn(0x7f550006b930 msgr2=0x7f55000b9150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:44.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.765+0000 7f5507930640 1 -- 172.21.15.164:0/4274648282 shutdown_connections 2024-03-23T14:57:44.769 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.765+0000 7f5507930640 1 -- 172.21.15.164:0/4274648282 wait complete. 2024-03-23T14:57:44.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.765+0000 7f5507930640 1 Processor -- start 2024-03-23T14:57:44.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.765+0000 7f5507930640 1 -- start start 2024-03-23T14:57:44.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.765+0000 7f5507930640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f55000adc70 0x7f550013e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:44.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.765+0000 7f5507930640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f55000b9ac0 0x7f55001e2d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:44.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.765+0000 7f5507930640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5500144b20 0x7f550013f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:44.771 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.765+0000 7f5507930640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f550013de30 con 0x7f55000b9ac0 2024-03-23T14:57:44.772 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.765+0000 7f5507930640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f550012e060 con 0x7f5500144b20 2024-03-23T14:57:44.772 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.765+0000 7f5507930640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f550013eb40 con 0x7f55000adc70 2024-03-23T14:57:44.772 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.765+0000 7f5504ecb640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f55000b9ac0 0x7f55001e2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:44.772 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.765+0000 7f5504ecb640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f55000b9ac0 0x7f55001e2d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:44704/0 (socket says 172.21.15.164:44704) 2024-03-23T14:57:44.772 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.765+0000 7f5504ecb640 1 -- 172.21.15.164:0/2794671007 learned_addr learned my addr 172.21.15.164:0/2794671007 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:44.772 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.765+0000 7f55056cc640 1 --2- 172.21.15.164:0/2794671007 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f55000adc70 0x7f550013e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:44.772 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.765+0000 7f5504ecb640 1 -- 172.21.15.164:0/2794671007 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f55000adc70 msgr2=0x7f550013e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:44.772 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.765+0000 7f5504ecb640 1 --2- 172.21.15.164:0/2794671007 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f55000adc70 0x7f550013e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:44.772 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.765+0000 7f5505ecd640 1 --2- 172.21.15.164:0/2794671007 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5500144b20 0x7f550013f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:44.772 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.765+0000 7f5504ecb640 1 -- 172.21.15.164:0/2794671007 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5500144b20 msgr2=0x7f550013f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:44.772 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.765+0000 7f5504ecb640 1 --2- 172.21.15.164:0/2794671007 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5500144b20 0x7f550013f2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:44.772 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.765+0000 7f5504ecb640 1 -- 172.21.15.164:0/2794671007 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54f0009420 con 0x7f55000b9ac0 2024-03-23T14:57:44.772 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.765+0000 7f5504ecb640 1 --2- 172.21.15.164:0/2794671007 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f55000b9ac0 0x7f55001e2d80 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f54f001a9b0 tx=0x7f54f001ab40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:44.772 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.765+0000 7f54f5ffb640 1 -- 172.21.15.164:0/2794671007 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f00b7450 con 0x7f55000b9ac0 2024-03-23T14:57:44.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.765+0000 7f54f5ffb640 1 -- 172.21.15.164:0/2794671007 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54f00b75b0 con 0x7f55000b9ac0 2024-03-23T14:57:44.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.765+0000 7f5507930640 1 -- 172.21.15.164:0/2794671007 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55001367e0 con 0x7f55000b9ac0 2024-03-23T14:57:44.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.765+0000 7f54f5ffb640 1 -- 172.21.15.164:0/2794671007 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f00b7960 con 0x7f55000b9ac0 2024-03-23T14:57:44.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.765+0000 7f5507930640 1 -- 172.21.15.164:0/2794671007 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55001e3a60 con 0x7f55000b9ac0 2024-03-23T14:57:44.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.765+0000 7f5507930640 1 -- 172.21.15.164:0/2794671007 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55000690a0 con 0x7f55000b9ac0 2024-03-23T14:57:44.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.769+0000 7f54f5ffb640 1 -- 172.21.15.164:0/2794671007 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7f54f00b7ac0 con 0x7f55000b9ac0 2024-03-23T14:57:44.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.769+0000 7f54f5ffb640 1 --2- 172.21.15.164:0/2794671007 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f54e40672a0 0x7f54e4069760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:44.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.769+0000 7f54f5ffb640 1 -- 172.21.15.164:0/2794671007 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5709+0+0 (secure 0 0 0) 0x7f54f011f860 con 0x7f55000b9ac0 2024-03-23T14:57:44.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.769+0000 7f55056cc640 1 --2- 172.21.15.164:0/2794671007 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f54e40672a0 0x7f54e4069760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:44.774 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.769+0000 7f55056cc640 1 --2- 172.21.15.164:0/2794671007 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f54e40672a0 0x7f54e4069760 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f54fc0011f0 tx=0x7f54fc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:44.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.773+0000 7f54f5ffb640 1 -- 172.21.15.164:0/2794671007 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7f54f00ec4a0 con 0x7f55000b9ac0 2024-03-23T14:57:44.932 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.929+0000 7f5507930640 1 -- 172.21.15.164:0/2794671007 --> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5500151ae0 con 0x7f54e40672a0 2024-03-23T14:57:44.946 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.941+0000 7f54f5ffb640 1 -- 172.21.15.164:0/2794671007 <== mgr.4107 v2:172.21.15.164:6832/2139087985 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251881 (secure 0 0 0) 0x7f5500151ae0 con 0x7f54e40672a0 2024-03-23T14:57:44.949 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-23T14:57:44.951 INFO:teuthology.orchestra.run.smithi164.stderr:dumped all 2024-03-23T14:57:44.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.945+0000 7f5507930640 1 -- 172.21.15.164:0/2794671007 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f54e40672a0 msgr2=0x7f54e4069760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:44.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.945+0000 7f5507930640 1 --2- 172.21.15.164:0/2794671007 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f54e40672a0 0x7f54e4069760 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f54fc0011f0 tx=0x7f54fc009040 comp rx=0 tx=0).stop 2024-03-23T14:57:44.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.945+0000 7f5507930640 1 -- 172.21.15.164:0/2794671007 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f55000b9ac0 msgr2=0x7f55001e2d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:44.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.945+0000 7f5507930640 1 --2- 172.21.15.164:0/2794671007 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f55000b9ac0 0x7f55001e2d80 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f54f001a9b0 tx=0x7f54f001ab40 comp rx=0 tx=0).stop 2024-03-23T14:57:44.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.945+0000 7f5507930640 1 -- 172.21.15.164:0/2794671007 shutdown_connections 2024-03-23T14:57:44.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.945+0000 7f5507930640 1 --2- 172.21.15.164:0/2794671007 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f54e40672a0 0x7f54e4069760 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:44.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.945+0000 7f5507930640 1 --2- 172.21.15.164:0/2794671007 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5500144b20 0x7f550013f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:44.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.945+0000 7f5507930640 1 --2- 172.21.15.164:0/2794671007 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f55000b9ac0 0x7f55001e2d80 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:44.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.945+0000 7f5507930640 1 --2- 172.21.15.164:0/2794671007 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f55000adc70 0x7f550013e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:44.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.945+0000 7f5507930640 1 -- 172.21.15.164:0/2794671007 >> 172.21.15.164:0/2794671007 conn(0x7f550006b930 msgr2=0x7f550013c4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:44.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.945+0000 7f5507930640 1 -- 172.21.15.164:0/2794671007 shutdown_connections 2024-03-23T14:57:44.952 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:44.945+0000 7f5507930640 1 -- 172.21.15.164:0/2794671007 wait complete. 2024-03-23T14:57:44.966 INFO:teuthology.orchestra.run.smithi164.stdout:{"pg_ready":true,"pg_map":{"version":50,"stamp":"2024-03-23T14:57:43.121566+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":87,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219772,"kb_used_data":3592,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754948,"statfs":{"total":773094113280,"available":772869066752,"internally_reserved":0,"allocated":3678208,"data_stored":2104531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":44,"apply_latency_ms":44,"commit_latency_ns":44000000,"apply_latency_ns":44000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001172"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":9,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.648999+0000","last_change":"2024-03-23T14:57:41.615391+0000","last_active":"2024-03-23T14:57:41.648999+0000","last_peered":"2024-03-23T14:57:41.648999+0000","last_clean":"2024-03-23T14:57:41.648999+0000","last_became_active":"2024-03-23T14:57:41.614713+0000","last_became_peered":"2024-03-23T14:57:41.614713+0000","last_unstale":"2024-03-23T14:57:41.648999+0000","last_undegraded":"2024-03-23T14:57:41.648999+0000","last_fullsized":"2024-03-23T14:57:41.648999+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_clean_scrub_stamp":"2024-03-23T14:57:40.577321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:43:37.787881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":9,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.627866+0000","last_change":"2024-03-23T14:57:41.621702+0000","last_active":"2024-03-23T14:57:41.627866+0000","last_peered":"2024-03-23T14:57:41.627866+0000","last_clean":"2024-03-23T14:57:41.627866+0000","last_became_active":"2024-03-23T14:57:41.621156+0000","last_became_peered":"2024-03-23T14:57:41.621156+0000","last_unstale":"2024-03-23T14:57:41.627866+0000","last_undegraded":"2024-03-23T14:57:41.627866+0000","last_fullsized":"2024-03-23T14:57:41.627866+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_clean_scrub_stamp":"2024-03-23T14:57:40.577321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:05:34.469230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":9,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.639444+0000","last_change":"2024-03-23T14:57:41.624751+0000","last_active":"2024-03-23T14:57:41.639444+0000","last_peered":"2024-03-23T14:57:41.639444+0000","last_clean":"2024-03-23T14:57:41.639444+0000","last_became_active":"2024-03-23T14:57:41.624125+0000","last_became_peered":"2024-03-23T14:57:41.624125+0000","last_unstale":"2024-03-23T14:57:41.639444+0000","last_undegraded":"2024-03-23T14:57:41.639444+0000","last_fullsized":"2024-03-23T14:57:41.639444+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_clean_scrub_stamp":"2024-03-23T14:57:40.577321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:42:36.132784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":9,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.639305+0000","last_change":"2024-03-23T14:57:41.635228+0000","last_active":"2024-03-23T14:57:41.639305+0000","last_peered":"2024-03-23T14:57:41.639305+0000","last_clean":"2024-03-23T14:57:41.639305+0000","last_became_active":"2024-03-23T14:57:41.634600+0000","last_became_peered":"2024-03-23T14:57:41.634600+0000","last_unstale":"2024-03-23T14:57:41.639305+0000","last_undegraded":"2024-03-23T14:57:41.639305+0000","last_fullsized":"2024-03-23T14:57:41.639305+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_clean_scrub_stamp":"2024-03-23T14:57:40.577321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:20:49.856427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":9,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.629837+0000","last_change":"2024-03-23T14:57:41.628530+0000","last_active":"2024-03-23T14:57:41.629837+0000","last_peered":"2024-03-23T14:57:41.629837+0000","last_clean":"2024-03-23T14:57:41.629837+0000","last_became_active":"2024-03-23T14:57:41.628168+0000","last_became_peered":"2024-03-23T14:57:41.628168+0000","last_unstale":"2024-03-23T14:57:41.629837+0000","last_undegraded":"2024-03-23T14:57:41.629837+0000","last_fullsized":"2024-03-23T14:57:41.629837+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_clean_scrub_stamp":"2024-03-23T14:57:40.577321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:14:46.990456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":9,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.627866+0000","last_change":"2024-03-23T14:57:41.621605+0000","last_active":"2024-03-23T14:57:41.627866+0000","last_peered":"2024-03-23T14:57:41.627866+0000","last_clean":"2024-03-23T14:57:41.627866+0000","last_became_active":"2024-03-23T14:57:41.621329+0000","last_became_peered":"2024-03-23T14:57:41.621329+0000","last_unstale":"2024-03-23T14:57:41.627866+0000","last_undegraded":"2024-03-23T14:57:41.627866+0000","last_fullsized":"2024-03-23T14:57:41.627866+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_clean_scrub_stamp":"2024-03-23T14:57:40.577321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:42:36.132784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":9,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.654437+0000","last_change":"2024-03-23T14:57:41.649953+0000","last_active":"2024-03-23T14:57:41.654437+0000","last_peered":"2024-03-23T14:57:41.654437+0000","last_clean":"2024-03-23T14:57:41.654437+0000","last_became_active":"2024-03-23T14:57:41.646113+0000","last_became_peered":"2024-03-23T14:57:41.646113+0000","last_unstale":"2024-03-23T14:57:41.654437+0000","last_undegraded":"2024-03-23T14:57:41.654437+0000","last_fullsized":"2024-03-23T14:57:41.654437+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_clean_scrub_stamp":"2024-03-23T14:57:40.577321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:42:36.132784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":9,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.654204+0000","last_change":"2024-03-23T14:57:41.645857+0000","last_active":"2024-03-23T14:57:41.654204+0000","last_peered":"2024-03-23T14:57:41.654204+0000","last_clean":"2024-03-23T14:57:41.654204+0000","last_became_active":"2024-03-23T14:57:41.633564+0000","last_became_peered":"2024-03-23T14:57:41.633564+0000","last_unstale":"2024-03-23T14:57:41.654204+0000","last_undegraded":"2024-03-23T14:57:41.654204+0000","last_fullsized":"2024-03-23T14:57:41.654204+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_clean_scrub_stamp":"2024-03-23T14:57:40.577321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:51:50.188772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":9,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.629994+0000","last_change":"2024-03-23T14:57:41.621763+0000","last_active":"2024-03-23T14:57:41.629994+0000","last_peered":"2024-03-23T14:57:41.629994+0000","last_clean":"2024-03-23T14:57:41.629994+0000","last_became_active":"2024-03-23T14:57:41.621025+0000","last_became_peered":"2024-03-23T14:57:41.621025+0000","last_unstale":"2024-03-23T14:57:41.629994+0000","last_undegraded":"2024-03-23T14:57:41.629994+0000","last_fullsized":"2024-03-23T14:57:41.629994+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_clean_scrub_stamp":"2024-03-23T14:57:40.577321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:28:32.371371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":9,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.633397+0000","last_change":"2024-03-23T14:57:41.628800+0000","last_active":"2024-03-23T14:57:41.633397+0000","last_peered":"2024-03-23T14:57:41.633397+0000","last_clean":"2024-03-23T14:57:41.633397+0000","last_became_active":"2024-03-23T14:57:41.628517+0000","last_became_peered":"2024-03-23T14:57:41.628517+0000","last_unstale":"2024-03-23T14:57:41.633397+0000","last_undegraded":"2024-03-23T14:57:41.633397+0000","last_fullsized":"2024-03-23T14:57:41.633397+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_clean_scrub_stamp":"2024-03-23T14:57:40.577321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:45:08.796108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":9,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.654103+0000","last_change":"2024-03-23T14:57:41.633753+0000","last_active":"2024-03-23T14:57:41.654103+0000","last_peered":"2024-03-23T14:57:41.654103+0000","last_clean":"2024-03-23T14:57:41.654103+0000","last_became_active":"2024-03-23T14:57:41.632768+0000","last_became_peered":"2024-03-23T14:57:41.632768+0000","last_unstale":"2024-03-23T14:57:41.654103+0000","last_undegraded":"2024-03-23T14:57:41.654103+0000","last_fullsized":"2024-03-23T14:57:41.654103+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_clean_scrub_stamp":"2024-03-23T14:57:40.577321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:20:49.856427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":9,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.641208+0000","last_change":"2024-03-23T14:57:41.626849+0000","last_active":"2024-03-23T14:57:41.641208+0000","last_peered":"2024-03-23T14:57:41.641208+0000","last_clean":"2024-03-23T14:57:41.641208+0000","last_became_active":"2024-03-23T14:57:41.626124+0000","last_became_peered":"2024-03-23T14:57:41.626124+0000","last_unstale":"2024-03-23T14:57:41.641208+0000","last_undegraded":"2024-03-23T14:57:41.641208+0000","last_fullsized":"2024-03-23T14:57:41.641208+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_clean_scrub_stamp":"2024-03-23T14:57:40.577321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:38:13.872392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":9,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.624920+0000","last_change":"2024-03-23T14:57:41.623838+0000","last_active":"2024-03-23T14:57:41.624920+0000","last_peered":"2024-03-23T14:57:41.624920+0000","last_clean":"2024-03-23T14:57:41.624920+0000","last_became_active":"2024-03-23T14:57:41.623449+0000","last_became_peered":"2024-03-23T14:57:41.623449+0000","last_unstale":"2024-03-23T14:57:41.624920+0000","last_undegraded":"2024-03-23T14:57:41.624920+0000","last_fullsized":"2024-03-23T14:57:41.624920+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_clean_scrub_stamp":"2024-03-23T14:57:40.577321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:59:22.990207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":9,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.627845+0000","last_change":"2024-03-23T14:57:41.620833+0000","last_active":"2024-03-23T14:57:41.627845+0000","last_peered":"2024-03-23T14:57:41.627845+0000","last_clean":"2024-03-23T14:57:41.627845+0000","last_became_active":"2024-03-23T14:57:41.620247+0000","last_became_peered":"2024-03-23T14:57:41.620247+0000","last_unstale":"2024-03-23T14:57:41.627845+0000","last_undegraded":"2024-03-23T14:57:41.627845+0000","last_fullsized":"2024-03-23T14:57:41.627845+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_clean_scrub_stamp":"2024-03-23T14:57:40.577321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:23:11.000724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":9,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.642318+0000","last_change":"2024-03-23T14:57:41.634231+0000","last_active":"2024-03-23T14:57:41.642318+0000","last_peered":"2024-03-23T14:57:41.642318+0000","last_clean":"2024-03-23T14:57:41.642318+0000","last_became_active":"2024-03-23T14:57:41.633584+0000","last_became_peered":"2024-03-23T14:57:41.633584+0000","last_unstale":"2024-03-23T14:57:41.642318+0000","last_undegraded":"2024-03-23T14:57:41.642318+0000","last_fullsized":"2024-03-23T14:57:41.642318+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_clean_scrub_stamp":"2024-03-23T14:57:40.577321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:44:02.215207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":9,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.656841+0000","last_change":"2024-03-23T14:57:41.652115+0000","last_active":"2024-03-23T14:57:41.656841+0000","last_peered":"2024-03-23T14:57:41.656841+0000","last_clean":"2024-03-23T14:57:41.656841+0000","last_became_active":"2024-03-23T14:57:41.644519+0000","last_became_peered":"2024-03-23T14:57:41.644519+0000","last_unstale":"2024-03-23T14:57:41.656841+0000","last_undegraded":"2024-03-23T14:57:41.656841+0000","last_fullsized":"2024-03-23T14:57:41.656841+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_clean_scrub_stamp":"2024-03-23T14:57:40.577321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:44:02.215207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":9,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.656752+0000","last_change":"2024-03-23T14:57:41.650919+0000","last_active":"2024-03-23T14:57:41.656752+0000","last_peered":"2024-03-23T14:57:41.656752+0000","last_clean":"2024-03-23T14:57:41.656752+0000","last_became_active":"2024-03-23T14:57:41.650239+0000","last_became_peered":"2024-03-23T14:57:41.650239+0000","last_unstale":"2024-03-23T14:57:41.656752+0000","last_undegraded":"2024-03-23T14:57:41.656752+0000","last_fullsized":"2024-03-23T14:57:41.656752+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_clean_scrub_stamp":"2024-03-23T14:57:40.577321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:38:13.872392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":9,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.634264+0000","last_change":"2024-03-23T14:57:41.627591+0000","last_active":"2024-03-23T14:57:41.634264+0000","last_peered":"2024-03-23T14:57:41.634264+0000","last_clean":"2024-03-23T14:57:41.634264+0000","last_became_active":"2024-03-23T14:57:41.627232+0000","last_became_peered":"2024-03-23T14:57:41.627232+0000","last_unstale":"2024-03-23T14:57:41.634264+0000","last_undegraded":"2024-03-23T14:57:41.634264+0000","last_fullsized":"2024-03-23T14:57:41.634264+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_clean_scrub_stamp":"2024-03-23T14:57:40.577321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:39:56.285297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":9,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.632591+0000","last_change":"2024-03-23T14:57:41.627012+0000","last_active":"2024-03-23T14:57:41.632591+0000","last_peered":"2024-03-23T14:57:41.632591+0000","last_clean":"2024-03-23T14:57:41.632591+0000","last_became_active":"2024-03-23T14:57:41.626686+0000","last_became_peered":"2024-03-23T14:57:41.626686+0000","last_unstale":"2024-03-23T14:57:41.632591+0000","last_undegraded":"2024-03-23T14:57:41.632591+0000","last_fullsized":"2024-03-23T14:57:41.632591+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_clean_scrub_stamp":"2024-03-23T14:57:40.577321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:30:32.413411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":9,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.624238+0000","last_change":"2024-03-23T14:57:41.620728+0000","last_active":"2024-03-23T14:57:41.624238+0000","last_peered":"2024-03-23T14:57:41.624238+0000","last_clean":"2024-03-23T14:57:41.624238+0000","last_became_active":"2024-03-23T14:57:41.620289+0000","last_became_peered":"2024-03-23T14:57:41.620289+0000","last_unstale":"2024-03-23T14:57:41.624238+0000","last_undegraded":"2024-03-23T14:57:41.624238+0000","last_fullsized":"2024-03-23T14:57:41.624238+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_clean_scrub_stamp":"2024-03-23T14:57:40.577321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:36:14.704031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":9,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.624970+0000","last_change":"2024-03-23T14:57:41.618455+0000","last_active":"2024-03-23T14:57:41.624970+0000","last_peered":"2024-03-23T14:57:41.624970+0000","last_clean":"2024-03-23T14:57:41.624970+0000","last_became_active":"2024-03-23T14:57:41.617285+0000","last_became_peered":"2024-03-23T14:57:41.617285+0000","last_unstale":"2024-03-23T14:57:41.624970+0000","last_undegraded":"2024-03-23T14:57:41.624970+0000","last_fullsized":"2024-03-23T14:57:41.624970+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_clean_scrub_stamp":"2024-03-23T14:57:40.577321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:34:29.928307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":9,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.628698+0000","last_change":"2024-03-23T14:57:41.620589+0000","last_active":"2024-03-23T14:57:41.628698+0000","last_peered":"2024-03-23T14:57:41.628698+0000","last_clean":"2024-03-23T14:57:41.628698+0000","last_became_active":"2024-03-23T14:57:41.619812+0000","last_became_peered":"2024-03-23T14:57:41.619812+0000","last_unstale":"2024-03-23T14:57:41.628698+0000","last_undegraded":"2024-03-23T14:57:41.628698+0000","last_fullsized":"2024-03-23T14:57:41.628698+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_clean_scrub_stamp":"2024-03-23T14:57:40.577321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:45:22.740960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":9,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.636873+0000","last_change":"2024-03-23T14:57:41.635167+0000","last_active":"2024-03-23T14:57:41.636873+0000","last_peered":"2024-03-23T14:57:41.636873+0000","last_clean":"2024-03-23T14:57:41.636873+0000","last_became_active":"2024-03-23T14:57:41.634890+0000","last_became_peered":"2024-03-23T14:57:41.634890+0000","last_unstale":"2024-03-23T14:57:41.636873+0000","last_undegraded":"2024-03-23T14:57:41.636873+0000","last_fullsized":"2024-03-23T14:57:41.636873+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_clean_scrub_stamp":"2024-03-23T14:57:40.577321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:45:08.796108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":9,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.614400+0000","last_change":"2024-03-23T14:57:41.609841+0000","last_active":"2024-03-23T14:57:41.614400+0000","last_peered":"2024-03-23T14:57:41.614400+0000","last_clean":"2024-03-23T14:57:41.614400+0000","last_became_active":"2024-03-23T14:57:41.609019+0000","last_became_peered":"2024-03-23T14:57:41.609019+0000","last_unstale":"2024-03-23T14:57:41.614400+0000","last_undegraded":"2024-03-23T14:57:41.614400+0000","last_fullsized":"2024-03-23T14:57:41.614400+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_clean_scrub_stamp":"2024-03-23T14:57:40.577321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:20:49.856427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":9,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.641892+0000","last_change":"2024-03-23T14:57:41.637844+0000","last_active":"2024-03-23T14:57:41.641892+0000","last_peered":"2024-03-23T14:57:41.641892+0000","last_clean":"2024-03-23T14:57:41.641892+0000","last_became_active":"2024-03-23T14:57:41.637554+0000","last_became_peered":"2024-03-23T14:57:41.637554+0000","last_unstale":"2024-03-23T14:57:41.641892+0000","last_undegraded":"2024-03-23T14:57:41.641892+0000","last_fullsized":"2024-03-23T14:57:41.641892+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_clean_scrub_stamp":"2024-03-23T14:57:40.577321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:26:34.952555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":9,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.640271+0000","last_change":"2024-03-23T14:57:41.635914+0000","last_active":"2024-03-23T14:57:41.640271+0000","last_peered":"2024-03-23T14:57:41.640271+0000","last_clean":"2024-03-23T14:57:41.640271+0000","last_became_active":"2024-03-23T14:57:41.635608+0000","last_became_peered":"2024-03-23T14:57:41.635608+0000","last_unstale":"2024-03-23T14:57:41.640271+0000","last_undegraded":"2024-03-23T14:57:41.640271+0000","last_fullsized":"2024-03-23T14:57:41.640271+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_clean_scrub_stamp":"2024-03-23T14:57:40.577321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:47:38.826546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":9,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.641840+0000","last_change":"2024-03-23T14:57:41.638249+0000","last_active":"2024-03-23T14:57:41.641840+0000","last_peered":"2024-03-23T14:57:41.641840+0000","last_clean":"2024-03-23T14:57:41.641840+0000","last_became_active":"2024-03-23T14:57:41.637000+0000","last_became_peered":"2024-03-23T14:57:41.637000+0000","last_unstale":"2024-03-23T14:57:41.641840+0000","last_undegraded":"2024-03-23T14:57:41.641840+0000","last_fullsized":"2024-03-23T14:57:41.641840+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_clean_scrub_stamp":"2024-03-23T14:57:40.577321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:51:50.188772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":9,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.620928+0000","last_change":"2024-03-23T14:57:41.615815+0000","last_active":"2024-03-23T14:57:41.620928+0000","last_peered":"2024-03-23T14:57:41.620928+0000","last_clean":"2024-03-23T14:57:41.620928+0000","last_became_active":"2024-03-23T14:57:41.615129+0000","last_became_peered":"2024-03-23T14:57:41.615129+0000","last_unstale":"2024-03-23T14:57:41.620928+0000","last_undegraded":"2024-03-23T14:57:41.620928+0000","last_fullsized":"2024-03-23T14:57:41.620928+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_clean_scrub_stamp":"2024-03-23T14:57:40.577321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:51:50.188772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":9,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.631833+0000","last_change":"2024-03-23T14:57:41.627518+0000","last_active":"2024-03-23T14:57:41.631833+0000","last_peered":"2024-03-23T14:57:41.631833+0000","last_clean":"2024-03-23T14:57:41.631833+0000","last_became_active":"2024-03-23T14:57:41.626733+0000","last_became_peered":"2024-03-23T14:57:41.626733+0000","last_unstale":"2024-03-23T14:57:41.631833+0000","last_undegraded":"2024-03-23T14:57:41.631833+0000","last_fullsized":"2024-03-23T14:57:41.631833+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_clean_scrub_stamp":"2024-03-23T14:57:40.577321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:43:37.787881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":9,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.654387+0000","last_change":"2024-03-23T14:57:41.645999+0000","last_active":"2024-03-23T14:57:41.654387+0000","last_peered":"2024-03-23T14:57:41.654387+0000","last_clean":"2024-03-23T14:57:41.654387+0000","last_became_active":"2024-03-23T14:57:41.645420+0000","last_became_peered":"2024-03-23T14:57:41.645420+0000","last_unstale":"2024-03-23T14:57:41.654387+0000","last_undegraded":"2024-03-23T14:57:41.654387+0000","last_fullsized":"2024-03-23T14:57:41.654387+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_clean_scrub_stamp":"2024-03-23T14:57:40.577321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:23:11.000724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":9,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.642748+0000","last_change":"2024-03-23T14:57:41.631694+0000","last_active":"2024-03-23T14:57:41.642748+0000","last_peered":"2024-03-23T14:57:41.642748+0000","last_clean":"2024-03-23T14:57:41.642748+0000","last_became_active":"2024-03-23T14:57:41.630120+0000","last_became_peered":"2024-03-23T14:57:41.630120+0000","last_unstale":"2024-03-23T14:57:41.642748+0000","last_undegraded":"2024-03-23T14:57:41.642748+0000","last_fullsized":"2024-03-23T14:57:41.642748+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_clean_scrub_stamp":"2024-03-23T14:57:40.577321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:34:29.928307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":9,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.656790+0000","last_change":"2024-03-23T14:57:41.651632+0000","last_active":"2024-03-23T14:57:41.656790+0000","last_peered":"2024-03-23T14:57:41.656790+0000","last_clean":"2024-03-23T14:57:41.656790+0000","last_became_active":"2024-03-23T14:57:41.641761+0000","last_became_peered":"2024-03-23T14:57:41.641761+0000","last_unstale":"2024-03-23T14:57:41.656790+0000","last_undegraded":"2024-03-23T14:57:41.656790+0000","last_fullsized":"2024-03-23T14:57:41.656790+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_clean_scrub_stamp":"2024-03-23T14:57:40.577321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:30:32.413411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":9,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.641517+0000","last_change":"2024-03-23T14:57:41.636034+0000","last_active":"2024-03-23T14:57:41.641517+0000","last_peered":"2024-03-23T14:57:41.641517+0000","last_clean":"2024-03-23T14:57:41.641517+0000","last_became_active":"2024-03-23T14:57:41.635678+0000","last_became_peered":"2024-03-23T14:57:41.635678+0000","last_unstale":"2024-03-23T14:57:41.641517+0000","last_undegraded":"2024-03-23T14:57:41.641517+0000","last_fullsized":"2024-03-23T14:57:41.641517+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_clean_scrub_stamp":"2024-03-23T14:57:40.577321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:10:52.026797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"28'4","reported_seq":16,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.654169+0000","last_change":"2024-03-23T14:57:40.629717+0000","last_active":"2024-03-23T14:57:41.654169+0000","last_peered":"2024-03-23T14:57:41.654169+0000","last_clean":"2024-03-23T14:57:41.654169+0000","last_became_active":"2024-03-23T14:57:40.629001+0000","last_became_peered":"2024-03-23T14:57:40.629001+0000","last_unstale":"2024-03-23T14:57:41.654169+0000","last_undegraded":"2024-03-23T14:57:41.654169+0000","last_fullsized":"2024-03-23T14:57:41.654169+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:39.565698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:39.565698+0000","last_clean_scrub_stamp":"2024-03-23T14:57:39.565698+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:42:06.517354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":9,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.636605+0000","last_change":"2024-03-23T14:57:41.612499+0000","last_active":"2024-03-23T14:57:41.636605+0000","last_peered":"2024-03-23T14:57:41.636605+0000","last_clean":"2024-03-23T14:57:41.636605+0000","last_became_active":"2024-03-23T14:57:41.611762+0000","last_became_peered":"2024-03-23T14:57:41.611762+0000","last_unstale":"2024-03-23T14:57:41.636605+0000","last_undegraded":"2024-03-23T14:57:41.636605+0000","last_fullsized":"2024-03-23T14:57:41.636605+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_clean_scrub_stamp":"2024-03-23T14:57:40.577321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:11:44.272923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"28'1","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.656184+0000","last_change":"2024-03-23T14:57:40.614049+0000","last_active":"2024-03-23T14:57:41.656184+0000","last_peered":"2024-03-23T14:57:41.656184+0000","last_clean":"2024-03-23T14:57:41.656184+0000","last_became_active":"2024-03-23T14:57:40.613755+0000","last_became_peered":"2024-03-23T14:57:40.613755+0000","last_unstale":"2024-03-23T14:57:41.656184+0000","last_undegraded":"2024-03-23T14:57:41.656184+0000","last_fullsized":"2024-03-23T14:57:41.656184+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:39.565698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:39.565698+0000","last_clean_scrub_stamp":"2024-03-23T14:57:39.565698+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:11:43.261300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":9,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.631565+0000","last_change":"2024-03-23T14:57:41.628564+0000","last_active":"2024-03-23T14:57:41.631565+0000","last_peered":"2024-03-23T14:57:41.631565+0000","last_clean":"2024-03-23T14:57:41.631565+0000","last_became_active":"2024-03-23T14:57:41.627780+0000","last_became_peered":"2024-03-23T14:57:41.627780+0000","last_unstale":"2024-03-23T14:57:41.631565+0000","last_undegraded":"2024-03-23T14:57:41.631565+0000","last_fullsized":"2024-03-23T14:57:41.631565+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_clean_scrub_stamp":"2024-03-23T14:57:40.577321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:00:23.723071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"28'2","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.642375+0000","last_change":"2024-03-23T14:57:40.609708+0000","last_active":"2024-03-23T14:57:41.642375+0000","last_peered":"2024-03-23T14:57:41.642375+0000","last_clean":"2024-03-23T14:57:41.642375+0000","last_became_active":"2024-03-23T14:57:40.609415+0000","last_became_peered":"2024-03-23T14:57:40.609415+0000","last_unstale":"2024-03-23T14:57:41.642375+0000","last_undegraded":"2024-03-23T14:57:41.642375+0000","last_fullsized":"2024-03-23T14:57:41.642375+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:39.565698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:39.565698+0000","last_clean_scrub_stamp":"2024-03-23T14:57:39.565698+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:20:48.844804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.638512+0000","last_change":"2024-03-23T14:57:41.632207+0000","last_active":"2024-03-23T14:57:41.638512+0000","last_peered":"2024-03-23T14:57:41.638512+0000","last_clean":"2024-03-23T14:57:41.638512+0000","last_became_active":"2024-03-23T14:57:41.631893+0000","last_became_peered":"2024-03-23T14:57:41.631893+0000","last_unstale":"2024-03-23T14:57:41.638512+0000","last_undegraded":"2024-03-23T14:57:41.638512+0000","last_fullsized":"2024-03-23T14:57:41.638512+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_clean_scrub_stamp":"2024-03-23T14:57:40.577321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:10:52.026797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"28'2","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.644320+0000","last_change":"2024-03-23T14:57:40.611599+0000","last_active":"2024-03-23T14:57:41.644320+0000","last_peered":"2024-03-23T14:57:41.644320+0000","last_clean":"2024-03-23T14:57:41.644320+0000","last_became_active":"2024-03-23T14:57:40.611261+0000","last_became_peered":"2024-03-23T14:57:40.611261+0000","last_unstale":"2024-03-23T14:57:41.644320+0000","last_undegraded":"2024-03-23T14:57:41.644320+0000","last_fullsized":"2024-03-23T14:57:41.644320+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:39.565698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:39.565698+0000","last_clean_scrub_stamp":"2024-03-23T14:57:39.565698+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:13:40.745947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":9,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.654334+0000","last_change":"2024-03-23T14:57:41.634087+0000","last_active":"2024-03-23T14:57:41.654334+0000","last_peered":"2024-03-23T14:57:41.654334+0000","last_clean":"2024-03-23T14:57:41.654334+0000","last_became_active":"2024-03-23T14:57:41.633094+0000","last_became_peered":"2024-03-23T14:57:41.633094+0000","last_unstale":"2024-03-23T14:57:41.654334+0000","last_undegraded":"2024-03-23T14:57:41.654334+0000","last_fullsized":"2024-03-23T14:57:41.654334+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_clean_scrub_stamp":"2024-03-23T14:57:40.577321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:26:34.952555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"28'1","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.640376+0000","last_change":"2024-03-23T14:57:40.609143+0000","last_active":"2024-03-23T14:57:41.640376+0000","last_peered":"2024-03-23T14:57:41.640376+0000","last_clean":"2024-03-23T14:57:41.640376+0000","last_became_active":"2024-03-23T14:57:40.608564+0000","last_became_peered":"2024-03-23T14:57:40.608564+0000","last_unstale":"2024-03-23T14:57:41.640376+0000","last_undegraded":"2024-03-23T14:57:41.640376+0000","last_fullsized":"2024-03-23T14:57:41.640376+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:39.565698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:39.565698+0000","last_clean_scrub_stamp":"2024-03-23T14:57:39.565698+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:52:40.375873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.626291+0000","last_change":"2024-03-23T14:57:41.621965+0000","last_active":"2024-03-23T14:57:41.626291+0000","last_peered":"2024-03-23T14:57:41.626291+0000","last_clean":"2024-03-23T14:57:41.626291+0000","last_became_active":"2024-03-23T14:57:41.621184+0000","last_became_peered":"2024-03-23T14:57:41.621184+0000","last_unstale":"2024-03-23T14:57:41.626291+0000","last_undegraded":"2024-03-23T14:57:41.626291+0000","last_fullsized":"2024-03-23T14:57:41.626291+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_clean_scrub_stamp":"2024-03-23T14:57:40.577321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:23:11.000724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"28'1","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.640104+0000","last_change":"2024-03-23T14:57:40.605238+0000","last_active":"2024-03-23T14:57:41.640104+0000","last_peered":"2024-03-23T14:57:41.640104+0000","last_clean":"2024-03-23T14:57:41.640104+0000","last_became_active":"2024-03-23T14:57:40.604489+0000","last_became_peered":"2024-03-23T14:57:40.604489+0000","last_unstale":"2024-03-23T14:57:41.640104+0000","last_undegraded":"2024-03-23T14:57:41.640104+0000","last_fullsized":"2024-03-23T14:57:41.640104+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:39.565698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:39.565698+0000","last_clean_scrub_stamp":"2024-03-23T14:57:39.565698+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:47:38.848526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.612995+0000","last_change":"2024-03-23T14:57:41.608851+0000","last_active":"2024-03-23T14:57:41.612995+0000","last_peered":"2024-03-23T14:57:41.612995+0000","last_clean":"2024-03-23T14:57:41.612995+0000","last_became_active":"2024-03-23T14:57:41.607807+0000","last_became_peered":"2024-03-23T14:57:41.607807+0000","last_unstale":"2024-03-23T14:57:41.612995+0000","last_undegraded":"2024-03-23T14:57:41.612995+0000","last_fullsized":"2024-03-23T14:57:41.612995+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_clean_scrub_stamp":"2024-03-23T14:57:40.577321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:13:13.184800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.601841+0000","last_change":"2024-03-23T14:57:40.607129+0000","last_active":"2024-03-23T14:57:41.601841+0000","last_peered":"2024-03-23T14:57:41.601841+0000","last_clean":"2024-03-23T14:57:41.601841+0000","last_became_active":"2024-03-23T14:57:40.606666+0000","last_became_peered":"2024-03-23T14:57:40.606666+0000","last_unstale":"2024-03-23T14:57:41.601841+0000","last_undegraded":"2024-03-23T14:57:41.601841+0000","last_fullsized":"2024-03-23T14:57:41.601841+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:39.565698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:39.565698+0000","last_clean_scrub_stamp":"2024-03-23T14:57:39.565698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:00:22.711448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.625345+0000","last_change":"2024-03-23T14:57:41.622864+0000","last_active":"2024-03-23T14:57:41.625345+0000","last_peered":"2024-03-23T14:57:41.625345+0000","last_clean":"2024-03-23T14:57:41.625345+0000","last_became_active":"2024-03-23T14:57:41.622119+0000","last_became_peered":"2024-03-23T14:57:41.622119+0000","last_unstale":"2024-03-23T14:57:41.625345+0000","last_undegraded":"2024-03-23T14:57:41.625345+0000","last_fullsized":"2024-03-23T14:57:41.625345+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_clean_scrub_stamp":"2024-03-23T14:57:40.577321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:43:37.787881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.597631+0000","last_change":"2024-03-23T14:57:40.618722+0000","last_active":"2024-03-23T14:57:41.597631+0000","last_peered":"2024-03-23T14:57:41.597631+0000","last_clean":"2024-03-23T14:57:41.597631+0000","last_became_active":"2024-03-23T14:57:40.617965+0000","last_became_peered":"2024-03-23T14:57:40.617965+0000","last_unstale":"2024-03-23T14:57:41.597631+0000","last_undegraded":"2024-03-23T14:57:41.597631+0000","last_fullsized":"2024-03-23T14:57:41.597631+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:39.565698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:39.565698+0000","last_clean_scrub_stamp":"2024-03-23T14:57:39.565698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:59:12.524685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.641350+0000","last_change":"2024-03-23T14:57:41.629551+0000","last_active":"2024-03-23T14:57:41.641350+0000","last_peered":"2024-03-23T14:57:41.641350+0000","last_clean":"2024-03-23T14:57:41.641350+0000","last_became_active":"2024-03-23T14:57:41.628673+0000","last_became_peered":"2024-03-23T14:57:41.628673+0000","last_unstale":"2024-03-23T14:57:41.641350+0000","last_undegraded":"2024-03-23T14:57:41.641350+0000","last_fullsized":"2024-03-23T14:57:41.641350+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_clean_scrub_stamp":"2024-03-23T14:57:40.577321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:30:32.413411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.606349+0000","last_change":"2024-03-23T14:57:40.617570+0000","last_active":"2024-03-23T14:57:41.606349+0000","last_peered":"2024-03-23T14:57:41.606349+0000","last_clean":"2024-03-23T14:57:41.606349+0000","last_became_active":"2024-03-23T14:57:40.617234+0000","last_became_peered":"2024-03-23T14:57:40.617234+0000","last_unstale":"2024-03-23T14:57:41.606349+0000","last_undegraded":"2024-03-23T14:57:41.606349+0000","last_fullsized":"2024-03-23T14:57:41.606349+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:39.565698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:39.565698+0000","last_clean_scrub_stamp":"2024-03-23T14:57:39.565698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:51:49.177149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.632764+0000","last_change":"2024-03-23T14:57:41.626893+0000","last_active":"2024-03-23T14:57:41.632764+0000","last_peered":"2024-03-23T14:57:41.632764+0000","last_clean":"2024-03-23T14:57:41.632764+0000","last_became_active":"2024-03-23T14:57:41.626470+0000","last_became_peered":"2024-03-23T14:57:41.626470+0000","last_unstale":"2024-03-23T14:57:41.632764+0000","last_undegraded":"2024-03-23T14:57:41.632764+0000","last_fullsized":"2024-03-23T14:57:41.632764+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_clean_scrub_stamp":"2024-03-23T14:57:40.577321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:38:13.872392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.595689+0000","last_change":"2024-03-23T14:57:40.606169+0000","last_active":"2024-03-23T14:57:41.595689+0000","last_peered":"2024-03-23T14:57:41.595689+0000","last_clean":"2024-03-23T14:57:41.595689+0000","last_became_active":"2024-03-23T14:57:40.605494+0000","last_became_peered":"2024-03-23T14:57:40.605494+0000","last_unstale":"2024-03-23T14:57:41.595689+0000","last_undegraded":"2024-03-23T14:57:41.595689+0000","last_fullsized":"2024-03-23T14:57:41.595689+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:39.565698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:39.565698+0000","last_clean_scrub_stamp":"2024-03-23T14:57:39.565698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:34:28.916684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.647988+0000","last_change":"2024-03-23T14:57:41.644969+0000","last_active":"2024-03-23T14:57:41.647988+0000","last_peered":"2024-03-23T14:57:41.647988+0000","last_clean":"2024-03-23T14:57:41.647988+0000","last_became_active":"2024-03-23T14:57:41.644576+0000","last_became_peered":"2024-03-23T14:57:41.644576+0000","last_unstale":"2024-03-23T14:57:41.647988+0000","last_undegraded":"2024-03-23T14:57:41.647988+0000","last_fullsized":"2024-03-23T14:57:41.647988+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_clean_scrub_stamp":"2024-03-23T14:57:40.577321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:21:33.715027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"28'3","reported_seq":15,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.657062+0000","last_change":"2024-03-23T14:57:40.626790+0000","last_active":"2024-03-23T14:57:41.657062+0000","last_peered":"2024-03-23T14:57:41.657062+0000","last_clean":"2024-03-23T14:57:41.657062+0000","last_became_active":"2024-03-23T14:57:40.626120+0000","last_became_peered":"2024-03-23T14:57:40.626120+0000","last_unstale":"2024-03-23T14:57:41.657062+0000","last_undegraded":"2024-03-23T14:57:41.657062+0000","last_fullsized":"2024-03-23T14:57:41.657062+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:39.565698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:39.565698+0000","last_clean_scrub_stamp":"2024-03-23T14:57:39.565698+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:10:51.015174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.639553+0000","last_change":"2024-03-23T14:57:41.626456+0000","last_active":"2024-03-23T14:57:41.639553+0000","last_peered":"2024-03-23T14:57:41.639553+0000","last_clean":"2024-03-23T14:57:41.639553+0000","last_became_active":"2024-03-23T14:57:41.625721+0000","last_became_peered":"2024-03-23T14:57:41.625721+0000","last_unstale":"2024-03-23T14:57:41.639553+0000","last_undegraded":"2024-03-23T14:57:41.639553+0000","last_fullsized":"2024-03-23T14:57:41.639553+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_clean_scrub_stamp":"2024-03-23T14:57:40.577321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:05:34.469230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"28'6","reported_seq":18,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.650937+0000","last_change":"2024-03-23T14:57:40.614043+0000","last_active":"2024-03-23T14:57:41.650937+0000","last_peered":"2024-03-23T14:57:41.650937+0000","last_clean":"2024-03-23T14:57:41.650937+0000","last_became_active":"2024-03-23T14:57:40.613431+0000","last_became_peered":"2024-03-23T14:57:40.613431+0000","last_unstale":"2024-03-23T14:57:41.650937+0000","last_undegraded":"2024-03-23T14:57:41.650937+0000","last_fullsized":"2024-03-23T14:57:41.650937+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:39.565698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:39.565698+0000","last_clean_scrub_stamp":"2024-03-23T14:57:39.565698+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:26:33.940932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.633363+0000","last_change":"2024-03-23T14:57:41.628389+0000","last_active":"2024-03-23T14:57:41.633363+0000","last_peered":"2024-03-23T14:57:41.633363+0000","last_clean":"2024-03-23T14:57:41.633363+0000","last_became_active":"2024-03-23T14:57:41.627873+0000","last_became_peered":"2024-03-23T14:57:41.627873+0000","last_unstale":"2024-03-23T14:57:41.633363+0000","last_undegraded":"2024-03-23T14:57:41.633363+0000","last_fullsized":"2024-03-23T14:57:41.633363+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_clean_scrub_stamp":"2024-03-23T14:57:40.577321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:08:47.903767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"28'2","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.649222+0000","last_change":"2024-03-23T14:57:40.617362+0000","last_active":"2024-03-23T14:57:41.649222+0000","last_peered":"2024-03-23T14:57:41.649222+0000","last_clean":"2024-03-23T14:57:41.649222+0000","last_became_active":"2024-03-23T14:57:40.616297+0000","last_became_peered":"2024-03-23T14:57:40.616297+0000","last_unstale":"2024-03-23T14:57:41.649222+0000","last_undegraded":"2024-03-23T14:57:41.649222+0000","last_fullsized":"2024-03-23T14:57:41.649222+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:39.565698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:39.565698+0000","last_clean_scrub_stamp":"2024-03-23T14:57:39.565698+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:54:36.135171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.627797+0000","last_change":"2024-03-23T14:57:41.621608+0000","last_active":"2024-03-23T14:57:41.627797+0000","last_peered":"2024-03-23T14:57:41.627797+0000","last_clean":"2024-03-23T14:57:41.627797+0000","last_became_active":"2024-03-23T14:57:41.621255+0000","last_became_peered":"2024-03-23T14:57:41.621255+0000","last_unstale":"2024-03-23T14:57:41.627797+0000","last_undegraded":"2024-03-23T14:57:41.627797+0000","last_fullsized":"2024-03-23T14:57:41.627797+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_clean_scrub_stamp":"2024-03-23T14:57:40.577321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:51:58.286721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.620829+0000","last_change":"2024-03-23T14:57:40.610401+0000","last_active":"2024-03-23T14:57:41.620829+0000","last_peered":"2024-03-23T14:57:41.620829+0000","last_clean":"2024-03-23T14:57:41.620829+0000","last_became_active":"2024-03-23T14:57:40.609051+0000","last_became_peered":"2024-03-23T14:57:40.609051+0000","last_unstale":"2024-03-23T14:57:41.620829+0000","last_undegraded":"2024-03-23T14:57:41.620829+0000","last_fullsized":"2024-03-23T14:57:41.620829+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:39.565698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:39.565698+0000","last_clean_scrub_stamp":"2024-03-23T14:57:39.565698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:13:12.173177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":9,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.656871+0000","last_change":"2024-03-23T14:57:41.652563+0000","last_active":"2024-03-23T14:57:41.656871+0000","last_peered":"2024-03-23T14:57:41.656871+0000","last_clean":"2024-03-23T14:57:41.656871+0000","last_became_active":"2024-03-23T14:57:41.645218+0000","last_became_peered":"2024-03-23T14:57:41.645218+0000","last_unstale":"2024-03-23T14:57:41.656871+0000","last_undegraded":"2024-03-23T14:57:41.656871+0000","last_fullsized":"2024-03-23T14:57:41.656871+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_clean_scrub_stamp":"2024-03-23T14:57:40.577321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:39:56.285297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.602267+0000","last_change":"2024-03-23T14:57:40.617115+0000","last_active":"2024-03-23T14:57:41.602267+0000","last_peered":"2024-03-23T14:57:41.602267+0000","last_clean":"2024-03-23T14:57:41.602267+0000","last_became_active":"2024-03-23T14:57:40.616481+0000","last_became_peered":"2024-03-23T14:57:40.616481+0000","last_unstale":"2024-03-23T14:57:41.602267+0000","last_undegraded":"2024-03-23T14:57:41.602267+0000","last_fullsized":"2024-03-23T14:57:41.602267+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:39.565698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:39.565698+0000","last_clean_scrub_stamp":"2024-03-23T14:57:39.565698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:45:07.784485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.642817+0000","last_change":"2024-03-23T14:57:41.631932+0000","last_active":"2024-03-23T14:57:41.642817+0000","last_peered":"2024-03-23T14:57:41.642817+0000","last_clean":"2024-03-23T14:57:41.642817+0000","last_became_active":"2024-03-23T14:57:41.630993+0000","last_became_peered":"2024-03-23T14:57:41.630993+0000","last_unstale":"2024-03-23T14:57:41.642817+0000","last_undegraded":"2024-03-23T14:57:41.642817+0000","last_fullsized":"2024-03-23T14:57:41.642817+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_clean_scrub_stamp":"2024-03-23T14:57:40.577321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:45:22.740960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"28'8","reported_seq":20,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.646408+0000","last_change":"2024-03-23T14:57:40.609730+0000","last_active":"2024-03-23T14:57:41.646408+0000","last_peered":"2024-03-23T14:57:41.646408+0000","last_clean":"2024-03-23T14:57:41.646408+0000","last_became_active":"2024-03-23T14:57:40.609324+0000","last_became_peered":"2024-03-23T14:57:40.609324+0000","last_unstale":"2024-03-23T14:57:41.646408+0000","last_undegraded":"2024-03-23T14:57:41.646408+0000","last_fullsized":"2024-03-23T14:57:41.646408+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:39.565698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:39.565698+0000","last_clean_scrub_stamp":"2024-03-23T14:57:39.565698+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:08:46.892145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":9,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.619427+0000","last_change":"2024-03-23T14:57:41.615444+0000","last_active":"2024-03-23T14:57:41.619427+0000","last_peered":"2024-03-23T14:57:41.619427+0000","last_clean":"2024-03-23T14:57:41.619427+0000","last_became_active":"2024-03-23T14:57:41.614722+0000","last_became_peered":"2024-03-23T14:57:41.614722+0000","last_unstale":"2024-03-23T14:57:41.619427+0000","last_undegraded":"2024-03-23T14:57:41.619427+0000","last_fullsized":"2024-03-23T14:57:41.619427+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_clean_scrub_stamp":"2024-03-23T14:57:40.577321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:11:44.272923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"28'1","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.641958+0000","last_change":"2024-03-23T14:57:40.610732+0000","last_active":"2024-03-23T14:57:41.641958+0000","last_peered":"2024-03-23T14:57:41.641958+0000","last_clean":"2024-03-23T14:57:41.641958+0000","last_became_active":"2024-03-23T14:57:40.610463+0000","last_became_peered":"2024-03-23T14:57:40.610463+0000","last_unstale":"2024-03-23T14:57:41.641958+0000","last_undegraded":"2024-03-23T14:57:41.641958+0000","last_fullsized":"2024-03-23T14:57:41.641958+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:39.565698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:39.565698+0000","last_clean_scrub_stamp":"2024-03-23T14:57:39.565698+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:45:21.729337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.617800+0000","last_change":"2024-03-23T14:57:41.612679+0000","last_active":"2024-03-23T14:57:41.617800+0000","last_peered":"2024-03-23T14:57:41.617800+0000","last_clean":"2024-03-23T14:57:41.617800+0000","last_became_active":"2024-03-23T14:57:41.611947+0000","last_became_peered":"2024-03-23T14:57:41.611947+0000","last_unstale":"2024-03-23T14:57:41.617800+0000","last_undegraded":"2024-03-23T14:57:41.617800+0000","last_fullsized":"2024-03-23T14:57:41.617800+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_clean_scrub_stamp":"2024-03-23T14:57:40.577321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:26:34.952555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"28'2","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.647865+0000","last_change":"2024-03-23T14:57:40.618151+0000","last_active":"2024-03-23T14:57:41.647865+0000","last_peered":"2024-03-23T14:57:41.647865+0000","last_clean":"2024-03-23T14:57:41.647865+0000","last_became_active":"2024-03-23T14:57:40.617379+0000","last_became_peered":"2024-03-23T14:57:40.617379+0000","last_unstale":"2024-03-23T14:57:41.647865+0000","last_undegraded":"2024-03-23T14:57:41.647865+0000","last_fullsized":"2024-03-23T14:57:41.647865+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:39.565698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:39.565698+0000","last_clean_scrub_stamp":"2024-03-23T14:57:39.565698+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:02:53.716281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":9,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.625491+0000","last_change":"2024-03-23T14:57:41.623624+0000","last_active":"2024-03-23T14:57:41.625491+0000","last_peered":"2024-03-23T14:57:41.625491+0000","last_clean":"2024-03-23T14:57:41.625491+0000","last_became_active":"2024-03-23T14:57:41.623311+0000","last_became_peered":"2024-03-23T14:57:41.623311+0000","last_unstale":"2024-03-23T14:57:41.625491+0000","last_undegraded":"2024-03-23T14:57:41.625491+0000","last_fullsized":"2024-03-23T14:57:41.625491+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_clean_scrub_stamp":"2024-03-23T14:57:40.577321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:00:23.723071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"28'1","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.643547+0000","last_change":"2024-03-23T14:57:40.619598+0000","last_active":"2024-03-23T14:57:41.643547+0000","last_peered":"2024-03-23T14:57:41.643547+0000","last_clean":"2024-03-23T14:57:41.643547+0000","last_became_active":"2024-03-23T14:57:40.618877+0000","last_became_peered":"2024-03-23T14:57:40.618877+0000","last_unstale":"2024-03-23T14:57:41.643547+0000","last_undegraded":"2024-03-23T14:57:41.643547+0000","last_fullsized":"2024-03-23T14:57:41.643547+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:39.565698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:39.565698+0000","last_clean_scrub_stamp":"2024-03-23T14:57:39.565698+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:35:11.065518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":9,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.644136+0000","last_change":"2024-03-23T14:57:41.636716+0000","last_active":"2024-03-23T14:57:41.644136+0000","last_peered":"2024-03-23T14:57:41.644136+0000","last_clean":"2024-03-23T14:57:41.644136+0000","last_became_active":"2024-03-23T14:57:41.636390+0000","last_became_peered":"2024-03-23T14:57:41.636390+0000","last_unstale":"2024-03-23T14:57:41.644136+0000","last_undegraded":"2024-03-23T14:57:41.644136+0000","last_fullsized":"2024-03-23T14:57:41.644136+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_clean_scrub_stamp":"2024-03-23T14:57:40.577321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:35:12.077141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.599643+0000","last_change":"2024-03-23T14:57:40.609459+0000","last_active":"2024-03-23T14:57:41.599643+0000","last_peered":"2024-03-23T14:57:41.599643+0000","last_clean":"2024-03-23T14:57:41.599643+0000","last_became_active":"2024-03-23T14:57:40.608715+0000","last_became_peered":"2024-03-23T14:57:40.608715+0000","last_unstale":"2024-03-23T14:57:41.599643+0000","last_undegraded":"2024-03-23T14:57:41.599643+0000","last_fullsized":"2024-03-23T14:57:41.599643+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:39.565698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:39.565698+0000","last_clean_scrub_stamp":"2024-03-23T14:57:39.565698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:21:57.359199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.622137+0000","last_change":"2024-03-23T14:57:41.619956+0000","last_active":"2024-03-23T14:57:41.622137+0000","last_peered":"2024-03-23T14:57:41.622137+0000","last_clean":"2024-03-23T14:57:41.622137+0000","last_became_active":"2024-03-23T14:57:41.619572+0000","last_became_peered":"2024-03-23T14:57:41.619572+0000","last_unstale":"2024-03-23T14:57:41.622137+0000","last_undegraded":"2024-03-23T14:57:41.622137+0000","last_fullsized":"2024-03-23T14:57:41.622137+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_clean_scrub_stamp":"2024-03-23T14:57:40.577321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:40:04.045827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"28'2","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.644029+0000","last_change":"2024-03-23T14:57:40.609969+0000","last_active":"2024-03-23T14:57:41.644029+0000","last_peered":"2024-03-23T14:57:41.644029+0000","last_clean":"2024-03-23T14:57:41.644029+0000","last_became_active":"2024-03-23T14:57:40.609479+0000","last_became_peered":"2024-03-23T14:57:40.609479+0000","last_unstale":"2024-03-23T14:57:41.644029+0000","last_undegraded":"2024-03-23T14:57:41.644029+0000","last_fullsized":"2024-03-23T14:57:41.644029+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:39.565698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:39.565698+0000","last_clean_scrub_stamp":"2024-03-23T14:57:39.565698+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:28:31.359748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":9,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.624903+0000","last_change":"2024-03-23T14:57:41.623008+0000","last_active":"2024-03-23T14:57:41.624903+0000","last_peered":"2024-03-23T14:57:41.624903+0000","last_clean":"2024-03-23T14:57:41.624903+0000","last_became_active":"2024-03-23T14:57:41.622712+0000","last_became_peered":"2024-03-23T14:57:41.622712+0000","last_unstale":"2024-03-23T14:57:41.624903+0000","last_undegraded":"2024-03-23T14:57:41.624903+0000","last_fullsized":"2024-03-23T14:57:41.624903+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_clean_scrub_stamp":"2024-03-23T14:57:40.577321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:36:08.323317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.603841+0000","last_change":"2024-03-23T14:57:40.614136+0000","last_active":"2024-03-23T14:57:41.603841+0000","last_peered":"2024-03-23T14:57:41.603841+0000","last_clean":"2024-03-23T14:57:41.603841+0000","last_became_active":"2024-03-23T14:57:40.612207+0000","last_became_peered":"2024-03-23T14:57:40.612207+0000","last_unstale":"2024-03-23T14:57:41.603841+0000","last_undegraded":"2024-03-23T14:57:41.603841+0000","last_fullsized":"2024-03-23T14:57:41.603841+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:39.565698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:39.565698+0000","last_clean_scrub_stamp":"2024-03-23T14:57:39.565698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:13:12.173177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.619076+0000","last_change":"2024-03-23T14:57:41.616704+0000","last_active":"2024-03-23T14:57:41.619076+0000","last_peered":"2024-03-23T14:57:41.619076+0000","last_clean":"2024-03-23T14:57:41.619076+0000","last_became_active":"2024-03-23T14:57:41.616442+0000","last_became_peered":"2024-03-23T14:57:41.616442+0000","last_unstale":"2024-03-23T14:57:41.619076+0000","last_undegraded":"2024-03-23T14:57:41.619076+0000","last_fullsized":"2024-03-23T14:57:41.619076+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_clean_scrub_stamp":"2024-03-23T14:57:40.577321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:57:48.675270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.600707+0000","last_change":"2024-03-23T14:57:40.620064+0000","last_active":"2024-03-23T14:57:41.600707+0000","last_peered":"2024-03-23T14:57:41.600707+0000","last_clean":"2024-03-23T14:57:41.600707+0000","last_became_active":"2024-03-23T14:57:40.619157+0000","last_became_peered":"2024-03-23T14:57:40.619157+0000","last_unstale":"2024-03-23T14:57:41.600707+0000","last_undegraded":"2024-03-23T14:57:41.600707+0000","last_fullsized":"2024-03-23T14:57:41.600707+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:39.565698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:39.565698+0000","last_clean_scrub_stamp":"2024-03-23T14:57:39.565698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:07:15.676075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.656712+0000","last_change":"2024-03-23T14:57:41.650655+0000","last_active":"2024-03-23T14:57:41.656712+0000","last_peered":"2024-03-23T14:57:41.656712+0000","last_clean":"2024-03-23T14:57:41.656712+0000","last_became_active":"2024-03-23T14:57:41.649963+0000","last_became_peered":"2024-03-23T14:57:41.649963+0000","last_unstale":"2024-03-23T14:57:41.656712+0000","last_undegraded":"2024-03-23T14:57:41.656712+0000","last_fullsized":"2024-03-23T14:57:41.656712+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_clean_scrub_stamp":"2024-03-23T14:57:40.577321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:05:34.469230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.595509+0000","last_change":"2024-03-23T14:57:40.626882+0000","last_active":"2024-03-23T14:57:41.595509+0000","last_peered":"2024-03-23T14:57:41.595509+0000","last_clean":"2024-03-23T14:57:41.595509+0000","last_became_active":"2024-03-23T14:57:40.626002+0000","last_became_peered":"2024-03-23T14:57:40.626002+0000","last_unstale":"2024-03-23T14:57:41.595509+0000","last_undegraded":"2024-03-23T14:57:41.595509+0000","last_fullsized":"2024-03-23T14:57:41.595509+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:39.565698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:39.565698+0000","last_clean_scrub_stamp":"2024-03-23T14:57:39.565698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:45:07.784485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.624283+0000","last_change":"2024-03-23T14:57:41.620475+0000","last_active":"2024-03-23T14:57:41.624283+0000","last_peered":"2024-03-23T14:57:41.624283+0000","last_clean":"2024-03-23T14:57:41.624283+0000","last_became_active":"2024-03-23T14:57:41.620213+0000","last_became_peered":"2024-03-23T14:57:41.620213+0000","last_unstale":"2024-03-23T14:57:41.624283+0000","last_undegraded":"2024-03-23T14:57:41.624283+0000","last_fullsized":"2024-03-23T14:57:41.624283+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_clean_scrub_stamp":"2024-03-23T14:57:40.577321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:53:18.316948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"28'2","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.644848+0000","last_change":"2024-03-23T14:57:40.626978+0000","last_active":"2024-03-23T14:57:41.644848+0000","last_peered":"2024-03-23T14:57:41.644848+0000","last_clean":"2024-03-23T14:57:41.644848+0000","last_became_active":"2024-03-23T14:57:40.626675+0000","last_became_peered":"2024-03-23T14:57:40.626675+0000","last_unstale":"2024-03-23T14:57:41.644848+0000","last_undegraded":"2024-03-23T14:57:41.644848+0000","last_fullsized":"2024-03-23T14:57:41.644848+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:39.565698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:39.565698+0000","last_clean_scrub_stamp":"2024-03-23T14:57:39.565698+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:28:31.359748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.639333+0000","last_change":"2024-03-23T14:57:41.624746+0000","last_active":"2024-03-23T14:57:41.639333+0000","last_peered":"2024-03-23T14:57:41.639333+0000","last_clean":"2024-03-23T14:57:41.639333+0000","last_became_active":"2024-03-23T14:57:41.624042+0000","last_became_peered":"2024-03-23T14:57:41.624042+0000","last_unstale":"2024-03-23T14:57:41.639333+0000","last_undegraded":"2024-03-23T14:57:41.639333+0000","last_fullsized":"2024-03-23T14:57:41.639333+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_clean_scrub_stamp":"2024-03-23T14:57:40.577321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:51:58.286721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.596083+0000","last_change":"2024-03-23T14:57:40.611199+0000","last_active":"2024-03-23T14:57:41.596083+0000","last_peered":"2024-03-23T14:57:41.596083+0000","last_clean":"2024-03-23T14:57:41.596083+0000","last_became_active":"2024-03-23T14:57:40.610826+0000","last_became_peered":"2024-03-23T14:57:40.610826+0000","last_unstale":"2024-03-23T14:57:41.596083+0000","last_undegraded":"2024-03-23T14:57:41.596083+0000","last_fullsized":"2024-03-23T14:57:41.596083+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:39.565698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:39.565698+0000","last_clean_scrub_stamp":"2024-03-23T14:57:39.565698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:45:21.729337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.625189+0000","last_change":"2024-03-23T14:57:41.623743+0000","last_active":"2024-03-23T14:57:41.625189+0000","last_peered":"2024-03-23T14:57:41.625189+0000","last_clean":"2024-03-23T14:57:41.625189+0000","last_became_active":"2024-03-23T14:57:41.623409+0000","last_became_peered":"2024-03-23T14:57:41.623409+0000","last_unstale":"2024-03-23T14:57:41.625189+0000","last_undegraded":"2024-03-23T14:57:41.625189+0000","last_fullsized":"2024-03-23T14:57:41.625189+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_clean_scrub_stamp":"2024-03-23T14:57:40.577321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:00:23.723071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.621474+0000","last_change":"2024-03-23T14:57:40.610710+0000","last_active":"2024-03-23T14:57:41.621474+0000","last_peered":"2024-03-23T14:57:41.621474+0000","last_clean":"2024-03-23T14:57:41.621474+0000","last_became_active":"2024-03-23T14:57:40.610229+0000","last_became_peered":"2024-03-23T14:57:40.610229+0000","last_unstale":"2024-03-23T14:57:41.621474+0000","last_undegraded":"2024-03-23T14:57:41.621474+0000","last_fullsized":"2024-03-23T14:57:41.621474+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:39.565698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:39.565698+0000","last_clean_scrub_stamp":"2024-03-23T14:57:39.565698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:11:43.261300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.636622+0000","last_change":"2024-03-23T14:57:41.632119+0000","last_active":"2024-03-23T14:57:41.636622+0000","last_peered":"2024-03-23T14:57:41.636622+0000","last_clean":"2024-03-23T14:57:41.636622+0000","last_became_active":"2024-03-23T14:57:41.631260+0000","last_became_peered":"2024-03-23T14:57:41.631260+0000","last_unstale":"2024-03-23T14:57:41.636622+0000","last_undegraded":"2024-03-23T14:57:41.636622+0000","last_fullsized":"2024-03-23T14:57:41.636622+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_clean_scrub_stamp":"2024-03-23T14:57:40.577321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:28:32.371371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.596847+0000","last_change":"2024-03-23T14:57:40.624269+0000","last_active":"2024-03-23T14:57:41.596847+0000","last_peered":"2024-03-23T14:57:41.596847+0000","last_clean":"2024-03-23T14:57:41.596847+0000","last_became_active":"2024-03-23T14:57:40.623914+0000","last_became_peered":"2024-03-23T14:57:40.623914+0000","last_unstale":"2024-03-23T14:57:41.596847+0000","last_undegraded":"2024-03-23T14:57:41.596847+0000","last_fullsized":"2024-03-23T14:57:41.596847+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:39.565698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:39.565698+0000","last_clean_scrub_stamp":"2024-03-23T14:57:39.565698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:43:36.776258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.636705+0000","last_change":"2024-03-23T14:57:41.633901+0000","last_active":"2024-03-23T14:57:41.636705+0000","last_peered":"2024-03-23T14:57:41.636705+0000","last_clean":"2024-03-23T14:57:41.636705+0000","last_became_active":"2024-03-23T14:57:41.633566+0000","last_became_peered":"2024-03-23T14:57:41.633566+0000","last_unstale":"2024-03-23T14:57:41.636705+0000","last_undegraded":"2024-03-23T14:57:41.636705+0000","last_fullsized":"2024-03-23T14:57:41.636705+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_clean_scrub_stamp":"2024-03-23T14:57:40.577321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:08:47.903767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.621544+0000","last_change":"2024-03-23T14:57:40.624317+0000","last_active":"2024-03-23T14:57:41.621544+0000","last_peered":"2024-03-23T14:57:41.621544+0000","last_clean":"2024-03-23T14:57:41.621544+0000","last_became_active":"2024-03-23T14:57:40.623908+0000","last_became_peered":"2024-03-23T14:57:40.623908+0000","last_unstale":"2024-03-23T14:57:41.621544+0000","last_undegraded":"2024-03-23T14:57:41.621544+0000","last_fullsized":"2024-03-23T14:57:41.621544+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:39.565698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:39.565698+0000","last_clean_scrub_stamp":"2024-03-23T14:57:39.565698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:34:28.916684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.632669+0000","last_change":"2024-03-23T14:57:41.627317+0000","last_active":"2024-03-23T14:57:41.632669+0000","last_peered":"2024-03-23T14:57:41.632669+0000","last_clean":"2024-03-23T14:57:41.632669+0000","last_became_active":"2024-03-23T14:57:41.626801+0000","last_became_peered":"2024-03-23T14:57:41.626801+0000","last_unstale":"2024-03-23T14:57:41.632669+0000","last_undegraded":"2024-03-23T14:57:41.632669+0000","last_fullsized":"2024-03-23T14:57:41.632669+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_clean_scrub_stamp":"2024-03-23T14:57:40.577321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:44:02.215207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.601104+0000","last_change":"2024-03-23T14:57:40.626506+0000","last_active":"2024-03-23T14:57:41.601104+0000","last_peered":"2024-03-23T14:57:41.601104+0000","last_clean":"2024-03-23T14:57:41.601104+0000","last_became_active":"2024-03-23T14:57:40.625694+0000","last_became_peered":"2024-03-23T14:57:40.625694+0000","last_unstale":"2024-03-23T14:57:41.601104+0000","last_undegraded":"2024-03-23T14:57:41.601104+0000","last_fullsized":"2024-03-23T14:57:41.601104+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:39.565698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:39.565698+0000","last_clean_scrub_stamp":"2024-03-23T14:57:39.565698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:42:06.517354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.637890+0000","last_change":"2024-03-23T14:57:41.630440+0000","last_active":"2024-03-23T14:57:41.637890+0000","last_peered":"2024-03-23T14:57:41.637890+0000","last_clean":"2024-03-23T14:57:41.637890+0000","last_became_active":"2024-03-23T14:57:41.629576+0000","last_became_peered":"2024-03-23T14:57:41.629576+0000","last_unstale":"2024-03-23T14:57:41.637890+0000","last_undegraded":"2024-03-23T14:57:41.637890+0000","last_fullsized":"2024-03-23T14:57:41.637890+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_clean_scrub_stamp":"2024-03-23T14:57:40.577321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:08:47.903767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.602536+0000","last_change":"2024-03-23T14:57:40.619843+0000","last_active":"2024-03-23T14:57:41.602536+0000","last_peered":"2024-03-23T14:57:41.602536+0000","last_clean":"2024-03-23T14:57:41.602536+0000","last_became_active":"2024-03-23T14:57:40.619510+0000","last_became_peered":"2024-03-23T14:57:40.619510+0000","last_unstale":"2024-03-23T14:57:41.602536+0000","last_undegraded":"2024-03-23T14:57:41.602536+0000","last_fullsized":"2024-03-23T14:57:41.602536+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:39.565698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:39.565698+0000","last_clean_scrub_stamp":"2024-03-23T14:57:39.565698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:10:51.015174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.654526+0000","last_change":"2024-03-23T14:57:41.650093+0000","last_active":"2024-03-23T14:57:41.654526+0000","last_peered":"2024-03-23T14:57:41.654526+0000","last_clean":"2024-03-23T14:57:41.654526+0000","last_became_active":"2024-03-23T14:57:41.641533+0000","last_became_peered":"2024-03-23T14:57:41.641533+0000","last_unstale":"2024-03-23T14:57:41.654526+0000","last_undegraded":"2024-03-23T14:57:41.654526+0000","last_fullsized":"2024-03-23T14:57:41.654526+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_clean_scrub_stamp":"2024-03-23T14:57:40.577321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:51:58.286721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.602934+0000","last_change":"2024-03-23T14:57:40.627054+0000","last_active":"2024-03-23T14:57:41.602934+0000","last_peered":"2024-03-23T14:57:41.602934+0000","last_clean":"2024-03-23T14:57:41.602934+0000","last_became_active":"2024-03-23T14:57:40.626376+0000","last_became_peered":"2024-03-23T14:57:40.626376+0000","last_unstale":"2024-03-23T14:57:41.602934+0000","last_undegraded":"2024-03-23T14:57:41.602934+0000","last_fullsized":"2024-03-23T14:57:41.602934+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:39.565698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:39.565698+0000","last_clean_scrub_stamp":"2024-03-23T14:57:39.565698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:07:15.676075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":185,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.615973+0000","last_change":"2024-03-23T14:56:48.722063+0000","last_active":"2024-03-23T14:57:41.615973+0000","last_peered":"2024-03-23T14:57:41.615973+0000","last_clean":"2024-03-23T14:57:41.615973+0000","last_became_active":"2024-03-23T14:56:48.720079+0000","last_became_peered":"2024-03-23T14:56:48.720079+0000","last_unstale":"2024-03-23T14:57:41.615973+0000","last_undegraded":"2024-03-23T14:57:41.615973+0000","last_fullsized":"2024-03-23T14:57:41.615973+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:47.478078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:47.478078+0000","last_clean_scrub_stamp":"2024-03-23T14:56:47.478078+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:55:51.231901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574863,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27896,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343944,"statfs":{"total":96636764160,"available":96608198656,"internally_reserved":0,"allocated":892928,"data_stored":701305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574863,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27332,"kb_used_data":312,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344508,"statfs":{"total":96636764160,"available":96608776192,"internally_reserved":0,"allocated":319488,"data_stored":118328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574863,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27316,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344524,"statfs":{"total":96636764160,"available":96608792576,"internally_reserved":0,"allocated":299008,"data_stored":110937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574863,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27332,"kb_used_data":312,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344508,"statfs":{"total":96636764160,"available":96608776192,"internally_reserved":0,"allocated":319488,"data_stored":117816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574864,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27340,"kb_used_data":320,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344500,"statfs":{"total":96636764160,"available":96608768000,"internally_reserved":0,"allocated":327680,"data_stored":119346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574863,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27328,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344512,"statfs":{"total":96636764160,"available":96608780288,"internally_reserved":0,"allocated":315392,"data_stored":117782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574864,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27316,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344524,"statfs":{"total":96636764160,"available":96608792576,"internally_reserved":0,"allocated":299008,"data_stored":110937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574862,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27912,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343928,"statfs":{"total":96636764160,"available":96608182272,"internally_reserved":0,"allocated":905216,"data_stored":708080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-23T14:57:44.970 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-23T14:57:45.117 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.113+0000 7fd23707d640 1 Processor -- start 2024-03-23T14:57:45.118 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.113+0000 7fd23707d640 1 -- start start 2024-03-23T14:57:45.119 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.113+0000 7fd23707d640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd23009def0 0x7fd2300b45e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:45.120 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.113+0000 7fd23707d640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fd2300add50 0x7fd23006bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:45.120 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.113+0000 7fd23707d640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd23006d650 con 0x7fd2300b4b20 2024-03-23T14:57:45.120 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.113+0000 7fd23707d640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fd2300af4a0 con 0x7fd2300add50 2024-03-23T14:57:45.120 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.113+0000 7fd23707d640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fd23009e2d0 con 0x7fd23009def0 2024-03-23T14:57:45.120 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.113+0000 7fd22ffff640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fd2300add50 0x7fd23006bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:45.120 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.113+0000 7fd22ffff640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fd2300add50 0x7fd23006bf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:44718/0 (socket says 172.21.15.164:44718) 2024-03-23T14:57:45.120 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.113+0000 7fd22ffff640 1 -- 172.21.15.164:0/782298933 learned_addr learned my addr 172.21.15.164:0/782298933 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:45.120 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.113+0000 7fd234e19640 1 --2- 172.21.15.164:0/782298933 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd23009def0 0x7fd2300b45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:45.120 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.113+0000 7fd22ffff640 1 -- 172.21.15.164:0/782298933 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd23009def0 msgr2=0x7fd2300b45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:45.120 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.113+0000 7fd22ffff640 1 --2- 172.21.15.164:0/782298933 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd23009def0 0x7fd2300b45e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:45.120 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.113+0000 7fd22ffff640 1 -- 172.21.15.164:0/782298933 >> v1:172.21.15.173:6789/0 conn(0x7fd2300b4b20 legacy=0x7fd23006c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:45.120 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.113+0000 7fd22ffff640 1 -- 172.21.15.164:0/782298933 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2300aeca0 con 0x7fd2300add50 2024-03-23T14:57:45.122 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.117+0000 7fd22ffff640 1 --2- 172.21.15.164:0/782298933 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd2300add50 0x7fd23006bf70 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fd21c009870 tx=0x7fd21c01c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d4da05b7180f5b9 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:45.122 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.117+0000 7fd22f7fe640 1 -- 172.21.15.164:0/782298933 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd21c0ac600 con 0x7fd2300add50 2024-03-23T14:57:45.122 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.117+0000 7fd22f7fe640 1 -- 172.21.15.164:0/782298933 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd21c0a8050 con 0x7fd2300add50 2024-03-23T14:57:45.123 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.117+0000 7fd22f7fe640 1 -- 172.21.15.164:0/782298933 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd21c0ac980 con 0x7fd2300add50 2024-03-23T14:57:45.123 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.117+0000 7fd23707d640 1 -- 172.21.15.164:0/782298933 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd2300add50 msgr2=0x7fd23006bf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:45.123 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.117+0000 7fd23707d640 1 --2- 172.21.15.164:0/782298933 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd2300add50 0x7fd23006bf70 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fd21c009870 tx=0x7fd21c01c440 comp rx=0 tx=0).stop 2024-03-23T14:57:45.124 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.117+0000 7fd23707d640 1 -- 172.21.15.164:0/782298933 shutdown_connections 2024-03-23T14:57:45.124 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.117+0000 7fd23707d640 1 --2- 172.21.15.164:0/782298933 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd2300add50 0x7fd23006bf70 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:45.124 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.117+0000 7fd23707d640 1 --2- 172.21.15.164:0/782298933 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd23009def0 0x7fd2300b45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:45.124 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.117+0000 7fd23707d640 1 -- 172.21.15.164:0/782298933 >> 172.21.15.164:0/782298933 conn(0x7fd2300b8ca0 msgr2=0x7fd2300b90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:45.124 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.117+0000 7fd23707d640 1 -- 172.21.15.164:0/782298933 shutdown_connections 2024-03-23T14:57:45.124 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.117+0000 7fd23707d640 1 -- 172.21.15.164:0/782298933 wait complete. 2024-03-23T14:57:45.126 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.121+0000 7fd23707d640 1 Processor -- start 2024-03-23T14:57:45.126 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.121+0000 7fd23707d640 1 -- start start 2024-03-23T14:57:45.126 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.121+0000 7fd23707d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd23009def0 0x7fd2300b7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:45.126 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.121+0000 7fd23707d640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd2300add50 0x7fd2301ddec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:45.126 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.121+0000 7fd23707d640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd2300b4b20 0x7fd2301de400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:45.126 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.121+0000 7fd23707d640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd230146880 con 0x7fd23009def0 2024-03-23T14:57:45.126 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.121+0000 7fd23707d640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd230141940 con 0x7fd2300add50 2024-03-23T14:57:45.127 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.121+0000 7fd23707d640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fd230146d40 con 0x7fd2300b4b20 2024-03-23T14:57:45.127 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.121+0000 7fd234e19640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd23009def0 0x7fd2300b7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:45.127 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.121+0000 7fd234e19640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd23009def0 0x7fd2300b7cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:44734/0 (socket says 172.21.15.164:44734) 2024-03-23T14:57:45.127 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.121+0000 7fd234e19640 1 -- 172.21.15.164:0/3841126230 learned_addr learned my addr 172.21.15.164:0/3841126230 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:45.127 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.121+0000 7fd22ffff640 1 --2- 172.21.15.164:0/3841126230 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd2300add50 0x7fd2301ddec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:45.127 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.121+0000 7fd23561a640 1 --2- 172.21.15.164:0/3841126230 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd2300b4b20 0x7fd2301de400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:45.127 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.121+0000 7fd234e19640 1 -- 172.21.15.164:0/3841126230 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd2300b4b20 msgr2=0x7fd2301de400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:45.127 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.121+0000 7fd234e19640 1 --2- 172.21.15.164:0/3841126230 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd2300b4b20 0x7fd2301de400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:45.127 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.121+0000 7fd234e19640 1 -- 172.21.15.164:0/3841126230 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd2300add50 msgr2=0x7fd2301ddec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:45.127 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.121+0000 7fd234e19640 1 --2- 172.21.15.164:0/3841126230 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd2300add50 0x7fd2301ddec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:45.127 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.121+0000 7fd234e19640 1 -- 172.21.15.164:0/3841126230 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd21c0094d0 con 0x7fd23009def0 2024-03-23T14:57:45.127 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.121+0000 7fd23561a640 1 --2- 172.21.15.164:0/3841126230 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd2300b4b20 0x7fd2301de400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T14:57:45.127 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.121+0000 7fd234e19640 1 --2- 172.21.15.164:0/3841126230 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd23009def0 0x7fd2300b7cb0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fd22400e930 tx=0x7fd22400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:45.127 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.121+0000 7fd22d7fa640 1 -- 172.21.15.164:0/3841126230 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd224009690 con 0x7fd23009def0 2024-03-23T14:57:45.127 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.121+0000 7fd22d7fa640 1 -- 172.21.15.164:0/3841126230 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd22400ca00 con 0x7fd23009def0 2024-03-23T14:57:45.127 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.121+0000 7fd23707d640 1 -- 172.21.15.164:0/3841126230 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd230145bb0 con 0x7fd23009def0 2024-03-23T14:57:45.128 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.121+0000 7fd22d7fa640 1 -- 172.21.15.164:0/3841126230 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd22400cd70 con 0x7fd23009def0 2024-03-23T14:57:45.128 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.121+0000 7fd23707d640 1 -- 172.21.15.164:0/3841126230 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd230146a60 con 0x7fd23009def0 2024-03-23T14:57:45.129 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.121+0000 7fd23707d640 1 -- 172.21.15.164:0/3841126230 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd230145a10 con 0x7fd23009def0 2024-03-23T14:57:45.129 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.125+0000 7fd22d7fa640 1 -- 172.21.15.164:0/3841126230 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7fd224013470 con 0x7fd23009def0 2024-03-23T14:57:45.129 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.125+0000 7fd22d7fa640 1 --2- 172.21.15.164:0/3841126230 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fd2140672d0 0x7fd214069790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:45.129 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.125+0000 7fd22d7fa640 1 -- 172.21.15.164:0/3841126230 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5709+0+0 (secure 0 0 0) 0x7fd2240851b0 con 0x7fd23009def0 2024-03-23T14:57:45.129 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.125+0000 7fd22ffff640 1 --2- 172.21.15.164:0/3841126230 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fd2140672d0 0x7fd214069790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:45.129 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.125+0000 7fd22ffff640 1 --2- 172.21.15.164:0/3841126230 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fd2140672d0 0x7fd214069790 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fd21c0ae000 tx=0x7fd21c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:45.132 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.129+0000 7fd22d7fa640 1 -- 172.21.15.164:0/3841126230 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7fd230145a10 con 0x7fd23009def0 2024-03-23T14:57:45.447 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.441+0000 7fd23707d640 1 -- 172.21.15.164:0/3841126230 --> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd2301515a0 con 0x7fd2140672d0 2024-03-23T14:57:45.459 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.453+0000 7fd22d7fa640 1 -- 172.21.15.164:0/3841126230 <== mgr.4107 v2:172.21.15.164:6832/2139087985 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251881 (secure 0 0 0) 0x7fd21c0bb090 con 0x7fd2140672d0 2024-03-23T14:57:45.461 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-23T14:57:45.462 INFO:teuthology.orchestra.run.smithi164.stderr:dumped all 2024-03-23T14:57:45.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.457+0000 7fd23707d640 1 -- 172.21.15.164:0/3841126230 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fd2140672d0 msgr2=0x7fd214069790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:45.464 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.457+0000 7fd23707d640 1 --2- 172.21.15.164:0/3841126230 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fd2140672d0 0x7fd214069790 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fd21c0ae000 tx=0x7fd21c008040 comp rx=0 tx=0).stop 2024-03-23T14:57:45.464 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.457+0000 7fd23707d640 1 -- 172.21.15.164:0/3841126230 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd23009def0 msgr2=0x7fd2300b7cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:45.464 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.457+0000 7fd23707d640 1 --2- 172.21.15.164:0/3841126230 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd23009def0 0x7fd2300b7cb0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fd22400e930 tx=0x7fd22400edf0 comp rx=0 tx=0).stop 2024-03-23T14:57:45.464 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.457+0000 7fd23707d640 1 -- 172.21.15.164:0/3841126230 shutdown_connections 2024-03-23T14:57:45.464 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.457+0000 7fd23707d640 1 --2- 172.21.15.164:0/3841126230 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd2300b4b20 0x7fd2301de400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:45.464 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.457+0000 7fd23707d640 1 --2- 172.21.15.164:0/3841126230 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fd2140672d0 0x7fd214069790 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:45.464 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.457+0000 7fd23707d640 1 --2- 172.21.15.164:0/3841126230 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd2300add50 0x7fd2301ddec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:45.464 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.457+0000 7fd23707d640 1 --2- 172.21.15.164:0/3841126230 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd23009def0 0x7fd2300b7cb0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:45.464 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.457+0000 7fd23707d640 1 -- 172.21.15.164:0/3841126230 >> 172.21.15.164:0/3841126230 conn(0x7fd2300b8ca0 msgr2=0x7fd23013e560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:45.464 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.457+0000 7fd23707d640 1 -- 172.21.15.164:0/3841126230 shutdown_connections 2024-03-23T14:57:45.464 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.457+0000 7fd23707d640 1 -- 172.21.15.164:0/3841126230 wait complete. 2024-03-23T14:57:45.482 INFO:teuthology.orchestra.run.smithi164.stdout:{"pg_ready":true,"pg_map":{"version":51,"stamp":"2024-03-23T14:57:45.122035+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":87,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219772,"kb_used_data":3592,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754948,"statfs":{"total":773094113280,"available":772869066752,"internally_reserved":0,"allocated":3678208,"data_stored":2104531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":44,"apply_latency_ms":44,"commit_latency_ns":44000000,"apply_latency_ns":44000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.542269"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":9,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.648999+0000","last_change":"2024-03-23T14:57:41.615391+0000","last_active":"2024-03-23T14:57:41.648999+0000","last_peered":"2024-03-23T14:57:41.648999+0000","last_clean":"2024-03-23T14:57:41.648999+0000","last_became_active":"2024-03-23T14:57:41.614713+0000","last_became_peered":"2024-03-23T14:57:41.614713+0000","last_unstale":"2024-03-23T14:57:41.648999+0000","last_undegraded":"2024-03-23T14:57:41.648999+0000","last_fullsized":"2024-03-23T14:57:41.648999+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_clean_scrub_stamp":"2024-03-23T14:57:40.577321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:43:37.787881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":9,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.627866+0000","last_change":"2024-03-23T14:57:41.621702+0000","last_active":"2024-03-23T14:57:41.627866+0000","last_peered":"2024-03-23T14:57:41.627866+0000","last_clean":"2024-03-23T14:57:41.627866+0000","last_became_active":"2024-03-23T14:57:41.621156+0000","last_became_peered":"2024-03-23T14:57:41.621156+0000","last_unstale":"2024-03-23T14:57:41.627866+0000","last_undegraded":"2024-03-23T14:57:41.627866+0000","last_fullsized":"2024-03-23T14:57:41.627866+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_clean_scrub_stamp":"2024-03-23T14:57:40.577321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:05:34.469230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":9,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.639444+0000","last_change":"2024-03-23T14:57:41.624751+0000","last_active":"2024-03-23T14:57:41.639444+0000","last_peered":"2024-03-23T14:57:41.639444+0000","last_clean":"2024-03-23T14:57:41.639444+0000","last_became_active":"2024-03-23T14:57:41.624125+0000","last_became_peered":"2024-03-23T14:57:41.624125+0000","last_unstale":"2024-03-23T14:57:41.639444+0000","last_undegraded":"2024-03-23T14:57:41.639444+0000","last_fullsized":"2024-03-23T14:57:41.639444+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_clean_scrub_stamp":"2024-03-23T14:57:40.577321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:42:36.132784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":9,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.639305+0000","last_change":"2024-03-23T14:57:41.635228+0000","last_active":"2024-03-23T14:57:41.639305+0000","last_peered":"2024-03-23T14:57:41.639305+0000","last_clean":"2024-03-23T14:57:41.639305+0000","last_became_active":"2024-03-23T14:57:41.634600+0000","last_became_peered":"2024-03-23T14:57:41.634600+0000","last_unstale":"2024-03-23T14:57:41.639305+0000","last_undegraded":"2024-03-23T14:57:41.639305+0000","last_fullsized":"2024-03-23T14:57:41.639305+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_clean_scrub_stamp":"2024-03-23T14:57:40.577321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:20:49.856427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":9,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.629837+0000","last_change":"2024-03-23T14:57:41.628530+0000","last_active":"2024-03-23T14:57:41.629837+0000","last_peered":"2024-03-23T14:57:41.629837+0000","last_clean":"2024-03-23T14:57:41.629837+0000","last_became_active":"2024-03-23T14:57:41.628168+0000","last_became_peered":"2024-03-23T14:57:41.628168+0000","last_unstale":"2024-03-23T14:57:41.629837+0000","last_undegraded":"2024-03-23T14:57:41.629837+0000","last_fullsized":"2024-03-23T14:57:41.629837+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_clean_scrub_stamp":"2024-03-23T14:57:40.577321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:14:46.990456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":9,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.627866+0000","last_change":"2024-03-23T14:57:41.621605+0000","last_active":"2024-03-23T14:57:41.627866+0000","last_peered":"2024-03-23T14:57:41.627866+0000","last_clean":"2024-03-23T14:57:41.627866+0000","last_became_active":"2024-03-23T14:57:41.621329+0000","last_became_peered":"2024-03-23T14:57:41.621329+0000","last_unstale":"2024-03-23T14:57:41.627866+0000","last_undegraded":"2024-03-23T14:57:41.627866+0000","last_fullsized":"2024-03-23T14:57:41.627866+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_clean_scrub_stamp":"2024-03-23T14:57:40.577321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:42:36.132784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":9,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.654437+0000","last_change":"2024-03-23T14:57:41.649953+0000","last_active":"2024-03-23T14:57:41.654437+0000","last_peered":"2024-03-23T14:57:41.654437+0000","last_clean":"2024-03-23T14:57:41.654437+0000","last_became_active":"2024-03-23T14:57:41.646113+0000","last_became_peered":"2024-03-23T14:57:41.646113+0000","last_unstale":"2024-03-23T14:57:41.654437+0000","last_undegraded":"2024-03-23T14:57:41.654437+0000","last_fullsized":"2024-03-23T14:57:41.654437+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_clean_scrub_stamp":"2024-03-23T14:57:40.577321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:42:36.132784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":9,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.654204+0000","last_change":"2024-03-23T14:57:41.645857+0000","last_active":"2024-03-23T14:57:41.654204+0000","last_peered":"2024-03-23T14:57:41.654204+0000","last_clean":"2024-03-23T14:57:41.654204+0000","last_became_active":"2024-03-23T14:57:41.633564+0000","last_became_peered":"2024-03-23T14:57:41.633564+0000","last_unstale":"2024-03-23T14:57:41.654204+0000","last_undegraded":"2024-03-23T14:57:41.654204+0000","last_fullsized":"2024-03-23T14:57:41.654204+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_clean_scrub_stamp":"2024-03-23T14:57:40.577321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:51:50.188772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":9,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.629994+0000","last_change":"2024-03-23T14:57:41.621763+0000","last_active":"2024-03-23T14:57:41.629994+0000","last_peered":"2024-03-23T14:57:41.629994+0000","last_clean":"2024-03-23T14:57:41.629994+0000","last_became_active":"2024-03-23T14:57:41.621025+0000","last_became_peered":"2024-03-23T14:57:41.621025+0000","last_unstale":"2024-03-23T14:57:41.629994+0000","last_undegraded":"2024-03-23T14:57:41.629994+0000","last_fullsized":"2024-03-23T14:57:41.629994+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_clean_scrub_stamp":"2024-03-23T14:57:40.577321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:28:32.371371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":9,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.633397+0000","last_change":"2024-03-23T14:57:41.628800+0000","last_active":"2024-03-23T14:57:41.633397+0000","last_peered":"2024-03-23T14:57:41.633397+0000","last_clean":"2024-03-23T14:57:41.633397+0000","last_became_active":"2024-03-23T14:57:41.628517+0000","last_became_peered":"2024-03-23T14:57:41.628517+0000","last_unstale":"2024-03-23T14:57:41.633397+0000","last_undegraded":"2024-03-23T14:57:41.633397+0000","last_fullsized":"2024-03-23T14:57:41.633397+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_clean_scrub_stamp":"2024-03-23T14:57:40.577321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:45:08.796108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":9,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.654103+0000","last_change":"2024-03-23T14:57:41.633753+0000","last_active":"2024-03-23T14:57:41.654103+0000","last_peered":"2024-03-23T14:57:41.654103+0000","last_clean":"2024-03-23T14:57:41.654103+0000","last_became_active":"2024-03-23T14:57:41.632768+0000","last_became_peered":"2024-03-23T14:57:41.632768+0000","last_unstale":"2024-03-23T14:57:41.654103+0000","last_undegraded":"2024-03-23T14:57:41.654103+0000","last_fullsized":"2024-03-23T14:57:41.654103+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_clean_scrub_stamp":"2024-03-23T14:57:40.577321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:20:49.856427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":9,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.641208+0000","last_change":"2024-03-23T14:57:41.626849+0000","last_active":"2024-03-23T14:57:41.641208+0000","last_peered":"2024-03-23T14:57:41.641208+0000","last_clean":"2024-03-23T14:57:41.641208+0000","last_became_active":"2024-03-23T14:57:41.626124+0000","last_became_peered":"2024-03-23T14:57:41.626124+0000","last_unstale":"2024-03-23T14:57:41.641208+0000","last_undegraded":"2024-03-23T14:57:41.641208+0000","last_fullsized":"2024-03-23T14:57:41.641208+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_clean_scrub_stamp":"2024-03-23T14:57:40.577321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:38:13.872392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":9,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.624920+0000","last_change":"2024-03-23T14:57:41.623838+0000","last_active":"2024-03-23T14:57:41.624920+0000","last_peered":"2024-03-23T14:57:41.624920+0000","last_clean":"2024-03-23T14:57:41.624920+0000","last_became_active":"2024-03-23T14:57:41.623449+0000","last_became_peered":"2024-03-23T14:57:41.623449+0000","last_unstale":"2024-03-23T14:57:41.624920+0000","last_undegraded":"2024-03-23T14:57:41.624920+0000","last_fullsized":"2024-03-23T14:57:41.624920+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_clean_scrub_stamp":"2024-03-23T14:57:40.577321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:59:22.990207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":9,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.627845+0000","last_change":"2024-03-23T14:57:41.620833+0000","last_active":"2024-03-23T14:57:41.627845+0000","last_peered":"2024-03-23T14:57:41.627845+0000","last_clean":"2024-03-23T14:57:41.627845+0000","last_became_active":"2024-03-23T14:57:41.620247+0000","last_became_peered":"2024-03-23T14:57:41.620247+0000","last_unstale":"2024-03-23T14:57:41.627845+0000","last_undegraded":"2024-03-23T14:57:41.627845+0000","last_fullsized":"2024-03-23T14:57:41.627845+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_clean_scrub_stamp":"2024-03-23T14:57:40.577321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:23:11.000724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":9,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.642318+0000","last_change":"2024-03-23T14:57:41.634231+0000","last_active":"2024-03-23T14:57:41.642318+0000","last_peered":"2024-03-23T14:57:41.642318+0000","last_clean":"2024-03-23T14:57:41.642318+0000","last_became_active":"2024-03-23T14:57:41.633584+0000","last_became_peered":"2024-03-23T14:57:41.633584+0000","last_unstale":"2024-03-23T14:57:41.642318+0000","last_undegraded":"2024-03-23T14:57:41.642318+0000","last_fullsized":"2024-03-23T14:57:41.642318+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_clean_scrub_stamp":"2024-03-23T14:57:40.577321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:44:02.215207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":9,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.656841+0000","last_change":"2024-03-23T14:57:41.652115+0000","last_active":"2024-03-23T14:57:41.656841+0000","last_peered":"2024-03-23T14:57:41.656841+0000","last_clean":"2024-03-23T14:57:41.656841+0000","last_became_active":"2024-03-23T14:57:41.644519+0000","last_became_peered":"2024-03-23T14:57:41.644519+0000","last_unstale":"2024-03-23T14:57:41.656841+0000","last_undegraded":"2024-03-23T14:57:41.656841+0000","last_fullsized":"2024-03-23T14:57:41.656841+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_clean_scrub_stamp":"2024-03-23T14:57:40.577321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:44:02.215207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":9,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.656752+0000","last_change":"2024-03-23T14:57:41.650919+0000","last_active":"2024-03-23T14:57:41.656752+0000","last_peered":"2024-03-23T14:57:41.656752+0000","last_clean":"2024-03-23T14:57:41.656752+0000","last_became_active":"2024-03-23T14:57:41.650239+0000","last_became_peered":"2024-03-23T14:57:41.650239+0000","last_unstale":"2024-03-23T14:57:41.656752+0000","last_undegraded":"2024-03-23T14:57:41.656752+0000","last_fullsized":"2024-03-23T14:57:41.656752+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_clean_scrub_stamp":"2024-03-23T14:57:40.577321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:38:13.872392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":9,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.634264+0000","last_change":"2024-03-23T14:57:41.627591+0000","last_active":"2024-03-23T14:57:41.634264+0000","last_peered":"2024-03-23T14:57:41.634264+0000","last_clean":"2024-03-23T14:57:41.634264+0000","last_became_active":"2024-03-23T14:57:41.627232+0000","last_became_peered":"2024-03-23T14:57:41.627232+0000","last_unstale":"2024-03-23T14:57:41.634264+0000","last_undegraded":"2024-03-23T14:57:41.634264+0000","last_fullsized":"2024-03-23T14:57:41.634264+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_clean_scrub_stamp":"2024-03-23T14:57:40.577321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:39:56.285297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":9,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.632591+0000","last_change":"2024-03-23T14:57:41.627012+0000","last_active":"2024-03-23T14:57:41.632591+0000","last_peered":"2024-03-23T14:57:41.632591+0000","last_clean":"2024-03-23T14:57:41.632591+0000","last_became_active":"2024-03-23T14:57:41.626686+0000","last_became_peered":"2024-03-23T14:57:41.626686+0000","last_unstale":"2024-03-23T14:57:41.632591+0000","last_undegraded":"2024-03-23T14:57:41.632591+0000","last_fullsized":"2024-03-23T14:57:41.632591+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_clean_scrub_stamp":"2024-03-23T14:57:40.577321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:30:32.413411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":9,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.624238+0000","last_change":"2024-03-23T14:57:41.620728+0000","last_active":"2024-03-23T14:57:41.624238+0000","last_peered":"2024-03-23T14:57:41.624238+0000","last_clean":"2024-03-23T14:57:41.624238+0000","last_became_active":"2024-03-23T14:57:41.620289+0000","last_became_peered":"2024-03-23T14:57:41.620289+0000","last_unstale":"2024-03-23T14:57:41.624238+0000","last_undegraded":"2024-03-23T14:57:41.624238+0000","last_fullsized":"2024-03-23T14:57:41.624238+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_clean_scrub_stamp":"2024-03-23T14:57:40.577321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:36:14.704031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":9,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.624970+0000","last_change":"2024-03-23T14:57:41.618455+0000","last_active":"2024-03-23T14:57:41.624970+0000","last_peered":"2024-03-23T14:57:41.624970+0000","last_clean":"2024-03-23T14:57:41.624970+0000","last_became_active":"2024-03-23T14:57:41.617285+0000","last_became_peered":"2024-03-23T14:57:41.617285+0000","last_unstale":"2024-03-23T14:57:41.624970+0000","last_undegraded":"2024-03-23T14:57:41.624970+0000","last_fullsized":"2024-03-23T14:57:41.624970+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_clean_scrub_stamp":"2024-03-23T14:57:40.577321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:34:29.928307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":9,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.628698+0000","last_change":"2024-03-23T14:57:41.620589+0000","last_active":"2024-03-23T14:57:41.628698+0000","last_peered":"2024-03-23T14:57:41.628698+0000","last_clean":"2024-03-23T14:57:41.628698+0000","last_became_active":"2024-03-23T14:57:41.619812+0000","last_became_peered":"2024-03-23T14:57:41.619812+0000","last_unstale":"2024-03-23T14:57:41.628698+0000","last_undegraded":"2024-03-23T14:57:41.628698+0000","last_fullsized":"2024-03-23T14:57:41.628698+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_clean_scrub_stamp":"2024-03-23T14:57:40.577321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:45:22.740960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":9,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.636873+0000","last_change":"2024-03-23T14:57:41.635167+0000","last_active":"2024-03-23T14:57:41.636873+0000","last_peered":"2024-03-23T14:57:41.636873+0000","last_clean":"2024-03-23T14:57:41.636873+0000","last_became_active":"2024-03-23T14:57:41.634890+0000","last_became_peered":"2024-03-23T14:57:41.634890+0000","last_unstale":"2024-03-23T14:57:41.636873+0000","last_undegraded":"2024-03-23T14:57:41.636873+0000","last_fullsized":"2024-03-23T14:57:41.636873+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_clean_scrub_stamp":"2024-03-23T14:57:40.577321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:45:08.796108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":9,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.614400+0000","last_change":"2024-03-23T14:57:41.609841+0000","last_active":"2024-03-23T14:57:41.614400+0000","last_peered":"2024-03-23T14:57:41.614400+0000","last_clean":"2024-03-23T14:57:41.614400+0000","last_became_active":"2024-03-23T14:57:41.609019+0000","last_became_peered":"2024-03-23T14:57:41.609019+0000","last_unstale":"2024-03-23T14:57:41.614400+0000","last_undegraded":"2024-03-23T14:57:41.614400+0000","last_fullsized":"2024-03-23T14:57:41.614400+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_clean_scrub_stamp":"2024-03-23T14:57:40.577321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:20:49.856427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":9,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.641892+0000","last_change":"2024-03-23T14:57:41.637844+0000","last_active":"2024-03-23T14:57:41.641892+0000","last_peered":"2024-03-23T14:57:41.641892+0000","last_clean":"2024-03-23T14:57:41.641892+0000","last_became_active":"2024-03-23T14:57:41.637554+0000","last_became_peered":"2024-03-23T14:57:41.637554+0000","last_unstale":"2024-03-23T14:57:41.641892+0000","last_undegraded":"2024-03-23T14:57:41.641892+0000","last_fullsized":"2024-03-23T14:57:41.641892+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_clean_scrub_stamp":"2024-03-23T14:57:40.577321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:26:34.952555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":9,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.640271+0000","last_change":"2024-03-23T14:57:41.635914+0000","last_active":"2024-03-23T14:57:41.640271+0000","last_peered":"2024-03-23T14:57:41.640271+0000","last_clean":"2024-03-23T14:57:41.640271+0000","last_became_active":"2024-03-23T14:57:41.635608+0000","last_became_peered":"2024-03-23T14:57:41.635608+0000","last_unstale":"2024-03-23T14:57:41.640271+0000","last_undegraded":"2024-03-23T14:57:41.640271+0000","last_fullsized":"2024-03-23T14:57:41.640271+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_clean_scrub_stamp":"2024-03-23T14:57:40.577321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:47:38.826546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":9,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.641840+0000","last_change":"2024-03-23T14:57:41.638249+0000","last_active":"2024-03-23T14:57:41.641840+0000","last_peered":"2024-03-23T14:57:41.641840+0000","last_clean":"2024-03-23T14:57:41.641840+0000","last_became_active":"2024-03-23T14:57:41.637000+0000","last_became_peered":"2024-03-23T14:57:41.637000+0000","last_unstale":"2024-03-23T14:57:41.641840+0000","last_undegraded":"2024-03-23T14:57:41.641840+0000","last_fullsized":"2024-03-23T14:57:41.641840+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_clean_scrub_stamp":"2024-03-23T14:57:40.577321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:51:50.188772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":9,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.620928+0000","last_change":"2024-03-23T14:57:41.615815+0000","last_active":"2024-03-23T14:57:41.620928+0000","last_peered":"2024-03-23T14:57:41.620928+0000","last_clean":"2024-03-23T14:57:41.620928+0000","last_became_active":"2024-03-23T14:57:41.615129+0000","last_became_peered":"2024-03-23T14:57:41.615129+0000","last_unstale":"2024-03-23T14:57:41.620928+0000","last_undegraded":"2024-03-23T14:57:41.620928+0000","last_fullsized":"2024-03-23T14:57:41.620928+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_clean_scrub_stamp":"2024-03-23T14:57:40.577321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:51:50.188772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":9,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.631833+0000","last_change":"2024-03-23T14:57:41.627518+0000","last_active":"2024-03-23T14:57:41.631833+0000","last_peered":"2024-03-23T14:57:41.631833+0000","last_clean":"2024-03-23T14:57:41.631833+0000","last_became_active":"2024-03-23T14:57:41.626733+0000","last_became_peered":"2024-03-23T14:57:41.626733+0000","last_unstale":"2024-03-23T14:57:41.631833+0000","last_undegraded":"2024-03-23T14:57:41.631833+0000","last_fullsized":"2024-03-23T14:57:41.631833+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_clean_scrub_stamp":"2024-03-23T14:57:40.577321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:43:37.787881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":9,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.654387+0000","last_change":"2024-03-23T14:57:41.645999+0000","last_active":"2024-03-23T14:57:41.654387+0000","last_peered":"2024-03-23T14:57:41.654387+0000","last_clean":"2024-03-23T14:57:41.654387+0000","last_became_active":"2024-03-23T14:57:41.645420+0000","last_became_peered":"2024-03-23T14:57:41.645420+0000","last_unstale":"2024-03-23T14:57:41.654387+0000","last_undegraded":"2024-03-23T14:57:41.654387+0000","last_fullsized":"2024-03-23T14:57:41.654387+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_clean_scrub_stamp":"2024-03-23T14:57:40.577321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:23:11.000724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":9,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.642748+0000","last_change":"2024-03-23T14:57:41.631694+0000","last_active":"2024-03-23T14:57:41.642748+0000","last_peered":"2024-03-23T14:57:41.642748+0000","last_clean":"2024-03-23T14:57:41.642748+0000","last_became_active":"2024-03-23T14:57:41.630120+0000","last_became_peered":"2024-03-23T14:57:41.630120+0000","last_unstale":"2024-03-23T14:57:41.642748+0000","last_undegraded":"2024-03-23T14:57:41.642748+0000","last_fullsized":"2024-03-23T14:57:41.642748+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_clean_scrub_stamp":"2024-03-23T14:57:40.577321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:34:29.928307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":9,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.656790+0000","last_change":"2024-03-23T14:57:41.651632+0000","last_active":"2024-03-23T14:57:41.656790+0000","last_peered":"2024-03-23T14:57:41.656790+0000","last_clean":"2024-03-23T14:57:41.656790+0000","last_became_active":"2024-03-23T14:57:41.641761+0000","last_became_peered":"2024-03-23T14:57:41.641761+0000","last_unstale":"2024-03-23T14:57:41.656790+0000","last_undegraded":"2024-03-23T14:57:41.656790+0000","last_fullsized":"2024-03-23T14:57:41.656790+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_clean_scrub_stamp":"2024-03-23T14:57:40.577321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:30:32.413411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":9,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.641517+0000","last_change":"2024-03-23T14:57:41.636034+0000","last_active":"2024-03-23T14:57:41.641517+0000","last_peered":"2024-03-23T14:57:41.641517+0000","last_clean":"2024-03-23T14:57:41.641517+0000","last_became_active":"2024-03-23T14:57:41.635678+0000","last_became_peered":"2024-03-23T14:57:41.635678+0000","last_unstale":"2024-03-23T14:57:41.641517+0000","last_undegraded":"2024-03-23T14:57:41.641517+0000","last_fullsized":"2024-03-23T14:57:41.641517+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_clean_scrub_stamp":"2024-03-23T14:57:40.577321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:10:52.026797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"28'4","reported_seq":16,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.654169+0000","last_change":"2024-03-23T14:57:40.629717+0000","last_active":"2024-03-23T14:57:41.654169+0000","last_peered":"2024-03-23T14:57:41.654169+0000","last_clean":"2024-03-23T14:57:41.654169+0000","last_became_active":"2024-03-23T14:57:40.629001+0000","last_became_peered":"2024-03-23T14:57:40.629001+0000","last_unstale":"2024-03-23T14:57:41.654169+0000","last_undegraded":"2024-03-23T14:57:41.654169+0000","last_fullsized":"2024-03-23T14:57:41.654169+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:39.565698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:39.565698+0000","last_clean_scrub_stamp":"2024-03-23T14:57:39.565698+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:42:06.517354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":9,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.636605+0000","last_change":"2024-03-23T14:57:41.612499+0000","last_active":"2024-03-23T14:57:41.636605+0000","last_peered":"2024-03-23T14:57:41.636605+0000","last_clean":"2024-03-23T14:57:41.636605+0000","last_became_active":"2024-03-23T14:57:41.611762+0000","last_became_peered":"2024-03-23T14:57:41.611762+0000","last_unstale":"2024-03-23T14:57:41.636605+0000","last_undegraded":"2024-03-23T14:57:41.636605+0000","last_fullsized":"2024-03-23T14:57:41.636605+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_clean_scrub_stamp":"2024-03-23T14:57:40.577321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:11:44.272923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"28'1","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.656184+0000","last_change":"2024-03-23T14:57:40.614049+0000","last_active":"2024-03-23T14:57:41.656184+0000","last_peered":"2024-03-23T14:57:41.656184+0000","last_clean":"2024-03-23T14:57:41.656184+0000","last_became_active":"2024-03-23T14:57:40.613755+0000","last_became_peered":"2024-03-23T14:57:40.613755+0000","last_unstale":"2024-03-23T14:57:41.656184+0000","last_undegraded":"2024-03-23T14:57:41.656184+0000","last_fullsized":"2024-03-23T14:57:41.656184+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:39.565698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:39.565698+0000","last_clean_scrub_stamp":"2024-03-23T14:57:39.565698+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:11:43.261300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":9,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.631565+0000","last_change":"2024-03-23T14:57:41.628564+0000","last_active":"2024-03-23T14:57:41.631565+0000","last_peered":"2024-03-23T14:57:41.631565+0000","last_clean":"2024-03-23T14:57:41.631565+0000","last_became_active":"2024-03-23T14:57:41.627780+0000","last_became_peered":"2024-03-23T14:57:41.627780+0000","last_unstale":"2024-03-23T14:57:41.631565+0000","last_undegraded":"2024-03-23T14:57:41.631565+0000","last_fullsized":"2024-03-23T14:57:41.631565+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_clean_scrub_stamp":"2024-03-23T14:57:40.577321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:00:23.723071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"28'2","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.642375+0000","last_change":"2024-03-23T14:57:40.609708+0000","last_active":"2024-03-23T14:57:41.642375+0000","last_peered":"2024-03-23T14:57:41.642375+0000","last_clean":"2024-03-23T14:57:41.642375+0000","last_became_active":"2024-03-23T14:57:40.609415+0000","last_became_peered":"2024-03-23T14:57:40.609415+0000","last_unstale":"2024-03-23T14:57:41.642375+0000","last_undegraded":"2024-03-23T14:57:41.642375+0000","last_fullsized":"2024-03-23T14:57:41.642375+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:39.565698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:39.565698+0000","last_clean_scrub_stamp":"2024-03-23T14:57:39.565698+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:20:48.844804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.638512+0000","last_change":"2024-03-23T14:57:41.632207+0000","last_active":"2024-03-23T14:57:41.638512+0000","last_peered":"2024-03-23T14:57:41.638512+0000","last_clean":"2024-03-23T14:57:41.638512+0000","last_became_active":"2024-03-23T14:57:41.631893+0000","last_became_peered":"2024-03-23T14:57:41.631893+0000","last_unstale":"2024-03-23T14:57:41.638512+0000","last_undegraded":"2024-03-23T14:57:41.638512+0000","last_fullsized":"2024-03-23T14:57:41.638512+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_clean_scrub_stamp":"2024-03-23T14:57:40.577321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:10:52.026797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"28'2","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.644320+0000","last_change":"2024-03-23T14:57:40.611599+0000","last_active":"2024-03-23T14:57:41.644320+0000","last_peered":"2024-03-23T14:57:41.644320+0000","last_clean":"2024-03-23T14:57:41.644320+0000","last_became_active":"2024-03-23T14:57:40.611261+0000","last_became_peered":"2024-03-23T14:57:40.611261+0000","last_unstale":"2024-03-23T14:57:41.644320+0000","last_undegraded":"2024-03-23T14:57:41.644320+0000","last_fullsized":"2024-03-23T14:57:41.644320+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:39.565698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:39.565698+0000","last_clean_scrub_stamp":"2024-03-23T14:57:39.565698+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:13:40.745947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":9,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.654334+0000","last_change":"2024-03-23T14:57:41.634087+0000","last_active":"2024-03-23T14:57:41.654334+0000","last_peered":"2024-03-23T14:57:41.654334+0000","last_clean":"2024-03-23T14:57:41.654334+0000","last_became_active":"2024-03-23T14:57:41.633094+0000","last_became_peered":"2024-03-23T14:57:41.633094+0000","last_unstale":"2024-03-23T14:57:41.654334+0000","last_undegraded":"2024-03-23T14:57:41.654334+0000","last_fullsized":"2024-03-23T14:57:41.654334+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_clean_scrub_stamp":"2024-03-23T14:57:40.577321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:26:34.952555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"28'1","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.640376+0000","last_change":"2024-03-23T14:57:40.609143+0000","last_active":"2024-03-23T14:57:41.640376+0000","last_peered":"2024-03-23T14:57:41.640376+0000","last_clean":"2024-03-23T14:57:41.640376+0000","last_became_active":"2024-03-23T14:57:40.608564+0000","last_became_peered":"2024-03-23T14:57:40.608564+0000","last_unstale":"2024-03-23T14:57:41.640376+0000","last_undegraded":"2024-03-23T14:57:41.640376+0000","last_fullsized":"2024-03-23T14:57:41.640376+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:39.565698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:39.565698+0000","last_clean_scrub_stamp":"2024-03-23T14:57:39.565698+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:52:40.375873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.626291+0000","last_change":"2024-03-23T14:57:41.621965+0000","last_active":"2024-03-23T14:57:41.626291+0000","last_peered":"2024-03-23T14:57:41.626291+0000","last_clean":"2024-03-23T14:57:41.626291+0000","last_became_active":"2024-03-23T14:57:41.621184+0000","last_became_peered":"2024-03-23T14:57:41.621184+0000","last_unstale":"2024-03-23T14:57:41.626291+0000","last_undegraded":"2024-03-23T14:57:41.626291+0000","last_fullsized":"2024-03-23T14:57:41.626291+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_clean_scrub_stamp":"2024-03-23T14:57:40.577321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:23:11.000724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"28'1","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.640104+0000","last_change":"2024-03-23T14:57:40.605238+0000","last_active":"2024-03-23T14:57:41.640104+0000","last_peered":"2024-03-23T14:57:41.640104+0000","last_clean":"2024-03-23T14:57:41.640104+0000","last_became_active":"2024-03-23T14:57:40.604489+0000","last_became_peered":"2024-03-23T14:57:40.604489+0000","last_unstale":"2024-03-23T14:57:41.640104+0000","last_undegraded":"2024-03-23T14:57:41.640104+0000","last_fullsized":"2024-03-23T14:57:41.640104+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:39.565698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:39.565698+0000","last_clean_scrub_stamp":"2024-03-23T14:57:39.565698+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:47:38.848526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.612995+0000","last_change":"2024-03-23T14:57:41.608851+0000","last_active":"2024-03-23T14:57:41.612995+0000","last_peered":"2024-03-23T14:57:41.612995+0000","last_clean":"2024-03-23T14:57:41.612995+0000","last_became_active":"2024-03-23T14:57:41.607807+0000","last_became_peered":"2024-03-23T14:57:41.607807+0000","last_unstale":"2024-03-23T14:57:41.612995+0000","last_undegraded":"2024-03-23T14:57:41.612995+0000","last_fullsized":"2024-03-23T14:57:41.612995+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_clean_scrub_stamp":"2024-03-23T14:57:40.577321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:13:13.184800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.601841+0000","last_change":"2024-03-23T14:57:40.607129+0000","last_active":"2024-03-23T14:57:41.601841+0000","last_peered":"2024-03-23T14:57:41.601841+0000","last_clean":"2024-03-23T14:57:41.601841+0000","last_became_active":"2024-03-23T14:57:40.606666+0000","last_became_peered":"2024-03-23T14:57:40.606666+0000","last_unstale":"2024-03-23T14:57:41.601841+0000","last_undegraded":"2024-03-23T14:57:41.601841+0000","last_fullsized":"2024-03-23T14:57:41.601841+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:39.565698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:39.565698+0000","last_clean_scrub_stamp":"2024-03-23T14:57:39.565698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:00:22.711448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.625345+0000","last_change":"2024-03-23T14:57:41.622864+0000","last_active":"2024-03-23T14:57:41.625345+0000","last_peered":"2024-03-23T14:57:41.625345+0000","last_clean":"2024-03-23T14:57:41.625345+0000","last_became_active":"2024-03-23T14:57:41.622119+0000","last_became_peered":"2024-03-23T14:57:41.622119+0000","last_unstale":"2024-03-23T14:57:41.625345+0000","last_undegraded":"2024-03-23T14:57:41.625345+0000","last_fullsized":"2024-03-23T14:57:41.625345+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_clean_scrub_stamp":"2024-03-23T14:57:40.577321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:43:37.787881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.597631+0000","last_change":"2024-03-23T14:57:40.618722+0000","last_active":"2024-03-23T14:57:41.597631+0000","last_peered":"2024-03-23T14:57:41.597631+0000","last_clean":"2024-03-23T14:57:41.597631+0000","last_became_active":"2024-03-23T14:57:40.617965+0000","last_became_peered":"2024-03-23T14:57:40.617965+0000","last_unstale":"2024-03-23T14:57:41.597631+0000","last_undegraded":"2024-03-23T14:57:41.597631+0000","last_fullsized":"2024-03-23T14:57:41.597631+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:39.565698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:39.565698+0000","last_clean_scrub_stamp":"2024-03-23T14:57:39.565698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T22:59:12.524685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.641350+0000","last_change":"2024-03-23T14:57:41.629551+0000","last_active":"2024-03-23T14:57:41.641350+0000","last_peered":"2024-03-23T14:57:41.641350+0000","last_clean":"2024-03-23T14:57:41.641350+0000","last_became_active":"2024-03-23T14:57:41.628673+0000","last_became_peered":"2024-03-23T14:57:41.628673+0000","last_unstale":"2024-03-23T14:57:41.641350+0000","last_undegraded":"2024-03-23T14:57:41.641350+0000","last_fullsized":"2024-03-23T14:57:41.641350+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_clean_scrub_stamp":"2024-03-23T14:57:40.577321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:30:32.413411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.606349+0000","last_change":"2024-03-23T14:57:40.617570+0000","last_active":"2024-03-23T14:57:41.606349+0000","last_peered":"2024-03-23T14:57:41.606349+0000","last_clean":"2024-03-23T14:57:41.606349+0000","last_became_active":"2024-03-23T14:57:40.617234+0000","last_became_peered":"2024-03-23T14:57:40.617234+0000","last_unstale":"2024-03-23T14:57:41.606349+0000","last_undegraded":"2024-03-23T14:57:41.606349+0000","last_fullsized":"2024-03-23T14:57:41.606349+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:39.565698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:39.565698+0000","last_clean_scrub_stamp":"2024-03-23T14:57:39.565698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:51:49.177149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.632764+0000","last_change":"2024-03-23T14:57:41.626893+0000","last_active":"2024-03-23T14:57:41.632764+0000","last_peered":"2024-03-23T14:57:41.632764+0000","last_clean":"2024-03-23T14:57:41.632764+0000","last_became_active":"2024-03-23T14:57:41.626470+0000","last_became_peered":"2024-03-23T14:57:41.626470+0000","last_unstale":"2024-03-23T14:57:41.632764+0000","last_undegraded":"2024-03-23T14:57:41.632764+0000","last_fullsized":"2024-03-23T14:57:41.632764+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_clean_scrub_stamp":"2024-03-23T14:57:40.577321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:38:13.872392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.595689+0000","last_change":"2024-03-23T14:57:40.606169+0000","last_active":"2024-03-23T14:57:41.595689+0000","last_peered":"2024-03-23T14:57:41.595689+0000","last_clean":"2024-03-23T14:57:41.595689+0000","last_became_active":"2024-03-23T14:57:40.605494+0000","last_became_peered":"2024-03-23T14:57:40.605494+0000","last_unstale":"2024-03-23T14:57:41.595689+0000","last_undegraded":"2024-03-23T14:57:41.595689+0000","last_fullsized":"2024-03-23T14:57:41.595689+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:39.565698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:39.565698+0000","last_clean_scrub_stamp":"2024-03-23T14:57:39.565698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:34:28.916684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.647988+0000","last_change":"2024-03-23T14:57:41.644969+0000","last_active":"2024-03-23T14:57:41.647988+0000","last_peered":"2024-03-23T14:57:41.647988+0000","last_clean":"2024-03-23T14:57:41.647988+0000","last_became_active":"2024-03-23T14:57:41.644576+0000","last_became_peered":"2024-03-23T14:57:41.644576+0000","last_unstale":"2024-03-23T14:57:41.647988+0000","last_undegraded":"2024-03-23T14:57:41.647988+0000","last_fullsized":"2024-03-23T14:57:41.647988+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_clean_scrub_stamp":"2024-03-23T14:57:40.577321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:21:33.715027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"28'3","reported_seq":15,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.657062+0000","last_change":"2024-03-23T14:57:40.626790+0000","last_active":"2024-03-23T14:57:41.657062+0000","last_peered":"2024-03-23T14:57:41.657062+0000","last_clean":"2024-03-23T14:57:41.657062+0000","last_became_active":"2024-03-23T14:57:40.626120+0000","last_became_peered":"2024-03-23T14:57:40.626120+0000","last_unstale":"2024-03-23T14:57:41.657062+0000","last_undegraded":"2024-03-23T14:57:41.657062+0000","last_fullsized":"2024-03-23T14:57:41.657062+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:39.565698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:39.565698+0000","last_clean_scrub_stamp":"2024-03-23T14:57:39.565698+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:10:51.015174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.639553+0000","last_change":"2024-03-23T14:57:41.626456+0000","last_active":"2024-03-23T14:57:41.639553+0000","last_peered":"2024-03-23T14:57:41.639553+0000","last_clean":"2024-03-23T14:57:41.639553+0000","last_became_active":"2024-03-23T14:57:41.625721+0000","last_became_peered":"2024-03-23T14:57:41.625721+0000","last_unstale":"2024-03-23T14:57:41.639553+0000","last_undegraded":"2024-03-23T14:57:41.639553+0000","last_fullsized":"2024-03-23T14:57:41.639553+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_clean_scrub_stamp":"2024-03-23T14:57:40.577321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:05:34.469230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"28'6","reported_seq":18,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.650937+0000","last_change":"2024-03-23T14:57:40.614043+0000","last_active":"2024-03-23T14:57:41.650937+0000","last_peered":"2024-03-23T14:57:41.650937+0000","last_clean":"2024-03-23T14:57:41.650937+0000","last_became_active":"2024-03-23T14:57:40.613431+0000","last_became_peered":"2024-03-23T14:57:40.613431+0000","last_unstale":"2024-03-23T14:57:41.650937+0000","last_undegraded":"2024-03-23T14:57:41.650937+0000","last_fullsized":"2024-03-23T14:57:41.650937+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:39.565698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:39.565698+0000","last_clean_scrub_stamp":"2024-03-23T14:57:39.565698+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:26:33.940932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.633363+0000","last_change":"2024-03-23T14:57:41.628389+0000","last_active":"2024-03-23T14:57:41.633363+0000","last_peered":"2024-03-23T14:57:41.633363+0000","last_clean":"2024-03-23T14:57:41.633363+0000","last_became_active":"2024-03-23T14:57:41.627873+0000","last_became_peered":"2024-03-23T14:57:41.627873+0000","last_unstale":"2024-03-23T14:57:41.633363+0000","last_undegraded":"2024-03-23T14:57:41.633363+0000","last_fullsized":"2024-03-23T14:57:41.633363+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_clean_scrub_stamp":"2024-03-23T14:57:40.577321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:08:47.903767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"28'2","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.649222+0000","last_change":"2024-03-23T14:57:40.617362+0000","last_active":"2024-03-23T14:57:41.649222+0000","last_peered":"2024-03-23T14:57:41.649222+0000","last_clean":"2024-03-23T14:57:41.649222+0000","last_became_active":"2024-03-23T14:57:40.616297+0000","last_became_peered":"2024-03-23T14:57:40.616297+0000","last_unstale":"2024-03-23T14:57:41.649222+0000","last_undegraded":"2024-03-23T14:57:41.649222+0000","last_fullsized":"2024-03-23T14:57:41.649222+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:39.565698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:39.565698+0000","last_clean_scrub_stamp":"2024-03-23T14:57:39.565698+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:54:36.135171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.627797+0000","last_change":"2024-03-23T14:57:41.621608+0000","last_active":"2024-03-23T14:57:41.627797+0000","last_peered":"2024-03-23T14:57:41.627797+0000","last_clean":"2024-03-23T14:57:41.627797+0000","last_became_active":"2024-03-23T14:57:41.621255+0000","last_became_peered":"2024-03-23T14:57:41.621255+0000","last_unstale":"2024-03-23T14:57:41.627797+0000","last_undegraded":"2024-03-23T14:57:41.627797+0000","last_fullsized":"2024-03-23T14:57:41.627797+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_clean_scrub_stamp":"2024-03-23T14:57:40.577321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:51:58.286721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.620829+0000","last_change":"2024-03-23T14:57:40.610401+0000","last_active":"2024-03-23T14:57:41.620829+0000","last_peered":"2024-03-23T14:57:41.620829+0000","last_clean":"2024-03-23T14:57:41.620829+0000","last_became_active":"2024-03-23T14:57:40.609051+0000","last_became_peered":"2024-03-23T14:57:40.609051+0000","last_unstale":"2024-03-23T14:57:41.620829+0000","last_undegraded":"2024-03-23T14:57:41.620829+0000","last_fullsized":"2024-03-23T14:57:41.620829+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:39.565698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:39.565698+0000","last_clean_scrub_stamp":"2024-03-23T14:57:39.565698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:13:12.173177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":9,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.656871+0000","last_change":"2024-03-23T14:57:41.652563+0000","last_active":"2024-03-23T14:57:41.656871+0000","last_peered":"2024-03-23T14:57:41.656871+0000","last_clean":"2024-03-23T14:57:41.656871+0000","last_became_active":"2024-03-23T14:57:41.645218+0000","last_became_peered":"2024-03-23T14:57:41.645218+0000","last_unstale":"2024-03-23T14:57:41.656871+0000","last_undegraded":"2024-03-23T14:57:41.656871+0000","last_fullsized":"2024-03-23T14:57:41.656871+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_clean_scrub_stamp":"2024-03-23T14:57:40.577321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:39:56.285297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.602267+0000","last_change":"2024-03-23T14:57:40.617115+0000","last_active":"2024-03-23T14:57:41.602267+0000","last_peered":"2024-03-23T14:57:41.602267+0000","last_clean":"2024-03-23T14:57:41.602267+0000","last_became_active":"2024-03-23T14:57:40.616481+0000","last_became_peered":"2024-03-23T14:57:40.616481+0000","last_unstale":"2024-03-23T14:57:41.602267+0000","last_undegraded":"2024-03-23T14:57:41.602267+0000","last_fullsized":"2024-03-23T14:57:41.602267+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:39.565698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:39.565698+0000","last_clean_scrub_stamp":"2024-03-23T14:57:39.565698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:45:07.784485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.642817+0000","last_change":"2024-03-23T14:57:41.631932+0000","last_active":"2024-03-23T14:57:41.642817+0000","last_peered":"2024-03-23T14:57:41.642817+0000","last_clean":"2024-03-23T14:57:41.642817+0000","last_became_active":"2024-03-23T14:57:41.630993+0000","last_became_peered":"2024-03-23T14:57:41.630993+0000","last_unstale":"2024-03-23T14:57:41.642817+0000","last_undegraded":"2024-03-23T14:57:41.642817+0000","last_fullsized":"2024-03-23T14:57:41.642817+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_clean_scrub_stamp":"2024-03-23T14:57:40.577321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:45:22.740960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"28'8","reported_seq":20,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.646408+0000","last_change":"2024-03-23T14:57:40.609730+0000","last_active":"2024-03-23T14:57:41.646408+0000","last_peered":"2024-03-23T14:57:41.646408+0000","last_clean":"2024-03-23T14:57:41.646408+0000","last_became_active":"2024-03-23T14:57:40.609324+0000","last_became_peered":"2024-03-23T14:57:40.609324+0000","last_unstale":"2024-03-23T14:57:41.646408+0000","last_undegraded":"2024-03-23T14:57:41.646408+0000","last_fullsized":"2024-03-23T14:57:41.646408+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:39.565698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:39.565698+0000","last_clean_scrub_stamp":"2024-03-23T14:57:39.565698+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:08:46.892145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":9,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.619427+0000","last_change":"2024-03-23T14:57:41.615444+0000","last_active":"2024-03-23T14:57:41.619427+0000","last_peered":"2024-03-23T14:57:41.619427+0000","last_clean":"2024-03-23T14:57:41.619427+0000","last_became_active":"2024-03-23T14:57:41.614722+0000","last_became_peered":"2024-03-23T14:57:41.614722+0000","last_unstale":"2024-03-23T14:57:41.619427+0000","last_undegraded":"2024-03-23T14:57:41.619427+0000","last_fullsized":"2024-03-23T14:57:41.619427+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_clean_scrub_stamp":"2024-03-23T14:57:40.577321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:11:44.272923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"28'1","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.641958+0000","last_change":"2024-03-23T14:57:40.610732+0000","last_active":"2024-03-23T14:57:41.641958+0000","last_peered":"2024-03-23T14:57:41.641958+0000","last_clean":"2024-03-23T14:57:41.641958+0000","last_became_active":"2024-03-23T14:57:40.610463+0000","last_became_peered":"2024-03-23T14:57:40.610463+0000","last_unstale":"2024-03-23T14:57:41.641958+0000","last_undegraded":"2024-03-23T14:57:41.641958+0000","last_fullsized":"2024-03-23T14:57:41.641958+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:39.565698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:39.565698+0000","last_clean_scrub_stamp":"2024-03-23T14:57:39.565698+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:45:21.729337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.617800+0000","last_change":"2024-03-23T14:57:41.612679+0000","last_active":"2024-03-23T14:57:41.617800+0000","last_peered":"2024-03-23T14:57:41.617800+0000","last_clean":"2024-03-23T14:57:41.617800+0000","last_became_active":"2024-03-23T14:57:41.611947+0000","last_became_peered":"2024-03-23T14:57:41.611947+0000","last_unstale":"2024-03-23T14:57:41.617800+0000","last_undegraded":"2024-03-23T14:57:41.617800+0000","last_fullsized":"2024-03-23T14:57:41.617800+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_clean_scrub_stamp":"2024-03-23T14:57:40.577321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:26:34.952555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"28'2","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.647865+0000","last_change":"2024-03-23T14:57:40.618151+0000","last_active":"2024-03-23T14:57:41.647865+0000","last_peered":"2024-03-23T14:57:41.647865+0000","last_clean":"2024-03-23T14:57:41.647865+0000","last_became_active":"2024-03-23T14:57:40.617379+0000","last_became_peered":"2024-03-23T14:57:40.617379+0000","last_unstale":"2024-03-23T14:57:41.647865+0000","last_undegraded":"2024-03-23T14:57:41.647865+0000","last_fullsized":"2024-03-23T14:57:41.647865+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:39.565698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:39.565698+0000","last_clean_scrub_stamp":"2024-03-23T14:57:39.565698+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:02:53.716281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":9,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.625491+0000","last_change":"2024-03-23T14:57:41.623624+0000","last_active":"2024-03-23T14:57:41.625491+0000","last_peered":"2024-03-23T14:57:41.625491+0000","last_clean":"2024-03-23T14:57:41.625491+0000","last_became_active":"2024-03-23T14:57:41.623311+0000","last_became_peered":"2024-03-23T14:57:41.623311+0000","last_unstale":"2024-03-23T14:57:41.625491+0000","last_undegraded":"2024-03-23T14:57:41.625491+0000","last_fullsized":"2024-03-23T14:57:41.625491+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_clean_scrub_stamp":"2024-03-23T14:57:40.577321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:00:23.723071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"28'1","reported_seq":13,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.643547+0000","last_change":"2024-03-23T14:57:40.619598+0000","last_active":"2024-03-23T14:57:41.643547+0000","last_peered":"2024-03-23T14:57:41.643547+0000","last_clean":"2024-03-23T14:57:41.643547+0000","last_became_active":"2024-03-23T14:57:40.618877+0000","last_became_peered":"2024-03-23T14:57:40.618877+0000","last_unstale":"2024-03-23T14:57:41.643547+0000","last_undegraded":"2024-03-23T14:57:41.643547+0000","last_fullsized":"2024-03-23T14:57:41.643547+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:39.565698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:39.565698+0000","last_clean_scrub_stamp":"2024-03-23T14:57:39.565698+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:35:11.065518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":9,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.644136+0000","last_change":"2024-03-23T14:57:41.636716+0000","last_active":"2024-03-23T14:57:41.644136+0000","last_peered":"2024-03-23T14:57:41.644136+0000","last_clean":"2024-03-23T14:57:41.644136+0000","last_became_active":"2024-03-23T14:57:41.636390+0000","last_became_peered":"2024-03-23T14:57:41.636390+0000","last_unstale":"2024-03-23T14:57:41.644136+0000","last_undegraded":"2024-03-23T14:57:41.644136+0000","last_fullsized":"2024-03-23T14:57:41.644136+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_clean_scrub_stamp":"2024-03-23T14:57:40.577321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T20:35:12.077141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.599643+0000","last_change":"2024-03-23T14:57:40.609459+0000","last_active":"2024-03-23T14:57:41.599643+0000","last_peered":"2024-03-23T14:57:41.599643+0000","last_clean":"2024-03-23T14:57:41.599643+0000","last_became_active":"2024-03-23T14:57:40.608715+0000","last_became_peered":"2024-03-23T14:57:40.608715+0000","last_unstale":"2024-03-23T14:57:41.599643+0000","last_undegraded":"2024-03-23T14:57:41.599643+0000","last_fullsized":"2024-03-23T14:57:41.599643+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:39.565698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:39.565698+0000","last_clean_scrub_stamp":"2024-03-23T14:57:39.565698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:21:57.359199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.622137+0000","last_change":"2024-03-23T14:57:41.619956+0000","last_active":"2024-03-23T14:57:41.622137+0000","last_peered":"2024-03-23T14:57:41.622137+0000","last_clean":"2024-03-23T14:57:41.622137+0000","last_became_active":"2024-03-23T14:57:41.619572+0000","last_became_peered":"2024-03-23T14:57:41.619572+0000","last_unstale":"2024-03-23T14:57:41.622137+0000","last_undegraded":"2024-03-23T14:57:41.622137+0000","last_fullsized":"2024-03-23T14:57:41.622137+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_clean_scrub_stamp":"2024-03-23T14:57:40.577321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:40:04.045827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"28'2","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.644029+0000","last_change":"2024-03-23T14:57:40.609969+0000","last_active":"2024-03-23T14:57:41.644029+0000","last_peered":"2024-03-23T14:57:41.644029+0000","last_clean":"2024-03-23T14:57:41.644029+0000","last_became_active":"2024-03-23T14:57:40.609479+0000","last_became_peered":"2024-03-23T14:57:40.609479+0000","last_unstale":"2024-03-23T14:57:41.644029+0000","last_undegraded":"2024-03-23T14:57:41.644029+0000","last_fullsized":"2024-03-23T14:57:41.644029+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:39.565698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:39.565698+0000","last_clean_scrub_stamp":"2024-03-23T14:57:39.565698+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:28:31.359748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":9,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.624903+0000","last_change":"2024-03-23T14:57:41.623008+0000","last_active":"2024-03-23T14:57:41.624903+0000","last_peered":"2024-03-23T14:57:41.624903+0000","last_clean":"2024-03-23T14:57:41.624903+0000","last_became_active":"2024-03-23T14:57:41.622712+0000","last_became_peered":"2024-03-23T14:57:41.622712+0000","last_unstale":"2024-03-23T14:57:41.624903+0000","last_undegraded":"2024-03-23T14:57:41.624903+0000","last_fullsized":"2024-03-23T14:57:41.624903+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_clean_scrub_stamp":"2024-03-23T14:57:40.577321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T19:36:08.323317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.603841+0000","last_change":"2024-03-23T14:57:40.614136+0000","last_active":"2024-03-23T14:57:41.603841+0000","last_peered":"2024-03-23T14:57:41.603841+0000","last_clean":"2024-03-23T14:57:41.603841+0000","last_became_active":"2024-03-23T14:57:40.612207+0000","last_became_peered":"2024-03-23T14:57:40.612207+0000","last_unstale":"2024-03-23T14:57:41.603841+0000","last_undegraded":"2024-03-23T14:57:41.603841+0000","last_fullsized":"2024-03-23T14:57:41.603841+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:39.565698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:39.565698+0000","last_clean_scrub_stamp":"2024-03-23T14:57:39.565698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:13:12.173177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.619076+0000","last_change":"2024-03-23T14:57:41.616704+0000","last_active":"2024-03-23T14:57:41.619076+0000","last_peered":"2024-03-23T14:57:41.619076+0000","last_clean":"2024-03-23T14:57:41.619076+0000","last_became_active":"2024-03-23T14:57:41.616442+0000","last_became_peered":"2024-03-23T14:57:41.616442+0000","last_unstale":"2024-03-23T14:57:41.619076+0000","last_undegraded":"2024-03-23T14:57:41.619076+0000","last_fullsized":"2024-03-23T14:57:41.619076+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_clean_scrub_stamp":"2024-03-23T14:57:40.577321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:57:48.675270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.600707+0000","last_change":"2024-03-23T14:57:40.620064+0000","last_active":"2024-03-23T14:57:41.600707+0000","last_peered":"2024-03-23T14:57:41.600707+0000","last_clean":"2024-03-23T14:57:41.600707+0000","last_became_active":"2024-03-23T14:57:40.619157+0000","last_became_peered":"2024-03-23T14:57:40.619157+0000","last_unstale":"2024-03-23T14:57:41.600707+0000","last_undegraded":"2024-03-23T14:57:41.600707+0000","last_fullsized":"2024-03-23T14:57:41.600707+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:39.565698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:39.565698+0000","last_clean_scrub_stamp":"2024-03-23T14:57:39.565698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:07:15.676075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.656712+0000","last_change":"2024-03-23T14:57:41.650655+0000","last_active":"2024-03-23T14:57:41.656712+0000","last_peered":"2024-03-23T14:57:41.656712+0000","last_clean":"2024-03-23T14:57:41.656712+0000","last_became_active":"2024-03-23T14:57:41.649963+0000","last_became_peered":"2024-03-23T14:57:41.649963+0000","last_unstale":"2024-03-23T14:57:41.656712+0000","last_undegraded":"2024-03-23T14:57:41.656712+0000","last_fullsized":"2024-03-23T14:57:41.656712+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_clean_scrub_stamp":"2024-03-23T14:57:40.577321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:05:34.469230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.595509+0000","last_change":"2024-03-23T14:57:40.626882+0000","last_active":"2024-03-23T14:57:41.595509+0000","last_peered":"2024-03-23T14:57:41.595509+0000","last_clean":"2024-03-23T14:57:41.595509+0000","last_became_active":"2024-03-23T14:57:40.626002+0000","last_became_peered":"2024-03-23T14:57:40.626002+0000","last_unstale":"2024-03-23T14:57:41.595509+0000","last_undegraded":"2024-03-23T14:57:41.595509+0000","last_fullsized":"2024-03-23T14:57:41.595509+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:39.565698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:39.565698+0000","last_clean_scrub_stamp":"2024-03-23T14:57:39.565698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:45:07.784485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.624283+0000","last_change":"2024-03-23T14:57:41.620475+0000","last_active":"2024-03-23T14:57:41.624283+0000","last_peered":"2024-03-23T14:57:41.624283+0000","last_clean":"2024-03-23T14:57:41.624283+0000","last_became_active":"2024-03-23T14:57:41.620213+0000","last_became_peered":"2024-03-23T14:57:41.620213+0000","last_unstale":"2024-03-23T14:57:41.624283+0000","last_undegraded":"2024-03-23T14:57:41.624283+0000","last_fullsized":"2024-03-23T14:57:41.624283+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_clean_scrub_stamp":"2024-03-23T14:57:40.577321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T00:53:18.316948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"28'2","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.644848+0000","last_change":"2024-03-23T14:57:40.626978+0000","last_active":"2024-03-23T14:57:41.644848+0000","last_peered":"2024-03-23T14:57:41.644848+0000","last_clean":"2024-03-23T14:57:41.644848+0000","last_became_active":"2024-03-23T14:57:40.626675+0000","last_became_peered":"2024-03-23T14:57:40.626675+0000","last_unstale":"2024-03-23T14:57:41.644848+0000","last_undegraded":"2024-03-23T14:57:41.644848+0000","last_fullsized":"2024-03-23T14:57:41.644848+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:39.565698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:39.565698+0000","last_clean_scrub_stamp":"2024-03-23T14:57:39.565698+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:28:31.359748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.639333+0000","last_change":"2024-03-23T14:57:41.624746+0000","last_active":"2024-03-23T14:57:41.639333+0000","last_peered":"2024-03-23T14:57:41.639333+0000","last_clean":"2024-03-23T14:57:41.639333+0000","last_became_active":"2024-03-23T14:57:41.624042+0000","last_became_peered":"2024-03-23T14:57:41.624042+0000","last_unstale":"2024-03-23T14:57:41.639333+0000","last_undegraded":"2024-03-23T14:57:41.639333+0000","last_fullsized":"2024-03-23T14:57:41.639333+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_clean_scrub_stamp":"2024-03-23T14:57:40.577321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:51:58.286721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.596083+0000","last_change":"2024-03-23T14:57:40.611199+0000","last_active":"2024-03-23T14:57:41.596083+0000","last_peered":"2024-03-23T14:57:41.596083+0000","last_clean":"2024-03-23T14:57:41.596083+0000","last_became_active":"2024-03-23T14:57:40.610826+0000","last_became_peered":"2024-03-23T14:57:40.610826+0000","last_unstale":"2024-03-23T14:57:41.596083+0000","last_undegraded":"2024-03-23T14:57:41.596083+0000","last_fullsized":"2024-03-23T14:57:41.596083+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:39.565698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:39.565698+0000","last_clean_scrub_stamp":"2024-03-23T14:57:39.565698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:45:21.729337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.625189+0000","last_change":"2024-03-23T14:57:41.623743+0000","last_active":"2024-03-23T14:57:41.625189+0000","last_peered":"2024-03-23T14:57:41.625189+0000","last_clean":"2024-03-23T14:57:41.625189+0000","last_became_active":"2024-03-23T14:57:41.623409+0000","last_became_peered":"2024-03-23T14:57:41.623409+0000","last_unstale":"2024-03-23T14:57:41.625189+0000","last_undegraded":"2024-03-23T14:57:41.625189+0000","last_fullsized":"2024-03-23T14:57:41.625189+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_clean_scrub_stamp":"2024-03-23T14:57:40.577321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:00:23.723071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.621474+0000","last_change":"2024-03-23T14:57:40.610710+0000","last_active":"2024-03-23T14:57:41.621474+0000","last_peered":"2024-03-23T14:57:41.621474+0000","last_clean":"2024-03-23T14:57:41.621474+0000","last_became_active":"2024-03-23T14:57:40.610229+0000","last_became_peered":"2024-03-23T14:57:40.610229+0000","last_unstale":"2024-03-23T14:57:41.621474+0000","last_undegraded":"2024-03-23T14:57:41.621474+0000","last_fullsized":"2024-03-23T14:57:41.621474+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:39.565698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:39.565698+0000","last_clean_scrub_stamp":"2024-03-23T14:57:39.565698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:11:43.261300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.636622+0000","last_change":"2024-03-23T14:57:41.632119+0000","last_active":"2024-03-23T14:57:41.636622+0000","last_peered":"2024-03-23T14:57:41.636622+0000","last_clean":"2024-03-23T14:57:41.636622+0000","last_became_active":"2024-03-23T14:57:41.631260+0000","last_became_peered":"2024-03-23T14:57:41.631260+0000","last_unstale":"2024-03-23T14:57:41.636622+0000","last_undegraded":"2024-03-23T14:57:41.636622+0000","last_fullsized":"2024-03-23T14:57:41.636622+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_clean_scrub_stamp":"2024-03-23T14:57:40.577321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T21:28:32.371371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.596847+0000","last_change":"2024-03-23T14:57:40.624269+0000","last_active":"2024-03-23T14:57:41.596847+0000","last_peered":"2024-03-23T14:57:41.596847+0000","last_clean":"2024-03-23T14:57:41.596847+0000","last_became_active":"2024-03-23T14:57:40.623914+0000","last_became_peered":"2024-03-23T14:57:40.623914+0000","last_unstale":"2024-03-23T14:57:41.596847+0000","last_undegraded":"2024-03-23T14:57:41.596847+0000","last_fullsized":"2024-03-23T14:57:41.596847+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:39.565698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:39.565698+0000","last_clean_scrub_stamp":"2024-03-23T14:57:39.565698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T23:43:36.776258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.636705+0000","last_change":"2024-03-23T14:57:41.633901+0000","last_active":"2024-03-23T14:57:41.636705+0000","last_peered":"2024-03-23T14:57:41.636705+0000","last_clean":"2024-03-23T14:57:41.636705+0000","last_became_active":"2024-03-23T14:57:41.633566+0000","last_became_peered":"2024-03-23T14:57:41.633566+0000","last_unstale":"2024-03-23T14:57:41.636705+0000","last_undegraded":"2024-03-23T14:57:41.636705+0000","last_fullsized":"2024-03-23T14:57:41.636705+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_clean_scrub_stamp":"2024-03-23T14:57:40.577321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:08:47.903767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.621544+0000","last_change":"2024-03-23T14:57:40.624317+0000","last_active":"2024-03-23T14:57:41.621544+0000","last_peered":"2024-03-23T14:57:41.621544+0000","last_clean":"2024-03-23T14:57:41.621544+0000","last_became_active":"2024-03-23T14:57:40.623908+0000","last_became_peered":"2024-03-23T14:57:40.623908+0000","last_unstale":"2024-03-23T14:57:41.621544+0000","last_undegraded":"2024-03-23T14:57:41.621544+0000","last_fullsized":"2024-03-23T14:57:41.621544+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:39.565698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:39.565698+0000","last_clean_scrub_stamp":"2024-03-23T14:57:39.565698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:34:28.916684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.632669+0000","last_change":"2024-03-23T14:57:41.627317+0000","last_active":"2024-03-23T14:57:41.632669+0000","last_peered":"2024-03-23T14:57:41.632669+0000","last_clean":"2024-03-23T14:57:41.632669+0000","last_became_active":"2024-03-23T14:57:41.626801+0000","last_became_peered":"2024-03-23T14:57:41.626801+0000","last_unstale":"2024-03-23T14:57:41.632669+0000","last_undegraded":"2024-03-23T14:57:41.632669+0000","last_fullsized":"2024-03-23T14:57:41.632669+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_clean_scrub_stamp":"2024-03-23T14:57:40.577321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T18:44:02.215207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.601104+0000","last_change":"2024-03-23T14:57:40.626506+0000","last_active":"2024-03-23T14:57:41.601104+0000","last_peered":"2024-03-23T14:57:41.601104+0000","last_clean":"2024-03-23T14:57:41.601104+0000","last_became_active":"2024-03-23T14:57:40.625694+0000","last_became_peered":"2024-03-23T14:57:40.625694+0000","last_unstale":"2024-03-23T14:57:41.601104+0000","last_undegraded":"2024-03-23T14:57:41.601104+0000","last_fullsized":"2024-03-23T14:57:41.601104+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:39.565698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:39.565698+0000","last_clean_scrub_stamp":"2024-03-23T14:57:39.565698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:42:06.517354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.637890+0000","last_change":"2024-03-23T14:57:41.630440+0000","last_active":"2024-03-23T14:57:41.637890+0000","last_peered":"2024-03-23T14:57:41.637890+0000","last_clean":"2024-03-23T14:57:41.637890+0000","last_became_active":"2024-03-23T14:57:41.629576+0000","last_became_peered":"2024-03-23T14:57:41.629576+0000","last_unstale":"2024-03-23T14:57:41.637890+0000","last_undegraded":"2024-03-23T14:57:41.637890+0000","last_fullsized":"2024-03-23T14:57:41.637890+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_clean_scrub_stamp":"2024-03-23T14:57:40.577321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:08:47.903767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.602536+0000","last_change":"2024-03-23T14:57:40.619843+0000","last_active":"2024-03-23T14:57:41.602536+0000","last_peered":"2024-03-23T14:57:41.602536+0000","last_clean":"2024-03-23T14:57:41.602536+0000","last_became_active":"2024-03-23T14:57:40.619510+0000","last_became_peered":"2024-03-23T14:57:40.619510+0000","last_unstale":"2024-03-23T14:57:41.602536+0000","last_undegraded":"2024-03-23T14:57:41.602536+0000","last_fullsized":"2024-03-23T14:57:41.602536+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:39.565698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:39.565698+0000","last_clean_scrub_stamp":"2024-03-23T14:57:39.565698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T17:10:51.015174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.654526+0000","last_change":"2024-03-23T14:57:41.650093+0000","last_active":"2024-03-23T14:57:41.654526+0000","last_peered":"2024-03-23T14:57:41.654526+0000","last_clean":"2024-03-23T14:57:41.654526+0000","last_became_active":"2024-03-23T14:57:41.641533+0000","last_became_peered":"2024-03-23T14:57:41.641533+0000","last_unstale":"2024-03-23T14:57:41.654526+0000","last_undegraded":"2024-03-23T14:57:41.654526+0000","last_fullsized":"2024-03-23T14:57:41.654526+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:40.577321+0000","last_clean_scrub_stamp":"2024-03-23T14:57:40.577321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T02:51:58.286721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.602934+0000","last_change":"2024-03-23T14:57:40.627054+0000","last_active":"2024-03-23T14:57:41.602934+0000","last_peered":"2024-03-23T14:57:41.602934+0000","last_clean":"2024-03-23T14:57:41.602934+0000","last_became_active":"2024-03-23T14:57:40.626376+0000","last_became_peered":"2024-03-23T14:57:40.626376+0000","last_unstale":"2024-03-23T14:57:41.602934+0000","last_undegraded":"2024-03-23T14:57:41.602934+0000","last_fullsized":"2024-03-23T14:57:41.602934+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:57:39.565698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:57:39.565698+0000","last_clean_scrub_stamp":"2024-03-23T14:57:39.565698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-25T01:07:15.676075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":185,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-23T14:57:41.615973+0000","last_change":"2024-03-23T14:56:48.722063+0000","last_active":"2024-03-23T14:57:41.615973+0000","last_peered":"2024-03-23T14:57:41.615973+0000","last_clean":"2024-03-23T14:57:41.615973+0000","last_became_active":"2024-03-23T14:56:48.720079+0000","last_became_peered":"2024-03-23T14:56:48.720079+0000","last_unstale":"2024-03-23T14:57:41.615973+0000","last_undegraded":"2024-03-23T14:57:41.615973+0000","last_fullsized":"2024-03-23T14:57:41.615973+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:47.478078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:47.478078+0000","last_clean_scrub_stamp":"2024-03-23T14:56:47.478078+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:55:51.231901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574863,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27896,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343944,"statfs":{"total":96636764160,"available":96608198656,"internally_reserved":0,"allocated":892928,"data_stored":701305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574863,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27332,"kb_used_data":312,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344508,"statfs":{"total":96636764160,"available":96608776192,"internally_reserved":0,"allocated":319488,"data_stored":118328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574863,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27316,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344524,"statfs":{"total":96636764160,"available":96608792576,"internally_reserved":0,"allocated":299008,"data_stored":110937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574863,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27332,"kb_used_data":312,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344508,"statfs":{"total":96636764160,"available":96608776192,"internally_reserved":0,"allocated":319488,"data_stored":117816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574864,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27340,"kb_used_data":320,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344500,"statfs":{"total":96636764160,"available":96608768000,"internally_reserved":0,"allocated":327680,"data_stored":119346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574863,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27328,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344512,"statfs":{"total":96636764160,"available":96608780288,"internally_reserved":0,"allocated":315392,"data_stored":117782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574864,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27316,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344524,"statfs":{"total":96636764160,"available":96608792576,"internally_reserved":0,"allocated":299008,"data_stored":110937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574862,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27912,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343928,"statfs":{"total":96636764160,"available":96608182272,"internally_reserved":0,"allocated":905216,"data_stored":708080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-23T14:57:45.485 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-23T14:57:45.486 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-03-23T14:57:45.564 INFO:tasks.ceph.mds.a.smithi164.stderr:2024-03-23T14:57:45.561+0000 7f5013669640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-23T14:57:45.645 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.641+0000 7f9596bf8640 1 Processor -- start 2024-03-23T14:57:45.645 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.641+0000 7f9596bf8640 1 -- start start 2024-03-23T14:57:45.646 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.641+0000 7f9596bf8640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f959002c2a0 0x7f959002c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:45.646 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.641+0000 7f9596bf8640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9590141a90 con 0x7f959014fc00 2024-03-23T14:57:45.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.641+0000 7f9596bf8640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95900b9470 con 0x7f959006e8e0 2024-03-23T14:57:45.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.641+0000 7f9596bf8640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f9590072af0 con 0x7f959002c2a0 2024-03-23T14:57:45.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.641+0000 7f958ffff640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f959002c2a0 0x7f959002c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:45.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.641+0000 7f9595195640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f959014fc00 0x7f9590151ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:46766/0 (socket says 172.21.15.164:46766) 2024-03-23T14:57:45.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.641+0000 7f9595195640 1 -- 172.21.15.164:0/1086990319 learned_addr learned my addr 172.21.15.164:0/1086990319 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-03-23T14:57:45.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.641+0000 7f958ffff640 1 -- 172.21.15.164:0/1086990319 >> v1:172.21.15.173:6789/0 conn(0x7f959006e8e0 legacy=0x7f9590141380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:45.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.641+0000 7f958ffff640 1 -- 172.21.15.164:0/1086990319 >> v1:172.21.15.164:6789/0 conn(0x7f959014fc00 legacy=0x7f9590151ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:45.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.641+0000 7f958f7fe640 1 -- 172.21.15.164:0/1086990319 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4093200779 0 0) 0x7f9590141a90 con 0x7f959014fc00 2024-03-23T14:57:45.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.641+0000 7f958ffff640 1 -- 172.21.15.164:0/1086990319 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f959006ecc0 con 0x7f959002c2a0 2024-03-23T14:57:45.649 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.645+0000 7f958ffff640 1 --2- 172.21.15.164:0/1086990319 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f959002c2a0 0x7f959002c6e0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f95900b9600 tx=0x7f958001ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=893260df7024b761 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:45.649 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.645+0000 7f958f7fe640 1 -- 172.21.15.164:0/1086990319 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95800a8990 con 0x7f959002c2a0 2024-03-23T14:57:45.649 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.645+0000 7f958f7fe640 1 -- 172.21.15.164:0/1086990319 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95800a8af0 con 0x7f959002c2a0 2024-03-23T14:57:45.649 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.645+0000 7f958f7fe640 1 -- 172.21.15.164:0/1086990319 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95800a8e80 con 0x7f959002c2a0 2024-03-23T14:57:45.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.645+0000 7f9596bf8640 1 -- 172.21.15.164:0/1086990319 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f959002c2a0 msgr2=0x7f959002c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:45.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.645+0000 7f9596bf8640 1 --2- 172.21.15.164:0/1086990319 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f959002c2a0 0x7f959002c6e0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f95900b9600 tx=0x7f958001ec90 comp rx=0 tx=0).stop 2024-03-23T14:57:45.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.645+0000 7f9596bf8640 1 -- 172.21.15.164:0/1086990319 shutdown_connections 2024-03-23T14:57:45.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.645+0000 7f9596bf8640 1 --2- 172.21.15.164:0/1086990319 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f959002c2a0 0x7f959002c6e0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:45.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.645+0000 7f9596bf8640 1 -- 172.21.15.164:0/1086990319 >> 172.21.15.164:0/1086990319 conn(0x7f95900b7970 msgr2=0x7f95900b7d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:45.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.645+0000 7f9596bf8640 1 -- 172.21.15.164:0/1086990319 shutdown_connections 2024-03-23T14:57:45.650 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.645+0000 7f9596bf8640 1 -- 172.21.15.164:0/1086990319 wait complete. 2024-03-23T14:57:45.652 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.645+0000 7f9596bf8640 1 Processor -- start 2024-03-23T14:57:45.653 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.645+0000 7f9596bf8640 1 -- start start 2024-03-23T14:57:45.653 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.645+0000 7f9596bf8640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f959002c2a0 0x7f959013a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:45.653 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.645+0000 7f9596bf8640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f959006e8e0 0x7f9590143580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:45.653 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.645+0000 7f9596bf8640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f959014fc00 0x7f95900b5540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:45.653 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.645+0000 7f9596bf8640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9590142e80 con 0x7f959006e8e0 2024-03-23T14:57:45.653 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.645+0000 7f9596bf8640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f959013b800 con 0x7f959002c2a0 2024-03-23T14:57:45.653 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.645+0000 7f9596bf8640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f9590141cc0 con 0x7f959014fc00 2024-03-23T14:57:45.653 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.645+0000 7f958ffff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f959006e8e0 0x7f9590143580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:45.653 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.645+0000 7f958ffff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f959006e8e0 0x7f9590143580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:44762/0 (socket says 172.21.15.164:44762) 2024-03-23T14:57:45.653 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.645+0000 7f958ffff640 1 -- 172.21.15.164:0/1931056828 learned_addr learned my addr 172.21.15.164:0/1931056828 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:45.653 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.645+0000 7f958ffff640 1 -- 172.21.15.164:0/1931056828 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f959014fc00 msgr2=0x7f95900b5540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:45.653 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.645+0000 7f9595195640 1 --2- 172.21.15.164:0/1931056828 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f959014fc00 0x7f95900b5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:45.653 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.645+0000 7f9594994640 1 --2- 172.21.15.164:0/1931056828 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f959002c2a0 0x7f959013a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:45.653 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.645+0000 7f958ffff640 1 --2- 172.21.15.164:0/1931056828 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f959014fc00 0x7f95900b5540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:45.654 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.645+0000 7f958ffff640 1 -- 172.21.15.164:0/1931056828 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f959002c2a0 msgr2=0x7f959013a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:45.654 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.645+0000 7f958ffff640 1 --2- 172.21.15.164:0/1931056828 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f959002c2a0 0x7f959013a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:45.654 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.645+0000 7f958ffff640 1 -- 172.21.15.164:0/1931056828 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9580009420 con 0x7f959006e8e0 2024-03-23T14:57:45.654 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.649+0000 7f958ffff640 1 --2- 172.21.15.164:0/1931056828 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f959006e8e0 0x7f9590143580 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f95800b3040 tx=0x7f958001ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:45.654 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.649+0000 7f958d7fa640 1 -- 172.21.15.164:0/1931056828 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95800a8990 con 0x7f959006e8e0 2024-03-23T14:57:45.654 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.649+0000 7f958d7fa640 1 -- 172.21.15.164:0/1931056828 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95800b9410 con 0x7f959006e8e0 2024-03-23T14:57:45.654 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.649+0000 7f9596bf8640 1 -- 172.21.15.164:0/1931056828 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95900b61b0 con 0x7f959006e8e0 2024-03-23T14:57:45.654 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.649+0000 7f958d7fa640 1 -- 172.21.15.164:0/1931056828 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95800b97c0 con 0x7f959006e8e0 2024-03-23T14:57:45.654 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.649+0000 7f9596bf8640 1 -- 172.21.15.164:0/1931056828 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9590141fa0 con 0x7f959006e8e0 2024-03-23T14:57:45.654 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.649+0000 7f9596bf8640 1 -- 172.21.15.164:0/1931056828 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f959013b660 con 0x7f959006e8e0 2024-03-23T14:57:45.655 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.649+0000 7f958d7fa640 1 -- 172.21.15.164:0/1931056828 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7f95800b7040 con 0x7f959006e8e0 2024-03-23T14:57:45.655 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.649+0000 7f958d7fa640 1 --2- 172.21.15.164:0/1931056828 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f95700672a0 0x7f9570069760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:45.655 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.649+0000 7f958d7fa640 1 -- 172.21.15.164:0/1931056828 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5709+0+0 (secure 0 0 0) 0x7f9580122d00 con 0x7f959006e8e0 2024-03-23T14:57:45.655 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.649+0000 7f9594994640 1 --2- 172.21.15.164:0/1931056828 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f95700672a0 0x7f9570069760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:45.655 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.649+0000 7f9594994640 1 --2- 172.21.15.164:0/1931056828 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f95700672a0 0x7f9570069760 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f95780012f0 tx=0x7f9578009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:45.658 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.653+0000 7f958d7fa640 1 -- 172.21.15.164:0/1931056828 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7f959013b660 con 0x7f959006e8e0 2024-03-23T14:57:45.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.857+0000 7f9596bf8640 1 -- 172.21.15.164:0/1931056828 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f959013b660 con 0x7f959006e8e0 2024-03-23T14:57:45.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.857+0000 7f958d7fa640 1 -- 172.21.15.164:0/1931056828 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) ==== 190+0+147 (secure 0 0 0) 0x7f959013b660 con 0x7f959006e8e0 2024-03-23T14:57:45.863 INFO:teuthology.orchestra.run.smithi164.stdout:[client.0] 2024-03-23T14:57:45.863 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQCB7f5ldBX+BhAAqsE6K6jE0UUaUO+MTCYmOg== 2024-03-23T14:57:45.863 INFO:teuthology.orchestra.run.smithi164.stdout: caps mds = "allow" 2024-03-23T14:57:45.864 INFO:teuthology.orchestra.run.smithi164.stdout: caps mon = "allow r" 2024-03-23T14:57:45.864 INFO:teuthology.orchestra.run.smithi164.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-03-23T14:57:45.864 INFO:teuthology.orchestra.run.smithi164.stderr:updated caps for client.0 2024-03-23T14:57:45.866 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.861+0000 7f9596bf8640 1 -- 172.21.15.164:0/1931056828 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f95700672a0 msgr2=0x7f9570069760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:45.866 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.861+0000 7f9596bf8640 1 --2- 172.21.15.164:0/1931056828 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f95700672a0 0x7f9570069760 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f95780012f0 tx=0x7f9578009040 comp rx=0 tx=0).stop 2024-03-23T14:57:45.867 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.861+0000 7f9596bf8640 1 -- 172.21.15.164:0/1931056828 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f959006e8e0 msgr2=0x7f9590143580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:45.867 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.861+0000 7f9596bf8640 1 --2- 172.21.15.164:0/1931056828 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f959006e8e0 0x7f9590143580 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f95800b3040 tx=0x7f958001ba30 comp rx=0 tx=0).stop 2024-03-23T14:57:45.867 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.861+0000 7f9596bf8640 1 -- 172.21.15.164:0/1931056828 shutdown_connections 2024-03-23T14:57:45.867 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.861+0000 7f9596bf8640 1 --2- 172.21.15.164:0/1931056828 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f959014fc00 0x7f95900b5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:45.867 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.861+0000 7f9596bf8640 1 --2- 172.21.15.164:0/1931056828 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f959006e8e0 0x7f9590143580 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:45.867 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.861+0000 7f9596bf8640 1 --2- 172.21.15.164:0/1931056828 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f95700672a0 0x7f9570069760 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:45.867 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.861+0000 7f9596bf8640 1 --2- 172.21.15.164:0/1931056828 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f959002c2a0 0x7f959013a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:45.867 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.861+0000 7f9596bf8640 1 -- 172.21.15.164:0/1931056828 >> 172.21.15.164:0/1931056828 conn(0x7f95900b7970 msgr2=0x7f959006cf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:45.867 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.861+0000 7f9596bf8640 1 -- 172.21.15.164:0/1931056828 shutdown_connections 2024-03-23T14:57:45.867 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:45.861+0000 7f9596bf8640 1 -- 172.21.15.164:0/1931056828 wait complete. 2024-03-23T14:57:45.882 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T14:57:46.039 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.033+0000 7feb06c3b640 1 Processor -- start 2024-03-23T14:57:46.039 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.033+0000 7feb06c3b640 1 -- start start 2024-03-23T14:57:46.040 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.037+0000 7feb06c3b640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7feb00147890 0x7feb000b0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:46.041 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.037+0000 7feb06c3b640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7feb000707c0 0x7feb00070c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:46.041 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.037+0000 7feb06c3b640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb001480c0 con 0x7feb0013df90 2024-03-23T14:57:46.041 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.037+0000 7feb06c3b640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7feb000705c0 con 0x7feb00147890 2024-03-23T14:57:46.041 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.037+0000 7feb06c3b640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 -- 0x7feb00148550 con 0x7feb000707c0 2024-03-23T14:57:46.041 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.037+0000 7feb049d7640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7feb00147890 0x7feb000b0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:46.041 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.037+0000 7feb049d7640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7feb00147890 0x7feb000b0910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:44770/0 (socket says 172.21.15.164:44770) 2024-03-23T14:57:46.041 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.037+0000 7feb049d7640 1 -- 172.21.15.164:0/3249936472 learned_addr learned my addr 172.21.15.164:0/3249936472 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:46.041 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.037+0000 7feafffff640 1 --2- 172.21.15.164:0/3249936472 >> v2:172.21.15.173:3300/0 conn(0x7feb000707c0 0x7feb00070c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:46.041 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.037+0000 7feb049d7640 1 -- 172.21.15.164:0/3249936472 >> v2:172.21.15.173:3300/0 conn(0x7feb000707c0 msgr2=0x7feb00070c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:46.041 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.037+0000 7feb049d7640 1 --2- 172.21.15.164:0/3249936472 >> v2:172.21.15.173:3300/0 conn(0x7feb000707c0 0x7feb00070c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:46.041 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.037+0000 7feb049d7640 1 -- 172.21.15.164:0/3249936472 >> v1:172.21.15.173:6789/0 conn(0x7feb0013df90 legacy=0x7feb001440c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:46.041 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.037+0000 7feb049d7640 1 -- 172.21.15.164:0/3249936472 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb000725c0 con 0x7feb00147890 2024-03-23T14:57:46.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.037+0000 7feb049d7640 1 --2- 172.21.15.164:0/3249936472 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feb00147890 0x7feb000b0910 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7feb0002d580 tx=0x7feaf401c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b7db5654629bdb1 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:46.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.037+0000 7feaff7fe640 1 -- 172.21.15.164:0/3249936472 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb00144fe0 con 0x7feb00147890 2024-03-23T14:57:46.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.037+0000 7feaff7fe640 1 -- 172.21.15.164:0/3249936472 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feaf40a7050 con 0x7feb00147890 2024-03-23T14:57:46.043 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.037+0000 7feaff7fe640 1 -- 172.21.15.164:0/3249936472 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf40ab820 con 0x7feb00147890 2024-03-23T14:57:46.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.037+0000 7feb06c3b640 1 -- 172.21.15.164:0/3249936472 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feb00147890 msgr2=0x7feb000b0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:46.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.037+0000 7feb06c3b640 1 --2- 172.21.15.164:0/3249936472 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feb00147890 0x7feb000b0910 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7feb0002d580 tx=0x7feaf401c3c0 comp rx=0 tx=0).stop 2024-03-23T14:57:46.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.041+0000 7feb06c3b640 1 -- 172.21.15.164:0/3249936472 shutdown_connections 2024-03-23T14:57:46.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.041+0000 7feb06c3b640 1 --2- 172.21.15.164:0/3249936472 >> v2:172.21.15.173:3300/0 conn(0x7feb000707c0 0x7feb00070c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:46.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.041+0000 7feb06c3b640 1 --2- 172.21.15.164:0/3249936472 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feb00147890 0x7feb000b0910 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:46.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.041+0000 7feb06c3b640 1 -- 172.21.15.164:0/3249936472 >> 172.21.15.164:0/3249936472 conn(0x7feb000b7970 msgr2=0x7feb000b7d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:46.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.041+0000 7feb06c3b640 1 -- 172.21.15.164:0/3249936472 shutdown_connections 2024-03-23T14:57:46.044 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.041+0000 7feb06c3b640 1 -- 172.21.15.164:0/3249936472 wait complete. 2024-03-23T14:57:46.046 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.041+0000 7feb06c3b640 1 Processor -- start 2024-03-23T14:57:46.046 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.041+0000 7feb06c3b640 1 -- start start 2024-03-23T14:57:46.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.041+0000 7feb06c3b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feb000707c0 0x7feb000b0e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:46.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.041+0000 7feb06c3b640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7feb0013df90 0x7feb000b2d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:46.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.041+0000 7feb06c3b640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feb00147890 0x7feb0013eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:46.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.041+0000 7feb06c3b640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7feb000b36e0 con 0x7feb000707c0 2024-03-23T14:57:46.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.041+0000 7feb06c3b640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7feb0013f2e0 con 0x7feb00147890 2024-03-23T14:57:46.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.041+0000 7feb06c3b640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7feb00140320 con 0x7feb0013df90 2024-03-23T14:57:46.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.041+0000 7feb049d7640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feb000707c0 0x7feb000b0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:46.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.041+0000 7feb049d7640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feb000707c0 0x7feb000b0e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:44778/0 (socket says 172.21.15.164:44778) 2024-03-23T14:57:46.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.041+0000 7feb049d7640 1 -- 172.21.15.164:0/1658144319 learned_addr learned my addr 172.21.15.164:0/1658144319 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:46.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.041+0000 7feb049d7640 1 -- 172.21.15.164:0/1658144319 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7feb0013df90 msgr2=0x7feb000b2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:46.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.041+0000 7feb049d7640 1 --2- 172.21.15.164:0/1658144319 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7feb0013df90 0x7feb000b2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:46.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.041+0000 7feb049d7640 1 -- 172.21.15.164:0/1658144319 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feb00147890 msgr2=0x7feb0013eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:46.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.041+0000 7feb049d7640 1 --2- 172.21.15.164:0/1658144319 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feb00147890 0x7feb0013eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:46.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.041+0000 7feb049d7640 1 -- 172.21.15.164:0/1658144319 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feaf40094d0 con 0x7feb000707c0 2024-03-23T14:57:46.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.041+0000 7feb049d7640 1 --2- 172.21.15.164:0/1658144319 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feb000707c0 0x7feb000b0e70 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7feaf401aa70 tx=0x7feaf401ac00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:46.047 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.041+0000 7feafd7fa640 1 -- 172.21.15.164:0/1658144319 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf40ab6c0 con 0x7feb000707c0 2024-03-23T14:57:46.048 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.041+0000 7feafd7fa640 1 -- 172.21.15.164:0/1658144319 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feaf40ab840 con 0x7feb000707c0 2024-03-23T14:57:46.048 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.041+0000 7feb06c3b640 1 -- 172.21.15.164:0/1658144319 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb000b3400 con 0x7feb000707c0 2024-03-23T14:57:46.048 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.041+0000 7feafd7fa640 1 -- 172.21.15.164:0/1658144319 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf40abbf0 con 0x7feb000707c0 2024-03-23T14:57:46.048 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.041+0000 7feb06c3b640 1 -- 172.21.15.164:0/1658144319 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb000b1f10 con 0x7feb000707c0 2024-03-23T14:57:46.048 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.041+0000 7feb06c3b640 1 -- 172.21.15.164:0/1658144319 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb0013eac0 con 0x7feb000707c0 2024-03-23T14:57:46.049 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.045+0000 7feafd7fa640 1 -- 172.21.15.164:0/1658144319 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7feaf40abd50 con 0x7feb000707c0 2024-03-23T14:57:46.049 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.045+0000 7feafd7fa640 1 --2- 172.21.15.164:0/1658144319 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7feaec0672a0 0x7feaec069760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:46.049 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.045+0000 7feafd7fa640 1 -- 172.21.15.164:0/1658144319 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5709+0+0 (secure 0 0 0) 0x7feaf4121180 con 0x7feb000707c0 2024-03-23T14:57:46.049 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.045+0000 7feafffff640 1 --2- 172.21.15.164:0/1658144319 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7feaec0672a0 0x7feaec069760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:46.049 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.045+0000 7feafffff640 1 --2- 172.21.15.164:0/1658144319 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7feaec0672a0 0x7feaec069760 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7feb00141c60 tx=0x7feaf0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:46.052 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.049+0000 7feafd7fa640 1 -- 172.21.15.164:0/1658144319 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7feaf40edea0 con 0x7feb000707c0 2024-03-23T14:57:46.230 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.225+0000 7feb06c3b640 1 -- 172.21.15.164:0/1658144319 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feb0013eac0 con 0x7feb000707c0 2024-03-23T14:57:46.230 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.225+0000 7feafd7fa640 1 -- 172.21.15.164:0/1658144319 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+3537 (secure 0 0 0) 0x7feb0013eac0 con 0x7feb000707c0 2024-03-23T14:57:46.231 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-23T14:57:46.231 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":13,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4619,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/1401585102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":1401585102},{"type":"v1","addr":"172.21.15.173:6833","nonce":1401585102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-23T14:57:40.961416+0000","modified":"2024-03-23T14:57:43.045871+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4935},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4935":{"gid":4935,"name":"a","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.164:6835/3261092607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":3261092607},{"type":"v1","addr":"172.21.15.164:6835","nonce":3261092607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4935,"qdb_cluster":[4935]},"id":2}]} 2024-03-23T14:57:46.231 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 13 2024-03-23T14:57:46.232 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.229+0000 7feb06c3b640 1 -- 172.21.15.164:0/1658144319 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7feaec0672a0 msgr2=0x7feaec069760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:46.232 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.229+0000 7feb06c3b640 1 --2- 172.21.15.164:0/1658144319 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7feaec0672a0 0x7feaec069760 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7feb00141c60 tx=0x7feaf0008040 comp rx=0 tx=0).stop 2024-03-23T14:57:46.233 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.229+0000 7feb06c3b640 1 -- 172.21.15.164:0/1658144319 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feb000707c0 msgr2=0x7feb000b0e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:46.233 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.229+0000 7feb06c3b640 1 --2- 172.21.15.164:0/1658144319 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feb000707c0 0x7feb000b0e70 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7feaf401aa70 tx=0x7feaf401ac00 comp rx=0 tx=0).stop 2024-03-23T14:57:46.233 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.229+0000 7feb06c3b640 1 -- 172.21.15.164:0/1658144319 shutdown_connections 2024-03-23T14:57:46.233 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.229+0000 7feb06c3b640 1 --2- 172.21.15.164:0/1658144319 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7feaec0672a0 0x7feaec069760 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:46.233 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.229+0000 7feb06c3b640 1 --2- 172.21.15.164:0/1658144319 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feb00147890 0x7feb0013eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:46.233 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.229+0000 7feb06c3b640 1 --2- 172.21.15.164:0/1658144319 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7feb0013df90 0x7feb000b2d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:46.233 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.229+0000 7feb06c3b640 1 --2- 172.21.15.164:0/1658144319 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feb000707c0 0x7feb000b0e70 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:46.233 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.229+0000 7feb06c3b640 1 -- 172.21.15.164:0/1658144319 >> 172.21.15.164:0/1658144319 conn(0x7feb000b7970 msgr2=0x7feb000b5b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:46.233 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.229+0000 7feb06c3b640 1 -- 172.21.15.164:0/1658144319 shutdown_connections 2024-03-23T14:57:46.233 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.229+0000 7feb06c3b640 1 -- 172.21.15.164:0/1658144319 wait complete. 2024-03-23T14:57:46.449 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 13, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-23T14:57:40.961416+0000', 'modified': '2024-03-23T14:57:43.045871+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4935}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4935': {'gid': 4935, 'name': 'a', 'rank': 0, 'incarnation': 11, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.164:6835/3261092607', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6834', 'nonce': 3261092607}, {'type': 'v1', 'addr': '172.21.15.164:6835', 'nonce': 3261092607}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4935, 'qdb_cluster': [4935]} 2024-03-23T14:57:46.449 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-23T14:57:46.449 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-03-23T14:57:46.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.573+0000 7f05970ce640 1 Processor -- start 2024-03-23T14:57:46.576 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.573+0000 7f05970ce640 1 -- start start 2024-03-23T14:57:46.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.573+0000 7f05970ce640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f05900b9ac0 0x7f05900ae920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:46.577 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.573+0000 7f05970ce640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f05900adc70 0x7f05901445e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:46.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.573+0000 7f05970ce640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05900b9ea0 con 0x7f0590144b20 2024-03-23T14:57:46.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.573+0000 7f05970ce640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f05900af400 con 0x7f05900b9ac0 2024-03-23T14:57:46.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.573+0000 7f05970ce640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f0590144fd0 con 0x7f05900adc70 2024-03-23T14:57:46.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.573+0000 7f0594e6a640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f05900b9ac0 0x7f05900ae920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:46.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.573+0000 7f0594e6a640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f05900b9ac0 0x7f05900ae920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:44792/0 (socket says 172.21.15.164:44792) 2024-03-23T14:57:46.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.573+0000 7f0594e6a640 1 -- 172.21.15.164:0/1214021811 learned_addr learned my addr 172.21.15.164:0/1214021811 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:46.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.573+0000 7f058ffff640 1 --2- 172.21.15.164:0/1214021811 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f05900adc70 0x7f05901445e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:46.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.573+0000 7f0594e6a640 1 -- 172.21.15.164:0/1214021811 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f05900adc70 msgr2=0x7f05901445e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:46.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.573+0000 7f0594e6a640 1 --2- 172.21.15.164:0/1214021811 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f05900adc70 0x7f05901445e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:46.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.573+0000 7f0594e6a640 1 -- 172.21.15.164:0/1214021811 >> v1:172.21.15.173:6789/0 conn(0x7f0590144b20 legacy=0x7f05901477b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:46.578 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.573+0000 7f0594e6a640 1 -- 172.21.15.164:0/1214021811 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05900b53e0 con 0x7f05900b9ac0 2024-03-23T14:57:46.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.573+0000 7f0594e6a640 1 --2- 172.21.15.164:0/1214021811 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f05900b9ac0 0x7f05900ae920 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f05840098a0 tx=0x7f058401c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9519480d893020e2 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:46.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.573+0000 7f058f7fe640 1 -- 172.21.15.164:0/1214021811 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05840ab600 con 0x7f05900b9ac0 2024-03-23T14:57:46.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.573+0000 7f058f7fe640 1 -- 172.21.15.164:0/1214021811 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05840ab760 con 0x7f05900b9ac0 2024-03-23T14:57:46.579 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.573+0000 7f058f7fe640 1 -- 172.21.15.164:0/1214021811 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05840abaf0 con 0x7f05900b9ac0 2024-03-23T14:57:46.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.577+0000 7f05970ce640 1 -- 172.21.15.164:0/1214021811 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f05900b9ac0 msgr2=0x7f05900ae920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:46.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.577+0000 7f05970ce640 1 --2- 172.21.15.164:0/1214021811 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f05900b9ac0 0x7f05900ae920 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f05840098a0 tx=0x7f058401c440 comp rx=0 tx=0).stop 2024-03-23T14:57:46.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.577+0000 7f05970ce640 1 -- 172.21.15.164:0/1214021811 shutdown_connections 2024-03-23T14:57:46.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.577+0000 7f05970ce640 1 --2- 172.21.15.164:0/1214021811 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f05900adc70 0x7f05901445e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:46.580 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.577+0000 7f05970ce640 1 --2- 172.21.15.164:0/1214021811 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f05900b9ac0 0x7f05900ae920 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:46.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.577+0000 7f05970ce640 1 -- 172.21.15.164:0/1214021811 >> 172.21.15.164:0/1214021811 conn(0x7f059006b930 msgr2=0x7f05900b9150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:46.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.577+0000 7f05970ce640 1 -- 172.21.15.164:0/1214021811 shutdown_connections 2024-03-23T14:57:46.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.577+0000 7f05970ce640 1 -- 172.21.15.164:0/1214021811 wait complete. 2024-03-23T14:57:46.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.577+0000 7f05970ce640 1 Processor -- start 2024-03-23T14:57:46.581 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.577+0000 7f05970ce640 1 -- start start 2024-03-23T14:57:46.583 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.577+0000 7f05970ce640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f05900adc70 0x7f059013f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:46.583 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.577+0000 7f05970ce640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f05900b9ac0 0x7f05901dfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:46.583 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.577+0000 7f05970ce640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0590144b20 0x7f05901e0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:46.583 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.577+0000 7f05970ce640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f059006fd60 con 0x7f05900adc70 2024-03-23T14:57:46.583 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.577+0000 7f05970ce640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f05901e0840 con 0x7f05900b9ac0 2024-03-23T14:57:46.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.577+0000 7f05970ce640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f0590070cd0 con 0x7f0590144b20 2024-03-23T14:57:46.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.577+0000 7f0594e6a640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f05900adc70 0x7f059013f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:46.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.577+0000 7f0594e6a640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f05900adc70 0x7f059013f580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:44804/0 (socket says 172.21.15.164:44804) 2024-03-23T14:57:46.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.577+0000 7f0594e6a640 1 -- 172.21.15.164:0/779999314 learned_addr learned my addr 172.21.15.164:0/779999314 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:46.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.577+0000 7f058ffff640 1 --2- 172.21.15.164:0/779999314 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f05900b9ac0 0x7f05901dfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:46.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.577+0000 7f059566b640 1 --2- 172.21.15.164:0/779999314 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0590144b20 0x7f05901e0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:46.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.577+0000 7f0594e6a640 1 -- 172.21.15.164:0/779999314 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0590144b20 msgr2=0x7f05901e0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:46.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.577+0000 7f0594e6a640 1 --2- 172.21.15.164:0/779999314 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0590144b20 0x7f05901e0300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:46.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.577+0000 7f0594e6a640 1 -- 172.21.15.164:0/779999314 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f05900b9ac0 msgr2=0x7f05901dfdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:46.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.577+0000 7f0594e6a640 1 --2- 172.21.15.164:0/779999314 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f05900b9ac0 0x7f05901dfdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:46.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.577+0000 7f0594e6a640 1 -- 172.21.15.164:0/779999314 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05840094d0 con 0x7f05900adc70 2024-03-23T14:57:46.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.577+0000 7f058ffff640 1 --2- 172.21.15.164:0/779999314 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f05900b9ac0 0x7f05901dfdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T14:57:46.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.577+0000 7f059566b640 1 --2- 172.21.15.164:0/779999314 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0590144b20 0x7f05901e0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T14:57:46.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.577+0000 7f0594e6a640 1 --2- 172.21.15.164:0/779999314 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f05900adc70 0x7f059013f580 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f05840ad040 tx=0x7f058401ac80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:46.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.577+0000 7f058d7fa640 1 -- 172.21.15.164:0/779999314 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05840b4800 con 0x7f05900adc70 2024-03-23T14:57:46.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.577+0000 7f058d7fa640 1 -- 172.21.15.164:0/779999314 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05840b4980 con 0x7f05900adc70 2024-03-23T14:57:46.584 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.577+0000 7f05970ce640 1 -- 172.21.15.164:0/779999314 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0590138420 con 0x7f05900adc70 2024-03-23T14:57:46.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.577+0000 7f058d7fa640 1 -- 172.21.15.164:0/779999314 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05840b4d30 con 0x7f05900adc70 2024-03-23T14:57:46.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.577+0000 7f05970ce640 1 -- 172.21.15.164:0/779999314 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f059006f980 con 0x7f05900adc70 2024-03-23T14:57:46.585 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.581+0000 7f058d7fa640 1 -- 172.21.15.164:0/779999314 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7f05840c7440 con 0x7f05900adc70 2024-03-23T14:57:46.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.581+0000 7f058d7fa640 1 --2- 172.21.15.164:0/779999314 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f05740672a0 0x7f0574069760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:46.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.581+0000 7f058ffff640 1 --2- 172.21.15.164:0/779999314 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f05740672a0 0x7f0574069760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:46.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.581+0000 7f058d7fa640 1 -- 172.21.15.164:0/779999314 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5709+0+0 (secure 0 0 0) 0x7f058411fba0 con 0x7f05900adc70 2024-03-23T14:57:46.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.581+0000 7f058ffff640 1 --2- 172.21.15.164:0/779999314 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f05740672a0 0x7f0574069760 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f057c006fd0 tx=0x7f057c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:46.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.581+0000 7f059a243000 1 Processor -- start 2024-03-23T14:57:46.586 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.581+0000 7f059a243000 1 -- start start 2024-03-23T14:57:46.588 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.581+0000 7f059a243000 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x563e5f74bc10 0x563e5f746ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:46.588 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.581+0000 7f059a243000 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563e5f74bff0 0x563e5f7473e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:46.588 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.581+0000 7f059a243000 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563e5f747920 0x563e5f7589d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:46.588 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.581+0000 7f059a243000 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x563e5f5e4ed0 con 0x563e5f74bff0 2024-03-23T14:57:46.588 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.581+0000 7f059a243000 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x563e5f5769e0 con 0x563e5f747920 2024-03-23T14:57:46.588 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.581+0000 7f059a243000 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x563e5f573f10 con 0x563e5f74bc10 2024-03-23T14:57:46.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.581+0000 7f058ffff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563e5f74bff0 0x563e5f7473e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:46.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.581+0000 7f058ffff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563e5f74bff0 0x563e5f7473e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:44820/0 (socket says 172.21.15.164:44820) 2024-03-23T14:57:46.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.581+0000 7f058ffff640 1 -- 172.21.15.164:0/2796544589 learned_addr learned my addr 172.21.15.164:0/2796544589 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:46.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.581+0000 7f0594e6a640 1 --2- 172.21.15.164:0/2796544589 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x563e5f74bc10 0x563e5f746ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:46.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.581+0000 7f059566b640 1 --2- 172.21.15.164:0/2796544589 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563e5f747920 0x563e5f7589d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:46.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.581+0000 7f058ffff640 1 -- 172.21.15.164:0/2796544589 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x563e5f74bc10 msgr2=0x563e5f746ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:46.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.581+0000 7f058ffff640 1 --2- 172.21.15.164:0/2796544589 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x563e5f74bc10 0x563e5f746ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:46.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.581+0000 7f058ffff640 1 -- 172.21.15.164:0/2796544589 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563e5f747920 msgr2=0x563e5f7589d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:46.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.581+0000 7f058ffff640 1 --2- 172.21.15.164:0/2796544589 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563e5f747920 0x563e5f7589d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:46.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.581+0000 7f058ffff640 1 -- 172.21.15.164:0/2796544589 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e5f265890 con 0x563e5f74bff0 2024-03-23T14:57:46.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.581+0000 7f059566b640 1 --2- 172.21.15.164:0/2796544589 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563e5f747920 0x563e5f7589d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T14:57:46.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.581+0000 7f058ffff640 1 --2- 172.21.15.164:0/2796544589 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563e5f74bff0 0x563e5f7473e0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f057c00db60 tx=0x7f057c010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68c33f0631b1fd70 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:46.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.581+0000 7f057a7fc640 1 -- 172.21.15.164:0/2796544589 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f057c00b050 con 0x563e5f74bff0 2024-03-23T14:57:46.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.581+0000 7f057a7fc640 1 -- 172.21.15.164:0/2796544589 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f057c017070 con 0x563e5f74bff0 2024-03-23T14:57:46.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.581+0000 7f057a7fc640 1 -- 172.21.15.164:0/2796544589 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f057c012c10 con 0x563e5f74bff0 2024-03-23T14:57:46.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.581+0000 7f059a243000 1 -- 172.21.15.164:0/2796544589 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563e5f74bff0 msgr2=0x563e5f7473e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:46.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.581+0000 7f059a243000 1 --2- 172.21.15.164:0/2796544589 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563e5f74bff0 0x563e5f7473e0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f057c00db60 tx=0x7f057c010000 comp rx=0 tx=0).stop 2024-03-23T14:57:46.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.585+0000 7f059a243000 1 -- 172.21.15.164:0/2796544589 shutdown_connections 2024-03-23T14:57:46.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.585+0000 7f059a243000 1 --2- 172.21.15.164:0/2796544589 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563e5f747920 0x563e5f7589d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:46.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.585+0000 7f059a243000 1 --2- 172.21.15.164:0/2796544589 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563e5f74bff0 0x563e5f7473e0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:46.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.585+0000 7f059a243000 1 --2- 172.21.15.164:0/2796544589 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x563e5f74bc10 0x563e5f746ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:46.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.585+0000 7f059a243000 1 -- 172.21.15.164:0/2796544589 >> 172.21.15.164:0/2796544589 conn(0x563e5f714960 msgr2=0x563e5f74b500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:46.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.585+0000 7f059a243000 1 -- 172.21.15.164:0/2796544589 shutdown_connections 2024-03-23T14:57:46.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.585+0000 7f059a243000 1 -- 172.21.15.164:0/2796544589 wait complete. 2024-03-23T14:57:46.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.585+0000 7f059a243000 1 -- start start 2024-03-23T14:57:46.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.585+0000 7f059a243000 1 Processor -- start 2024-03-23T14:57:46.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.585+0000 7f059a243000 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x563e5f747920 0x563e5f722a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:46.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.585+0000 7f059a243000 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563e5f74bc10 0x563e5f71fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:46.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.585+0000 7f059a243000 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563e5f74bff0 0x563e5f720300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:46.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.585+0000 7f059a243000 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x563e5f56f6f0 con 0x563e5f74bc10 2024-03-23T14:57:46.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.585+0000 7f059a243000 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x563e5f5096c0 con 0x563e5f74bff0 2024-03-23T14:57:46.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.585+0000 7f059a243000 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x563e5f509360 con 0x563e5f747920 2024-03-23T14:57:46.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.585+0000 7f059a243000 10 client.0 _ll_register_callbacks cb 0x563e5f52a280 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T14:57:46.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.585+0000 7f058ffff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563e5f74bc10 0x563e5f71fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:46.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.585+0000 7f058ffff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563e5f74bc10 0x563e5f71fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:44830/0 (socket says 172.21.15.164:44830) 2024-03-23T14:57:46.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.585+0000 7f058ffff640 1 -- 172.21.15.164:0/4160332557 learned_addr learned my addr 172.21.15.164:0/4160332557 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:46.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.585+0000 7f0594e6a640 1 --2- 172.21.15.164:0/4160332557 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x563e5f747920 0x563e5f722a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:46.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.585+0000 7f059566b640 1 --2- 172.21.15.164:0/4160332557 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563e5f74bff0 0x563e5f720300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:46.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.585+0000 7f058ffff640 1 -- 172.21.15.164:0/4160332557 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x563e5f747920 msgr2=0x563e5f722a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:46.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.585+0000 7f058ffff640 1 --2- 172.21.15.164:0/4160332557 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x563e5f747920 0x563e5f722a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:46.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.585+0000 7f058ffff640 1 -- 172.21.15.164:0/4160332557 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563e5f74bff0 msgr2=0x563e5f720300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:46.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.585+0000 7f058ffff640 1 --2- 172.21.15.164:0/4160332557 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563e5f74bff0 0x563e5f720300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:46.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.585+0000 7f058ffff640 1 -- 172.21.15.164:0/4160332557 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f057c00d810 con 0x563e5f74bc10 2024-03-23T14:57:46.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.585+0000 7f058ffff640 1 --2- 172.21.15.164:0/4160332557 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563e5f74bc10 0x563e5f71fdc0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f057c00db00 tx=0x7f057c004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b59cda25c30fcc6 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:46.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.585+0000 7f056f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-03-23T14:57:46.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.585+0000 7f056f7fe640 1 -- 172.21.15.164:0/4160332557 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f057c00b040 con 0x563e5f74bc10 2024-03-23T14:57:46.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.585+0000 7f059a243000 1 -- 172.21.15.164:0/4160332557 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x563e5f5228b0 con 0x563e5f74bc10 2024-03-23T14:57:46.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.585+0000 7f056f7fe640 1 -- 172.21.15.164:0/4160332557 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(29..29 src has 1..29) ==== 5709+0+0 (secure 0 0 0) 0x7f057c01e9d0 con 0x563e5f74bc10 2024-03-23T14:57:46.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.585+0000 7f056f7fe640 1 -- 172.21.15.164:0/4160332557 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=13) ==== 24+0+0 (secure 0 0 0) 0x7f057c01eb90 con 0x563e5f74bc10 2024-03-23T14:57:46.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.585+0000 7f059a243000 10 client.5112 fetch_fsmap learned FSMap version 13 2024-03-23T14:57:46.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.585+0000 7f059a243000 1 -- 172.21.15.164:0/4160332557 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=13,monmap=2+}) -- 0x563e5f52eed0 con 0x563e5f74bc10 2024-03-23T14:57:46.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.585+0000 7f056f7fe640 1 -- 172.21.15.164:0/4160332557 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x563e5f5228b0 con 0x563e5f74bc10 2024-03-23T14:57:46.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.585+0000 7f056f7fe640 1 -- 172.21.15.164:0/4160332557 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 13) ==== 2284+0+0 (secure 0 0 0) 0x7f057c01ecf0 con 0x563e5f74bc10 2024-03-23T14:57:46.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.585+0000 7f059a243000 10 client.5112 fetch_fsmap finished waiting for FSMap version 13 2024-03-23T14:57:46.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.585+0000 7f059a243000 10 client.5112 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-23T14:57:46.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.585+0000 7f059a243000 20 client.5112 populate_metadata read hostname 'smithi164' 2024-03-23T14:57:46.596 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.585+0000 7f059a243000 1 --2- 172.21.15.164:0/4160332557 >> [v2:172.21.15.164:6834/3261092607,v1:172.21.15.164:6835/3261092607] conn(0x563e5f7627e0 0x563e5f764bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:46.596 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.585+0000 7f059a243000 4 client.5112 mds_command: new command op to 4935 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-23T14:57:46.596 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.585+0000 7f059a243000 1 -- 172.21.15.164:0/4160332557 --> [v2:172.21.15.164:6834/3261092607,v1:172.21.15.164:6835/3261092607] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563e5f56c4d0 con 0x563e5f7627e0 2024-03-23T14:57:46.596 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.585+0000 7f0594e6a640 1 --2- 172.21.15.164:0/4160332557 >> [v2:172.21.15.164:6834/3261092607,v1:172.21.15.164:6835/3261092607] conn(0x563e5f7627e0 0x563e5f764bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:46.596 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.585+0000 7f0594e6a640 1 --2- 172.21.15.164:0/4160332557 >> [v2:172.21.15.164:6834/3261092607,v1:172.21.15.164:6835/3261092607] conn(0x563e5f7627e0 0x563e5f764bd0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dafb7f7dfef3c588 server_cookie=e35d2d776736c4c0 in_seq=0 out_seq=0 2024-03-23T14:57:46.596 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.585+0000 7f056f7fe640 10 client.5112 ms_handle_connect on v2:172.21.15.164:6834/3261092607 2024-03-23T14:57:46.596 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.585+0000 7f056f7fe640 1 -- 172.21.15.164:0/4160332557 <== mds.0 v2:172.21.15.164:6834/3261092607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x563e5f56c4d0 con 0x563e5f7627e0 2024-03-23T14:57:46.596 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.585+0000 7f056f7fe640 10 client.5112 handle_command_reply: tid=0 2024-03-23T14:57:46.596 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.585+0000 7f059a243000 1 client.5112 shutdown 2024-03-23T14:57:46.596 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.585+0000 7f059a243000 1 -- 172.21.15.164:0/4160332557 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563e5f74bc10 msgr2=0x563e5f71fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:46.596 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.585+0000 7f059a243000 1 --2- 172.21.15.164:0/4160332557 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563e5f74bc10 0x563e5f71fdc0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f057c00db00 tx=0x7f057c004740 comp rx=0 tx=0).stop 2024-03-23T14:57:46.596 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.585+0000 7f059a243000 1 -- 172.21.15.164:0/4160332557 shutdown_connections 2024-03-23T14:57:46.596 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.585+0000 7f059a243000 1 --2- 172.21.15.164:0/4160332557 >> [v2:172.21.15.164:6834/3261092607,v1:172.21.15.164:6835/3261092607] conn(0x563e5f7627e0 0x563e5f764bd0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:46.596 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.585+0000 7f059a243000 1 --2- 172.21.15.164:0/4160332557 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563e5f74bff0 0x563e5f720300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:46.596 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.585+0000 7f059a243000 1 --2- 172.21.15.164:0/4160332557 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563e5f74bc10 0x563e5f71fdc0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:46.596 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.585+0000 7f056f7fe640 0 client.5112 ms_handle_reset on v2:172.21.15.164:6834/3261092607 2024-03-23T14:57:46.596 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.585+0000 7f059a243000 1 --2- 172.21.15.164:0/4160332557 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x563e5f747920 0x563e5f722a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:46.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.585+0000 7f059a243000 1 -- 172.21.15.164:0/4160332557 >> 172.21.15.164:0/4160332557 conn(0x563e5f714960 msgr2=0x563e5f748590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:46.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.585+0000 7f059a243000 1 -- 172.21.15.164:0/4160332557 shutdown_connections 2024-03-23T14:57:46.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.585+0000 7f059a243000 1 -- 172.21.15.164:0/4160332557 wait complete. 2024-03-23T14:57:46.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.589+0000 7f059a243000 20 client.5112 trim_cache size 0 max 16384 2024-03-23T14:57:46.608 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.605+0000 7f059a243000 1 Processor -- start 2024-03-23T14:57:46.608 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.605+0000 7f059a243000 1 -- start start 2024-03-23T14:57:46.610 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.605+0000 7f059a243000 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563e5f714960 0x563e5f767e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:46.610 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.605+0000 7f059a243000 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563e5f747920 0x563e5f71f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:46.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.605+0000 7f059a243000 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x563e5f74bc10 0x563e5f745980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:46.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.605+0000 7f059a243000 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x563e5f226940 con 0x563e5f714960 2024-03-23T14:57:46.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.605+0000 7f059a243000 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x563e5f577430 con 0x563e5f747920 2024-03-23T14:57:46.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.605+0000 7f059a243000 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x563e5f5736f0 con 0x563e5f74bc10 2024-03-23T14:57:46.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.605+0000 7f0594e6a640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563e5f714960 0x563e5f767e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:46.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.605+0000 7f0594e6a640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563e5f714960 0x563e5f767e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:44840/0 (socket says 172.21.15.164:44840) 2024-03-23T14:57:46.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.605+0000 7f0594e6a640 1 -- 172.21.15.164:0/2320970880 learned_addr learned my addr 172.21.15.164:0/2320970880 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:46.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.605+0000 7f0594e6a640 1 -- 172.21.15.164:0/2320970880 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x563e5f74bc10 msgr2=0x563e5f745980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:46.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.605+0000 7f0594e6a640 1 --2- 172.21.15.164:0/2320970880 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x563e5f74bc10 0x563e5f745980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:46.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.605+0000 7f0594e6a640 1 -- 172.21.15.164:0/2320970880 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563e5f747920 msgr2=0x563e5f71f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:46.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.605+0000 7f0594e6a640 1 --2- 172.21.15.164:0/2320970880 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563e5f747920 0x563e5f71f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:46.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.605+0000 7f0594e6a640 1 -- 172.21.15.164:0/2320970880 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e5f514e70 con 0x563e5f714960 2024-03-23T14:57:46.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.605+0000 7f0594e6a640 1 --2- 172.21.15.164:0/2320970880 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563e5f714960 0x563e5f767e10 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f058401c920 tx=0x7f05840c0040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8015ea4d3d3d702f server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:46.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.605+0000 7f0579ffb640 1 -- 172.21.15.164:0/2320970880 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f058411e140 con 0x563e5f714960 2024-03-23T14:57:46.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.605+0000 7f0579ffb640 1 -- 172.21.15.164:0/2320970880 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0584108030 con 0x563e5f714960 2024-03-23T14:57:46.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.605+0000 7f0579ffb640 1 -- 172.21.15.164:0/2320970880 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05840ecb10 con 0x563e5f714960 2024-03-23T14:57:46.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.605+0000 7f059a243000 1 -- 172.21.15.164:0/2320970880 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563e5f714960 msgr2=0x563e5f767e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:46.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.605+0000 7f059a243000 1 --2- 172.21.15.164:0/2320970880 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563e5f714960 0x563e5f767e10 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f058401c920 tx=0x7f05840c0040 comp rx=0 tx=0).stop 2024-03-23T14:57:46.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.605+0000 7f059a243000 1 -- 172.21.15.164:0/2320970880 shutdown_connections 2024-03-23T14:57:46.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.605+0000 7f059a243000 1 --2- 172.21.15.164:0/2320970880 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x563e5f74bc10 0x563e5f745980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:46.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.605+0000 7f059a243000 1 --2- 172.21.15.164:0/2320970880 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563e5f747920 0x563e5f71f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:46.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.605+0000 7f059a243000 1 --2- 172.21.15.164:0/2320970880 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563e5f714960 0x563e5f767e10 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:46.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.605+0000 7f059a243000 1 -- 172.21.15.164:0/2320970880 >> 172.21.15.164:0/2320970880 conn(0x563e5f71bc50 msgr2=0x563e5f721c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:46.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.605+0000 7f059a243000 1 -- 172.21.15.164:0/2320970880 shutdown_connections 2024-03-23T14:57:46.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.605+0000 7f059a243000 1 -- 172.21.15.164:0/2320970880 wait complete. 2024-03-23T14:57:46.616 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.605+0000 7f059a243000 1 -- start start 2024-03-23T14:57:46.616 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.605+0000 7f059a243000 1 Processor -- start 2024-03-23T14:57:46.616 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.609+0000 7f059a243000 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563e5f714960 0x563e5f767fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:46.616 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.609+0000 7f059a243000 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563e5f747920 0x563e5f766130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:46.616 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.609+0000 7f059a243000 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x563e5f74bc10 0x563e5f766670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:46.616 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.609+0000 7f059a243000 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x563e5f52c720 con 0x563e5f747920 2024-03-23T14:57:46.617 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.609+0000 7f059a243000 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x563e5f576310 con 0x563e5f714960 2024-03-23T14:57:46.617 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.609+0000 7f059a243000 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x563e5f578400 con 0x563e5f74bc10 2024-03-23T14:57:46.617 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.609+0000 7f059a243000 10 client.0 _ll_register_callbacks cb 0x7f057c00da30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-23T14:57:46.617 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.609+0000 7f058ffff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563e5f747920 0x563e5f766130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:46.617 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.609+0000 7f058ffff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563e5f747920 0x563e5f766130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:44854/0 (socket says 172.21.15.164:44854) 2024-03-23T14:57:46.617 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.609+0000 7f058ffff640 1 -- 172.21.15.164:0/67939041 learned_addr learned my addr 172.21.15.164:0/67939041 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:46.617 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.609+0000 7f058ffff640 1 -- 172.21.15.164:0/67939041 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x563e5f74bc10 msgr2=0x563e5f766670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:46.617 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.609+0000 7f058ffff640 1 --2- 172.21.15.164:0/67939041 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x563e5f74bc10 0x563e5f766670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:46.617 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.609+0000 7f058ffff640 1 -- 172.21.15.164:0/67939041 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563e5f714960 msgr2=0x563e5f767fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:46.617 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.609+0000 7f058ffff640 1 --2- 172.21.15.164:0/67939041 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563e5f714960 0x563e5f767fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:46.617 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.609+0000 7f058ffff640 1 -- 172.21.15.164:0/67939041 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f059006f980 con 0x563e5f747920 2024-03-23T14:57:46.617 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.609+0000 7f058ffff640 1 --2- 172.21.15.164:0/67939041 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563e5f747920 0x563e5f766130 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f057c008010 tx=0x7f057c0046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b2246d5a229c269 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:46.617 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.609+0000 7f056f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.164:3300/0 2024-03-23T14:57:46.617 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.609+0000 7f056f7fe640 1 -- 172.21.15.164:0/67939041 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f057c00b040 con 0x563e5f747920 2024-03-23T14:57:46.617 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.609+0000 7f056f7fe640 1 -- 172.21.15.164:0/67939041 <== mon.0 v2:172.21.15.164:3300/0 2 ==== osd_map(29..29 src has 1..29) ==== 5709+0+0 (secure 0 0 0) 0x7f057c01ebe0 con 0x563e5f747920 2024-03-23T14:57:46.617 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.609+0000 7f059a243000 1 -- 172.21.15.164:0/67939041 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x563e5f576d10 con 0x563e5f747920 2024-03-23T14:57:46.617 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.609+0000 7f056f7fe640 1 -- 172.21.15.164:0/67939041 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_get_version_reply(handle=1 version=13) ==== 24+0+0 (secure 0 0 0) 0x7f057c01eda0 con 0x563e5f747920 2024-03-23T14:57:46.618 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.609+0000 7f059a243000 10 client.5118 fetch_fsmap learned FSMap version 13 2024-03-23T14:57:46.618 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.609+0000 7f059a243000 1 -- 172.21.15.164:0/67939041 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,fsmap=13,monmap=2+}) -- 0x563e5f265cc0 con 0x563e5f747920 2024-03-23T14:57:46.618 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.609+0000 7f056f7fe640 1 -- 172.21.15.164:0/67939041 <== mon.0 v2:172.21.15.164:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x563e5f576d10 con 0x563e5f747920 2024-03-23T14:57:46.618 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.609+0000 7f056f7fe640 1 -- 172.21.15.164:0/67939041 <== mon.0 v2:172.21.15.164:3300/0 5 ==== fsmap(e 13) ==== 2284+0+0 (secure 0 0 0) 0x7f057c046430 con 0x563e5f747920 2024-03-23T14:57:46.618 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.609+0000 7f059a243000 10 client.5118 fetch_fsmap finished waiting for FSMap version 13 2024-03-23T14:57:46.618 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.609+0000 7f059a243000 10 client.5118 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-23T14:57:46.618 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.609+0000 7f059a243000 20 client.5118 populate_metadata read hostname 'smithi164' 2024-03-23T14:57:46.618 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.609+0000 7f059a243000 1 --2- 172.21.15.164:0/67939041 >> [v2:172.21.15.164:6834/3261092607,v1:172.21.15.164:6835/3261092607] conn(0x563e5f74bff0 0x563e5f767000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:46.618 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.609+0000 7f059a243000 4 client.5118 mds_command: new command op to 4935 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-23T14:57:46.618 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.609+0000 7f059a243000 1 -- 172.21.15.164:0/67939041 --> [v2:172.21.15.164:6834/3261092607,v1:172.21.15.164:6835/3261092607] -- command(tid 0: {"prefix": "status"}) -- 0x563e5f59a9e0 con 0x563e5f74bff0 2024-03-23T14:57:46.618 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.609+0000 7f0594e6a640 1 --2- 172.21.15.164:0/67939041 >> [v2:172.21.15.164:6834/3261092607,v1:172.21.15.164:6835/3261092607] conn(0x563e5f74bff0 0x563e5f767000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:46.618 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.609+0000 7f056f7fe640 10 client.5118 ms_handle_connect on v2:172.21.15.164:6834/3261092607 2024-03-23T14:57:46.618 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.609+0000 7f0594e6a640 1 --2- 172.21.15.164:0/67939041 >> [v2:172.21.15.164:6834/3261092607,v1:172.21.15.164:6835/3261092607] conn(0x563e5f74bff0 0x563e5f767000 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c5570446925e004 server_cookie=7456d8e19217d543 in_seq=0 out_seq=0 2024-03-23T14:57:46.618 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.609+0000 7f056f7fe640 1 -- 172.21.15.164:0/67939041 <== mds.0 v2:172.21.15.164:6834/3261092607 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x563e5f59a9e0 con 0x563e5f74bff0 2024-03-23T14:57:46.619 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.609+0000 7f056f7fe640 10 client.5118 handle_command_reply: tid=0 2024-03-23T14:57:46.619 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.609+0000 7f059a243000 1 client.5118 shutdown 2024-03-23T14:57:46.619 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.609+0000 7f059a243000 1 -- 172.21.15.164:0/67939041 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563e5f747920 msgr2=0x563e5f766130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:46.619 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.609+0000 7f059a243000 1 --2- 172.21.15.164:0/67939041 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563e5f747920 0x563e5f766130 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f057c008010 tx=0x7f057c0046d0 comp rx=0 tx=0).stop 2024-03-23T14:57:46.619 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.609+0000 7f059a243000 1 -- 172.21.15.164:0/67939041 shutdown_connections 2024-03-23T14:57:46.619 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.609+0000 7f059a243000 1 --2- 172.21.15.164:0/67939041 >> [v2:172.21.15.164:6834/3261092607,v1:172.21.15.164:6835/3261092607] conn(0x563e5f74bff0 0x563e5f767000 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:46.619 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.609+0000 7f059a243000 1 --2- 172.21.15.164:0/67939041 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x563e5f74bc10 0x563e5f766670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:46.619 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.609+0000 7f059a243000 1 --2- 172.21.15.164:0/67939041 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x563e5f747920 0x563e5f766130 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:46.619 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.609+0000 7f059a243000 1 --2- 172.21.15.164:0/67939041 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563e5f714960 0x563e5f767fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:46.619 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.609+0000 7f059a243000 1 -- 172.21.15.164:0/67939041 >> 172.21.15.164:0/67939041 conn(0x563e5f71bc50 msgr2=0x563e5f71e980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:46.619 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.609+0000 7f056f7fe640 0 client.5118 ms_handle_reset on v2:172.21.15.164:6834/3261092607 2024-03-23T14:57:46.619 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.609+0000 7f059a243000 1 -- 172.21.15.164:0/67939041 shutdown_connections 2024-03-23T14:57:46.619 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.609+0000 7f059a243000 1 -- 172.21.15.164:0/67939041 wait complete. 2024-03-23T14:57:46.619 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.609+0000 7f059a243000 20 client.5118 trim_cache size 0 max 16384 2024-03-23T14:57:46.620 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-03-23T14:57:46.620 INFO:teuthology.orchestra.run.smithi164.stdout: "cluster_fsid": "45c7911b-b69d-4e68-907b-fbfeb780aa3b", 2024-03-23T14:57:46.620 INFO:teuthology.orchestra.run.smithi164.stdout: "whoami": 0, 2024-03-23T14:57:46.620 INFO:teuthology.orchestra.run.smithi164.stdout: "id": 4935, 2024-03-23T14:57:46.620 INFO:teuthology.orchestra.run.smithi164.stdout: "want_state": "up:active", 2024-03-23T14:57:46.620 INFO:teuthology.orchestra.run.smithi164.stdout: "state": "up:active", 2024-03-23T14:57:46.620 INFO:teuthology.orchestra.run.smithi164.stdout: "fs_name": "cephfs", 2024-03-23T14:57:46.620 INFO:teuthology.orchestra.run.smithi164.stdout: "rank_uptime": 5.0021861970000003, 2024-03-23T14:57:46.620 INFO:teuthology.orchestra.run.smithi164.stdout: "mdsmap_epoch": 13, 2024-03-23T14:57:46.620 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_epoch": 29, 2024-03-23T14:57:46.620 INFO:teuthology.orchestra.run.smithi164.stdout: "osdmap_epoch_barrier": 29, 2024-03-23T14:57:46.620 INFO:teuthology.orchestra.run.smithi164.stdout: "uptime": 11.922814434999999 2024-03-23T14:57:46.621 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-03-23T14:57:46.621 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.609+0000 7f05970ce640 1 -- 172.21.15.164:0/779999314 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f05740672a0 msgr2=0x7f0574069760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:46.621 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.609+0000 7f05970ce640 1 --2- 172.21.15.164:0/779999314 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f05740672a0 0x7f0574069760 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f057c006fd0 tx=0x7f057c008040 comp rx=0 tx=0).stop 2024-03-23T14:57:46.621 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.609+0000 7f05970ce640 1 -- 172.21.15.164:0/779999314 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f05900adc70 msgr2=0x7f059013f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:46.621 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.609+0000 7f05970ce640 1 --2- 172.21.15.164:0/779999314 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f05900adc70 0x7f059013f580 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f05840ad040 tx=0x7f058401ac80 comp rx=0 tx=0).stop 2024-03-23T14:57:46.621 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.609+0000 7f05970ce640 1 -- 172.21.15.164:0/779999314 shutdown_connections 2024-03-23T14:57:46.621 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.609+0000 7f05970ce640 1 --2- 172.21.15.164:0/779999314 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0590144b20 0x7f05901e0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:46.621 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.609+0000 7f05970ce640 1 --2- 172.21.15.164:0/779999314 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f05740672a0 0x7f0574069760 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:46.622 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.609+0000 7f05970ce640 1 --2- 172.21.15.164:0/779999314 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f05900b9ac0 0x7f05901dfdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:46.622 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.609+0000 7f05970ce640 1 --2- 172.21.15.164:0/779999314 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f05900adc70 0x7f059013f580 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:46.622 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.609+0000 7f05970ce640 1 -- 172.21.15.164:0/779999314 >> 172.21.15.164:0/779999314 conn(0x7f059006b930 msgr2=0x7f0590145440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:46.622 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.609+0000 7f05970ce640 1 -- 172.21.15.164:0/779999314 shutdown_connections 2024-03-23T14:57:46.622 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.609+0000 7f05970ce640 1 -- 172.21.15.164:0/779999314 wait complete. 2024-03-23T14:57:46.637 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-23T14:57:46.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.737+0000 7f909ff51640 1 Processor -- start 2024-03-23T14:57:46.743 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.737+0000 7f909ff51640 1 -- start start 2024-03-23T14:57:46.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.741+0000 7f909ff51640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f90980b9ac0 0x7f909813f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:46.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.741+0000 7f909ff51640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9098145240 con 0x7f909806f8e0 2024-03-23T14:57:46.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.741+0000 7f909ff51640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9098070050 con 0x7f909814db90 2024-03-23T14:57:46.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.741+0000 7f909ff51640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f90980b3b50 con 0x7f90980b9ac0 2024-03-23T14:57:46.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.741+0000 7f909d4ec640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f909806f8e0 0x7f90981459b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:46780/0 (socket says 172.21.15.164:46780) 2024-03-23T14:57:46.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.741+0000 7f909d4ec640 1 -- 172.21.15.164:0/638080845 learned_addr learned my addr 172.21.15.164:0/638080845 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-03-23T14:57:46.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.741+0000 7f909dced640 1 --2- 172.21.15.164:0/638080845 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f90980b9ac0 0x7f909813f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:46.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.741+0000 7f909cceb640 1 -- 172.21.15.164:0/638080845 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3848864527 0 0) 0x7f9098145240 con 0x7f909806f8e0 2024-03-23T14:57:46.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.741+0000 7f909cceb640 1 -- 172.21.15.164:0/638080845 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9080003530 con 0x7f909806f8e0 2024-03-23T14:57:46.746 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.741+0000 7f909cceb640 1 -- 172.21.15.164:0/638080845 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 562139163 0 0) 0x7f9080003530 con 0x7f909806f8e0 2024-03-23T14:57:46.747 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.741+0000 7f909cceb640 1 -- 172.21.15.164:0/638080845 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f90980b9ac0 msgr2=0x7f909813f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:46.747 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.741+0000 7f909cceb640 1 --2- 172.21.15.164:0/638080845 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f90980b9ac0 0x7f909813f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:46.747 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.741+0000 7f909cceb640 1 -- 172.21.15.164:0/638080845 >> v1:172.21.15.173:6789/0 conn(0x7f909814db90 legacy=0x7f909814ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:46.747 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.741+0000 7f909cceb640 1 -- 172.21.15.164:0/638080845 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90980b9ea0 con 0x7f909806f8e0 2024-03-23T14:57:46.747 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.741+0000 7f909cceb640 1 -- 172.21.15.164:0/638080845 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2526390548 0 0) 0x7f908c001680 con 0x7f909806f8e0 2024-03-23T14:57:46.747 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.741+0000 7f909cceb640 1 -- 172.21.15.164:0/638080845 >> v1:172.21.15.164:6789/0 conn(0x7f909806f8e0 legacy=0x7f90981459b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:46.747 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.741+0000 7f909cceb640 1 --2- 172.21.15.164:0/638080845 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f90800039b0 0x7f9080003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:46.747 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.741+0000 7f909cceb640 1 --2- 172.21.15.164:0/638080845 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f90800047d0 0x7f9080004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:46.747 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.741+0000 7f909cceb640 1 --2- 172.21.15.164:0/638080845 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f90980b9ac0 0x7f9080005260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:46.747 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.741+0000 7f909cceb640 1 -- 172.21.15.164:0/638080845 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f908000bb20 con 0x7f90800047d0 2024-03-23T14:57:46.747 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.741+0000 7f909cceb640 1 -- 172.21.15.164:0/638080845 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f908000bc90 con 0x7f90800039b0 2024-03-23T14:57:46.747 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.741+0000 7f909cceb640 1 -- 172.21.15.164:0/638080845 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f908000be50 con 0x7f90980b9ac0 2024-03-23T14:57:46.747 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.741+0000 7f909d4ec640 1 --2- 172.21.15.164:0/638080845 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f90800047d0 0x7f9080004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:46.747 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.741+0000 7f909e4ee640 1 --2- 172.21.15.164:0/638080845 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f90980b9ac0 0x7f9080005260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:46.747 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.741+0000 7f909dced640 1 --2- 172.21.15.164:0/638080845 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f90800039b0 0x7f9080003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:46.747 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.741+0000 7f909d4ec640 1 -- 172.21.15.164:0/638080845 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f90980b9ac0 msgr2=0x7f9080005260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:46.747 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.741+0000 7f909d4ec640 1 --2- 172.21.15.164:0/638080845 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f90980b9ac0 0x7f9080005260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:46.748 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.741+0000 7f909d4ec640 1 -- 172.21.15.164:0/638080845 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f90800039b0 msgr2=0x7f9080003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:46.748 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.741+0000 7f909d4ec640 1 --2- 172.21.15.164:0/638080845 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f90800039b0 0x7f9080003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:46.748 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.741+0000 7f909d4ec640 1 -- 172.21.15.164:0/638080845 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9080003530 con 0x7f90800047d0 2024-03-23T14:57:46.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.741+0000 7f909d4ec640 1 --2- 172.21.15.164:0/638080845 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f90800047d0 0x7f9080004c50 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f908c00c240 tx=0x7f908c01cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9979eba640d6ba9a server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:46.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.741+0000 7f909cceb640 1 -- 172.21.15.164:0/638080845 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f908c005ae0 con 0x7f90800047d0 2024-03-23T14:57:46.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.741+0000 7f909cceb640 1 -- 172.21.15.164:0/638080845 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f908c005c40 con 0x7f90800047d0 2024-03-23T14:57:46.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.741+0000 7f909cceb640 1 -- 172.21.15.164:0/638080845 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f908c0a5830 con 0x7f90800047d0 2024-03-23T14:57:46.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.745+0000 7f909ff51640 1 -- 172.21.15.164:0/638080845 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f90800047d0 msgr2=0x7f9080004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:46.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.745+0000 7f909ff51640 1 --2- 172.21.15.164:0/638080845 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f90800047d0 0x7f9080004c50 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f908c00c240 tx=0x7f908c01cc60 comp rx=0 tx=0).stop 2024-03-23T14:57:46.749 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.745+0000 7f909e4ee640 1 -- 172.21.15.164:0/638080845 reap_dead start 2024-03-23T14:57:46.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.745+0000 7f909ff51640 1 -- 172.21.15.164:0/638080845 shutdown_connections 2024-03-23T14:57:46.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.745+0000 7f909ff51640 1 -- 172.21.15.164:0/638080845 >> 172.21.15.164:0/638080845 conn(0x7f909806b930 msgr2=0x7f90980b9150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:46.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.745+0000 7f909ff51640 1 -- 172.21.15.164:0/638080845 shutdown_connections 2024-03-23T14:57:46.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.745+0000 7f909ff51640 1 -- 172.21.15.164:0/638080845 wait complete. 2024-03-23T14:57:46.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.745+0000 7f909ff51640 1 Processor -- start 2024-03-23T14:57:46.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.745+0000 7f909ff51640 1 -- start start 2024-03-23T14:57:46.752 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.745+0000 7f909ff51640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f90800047d0 0x7f909813ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:46.752 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.745+0000 7f909ff51640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f90981e2b70 0x7f909814e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:46.752 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.745+0000 7f909ff51640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f909814fd60 0x7f90981501e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:46.752 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.745+0000 7f909ff51640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9098135490 con 0x7f909814fd60 2024-03-23T14:57:46.752 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.745+0000 7f909ff51640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f90981e3ae0 con 0x7f90981e2b70 2024-03-23T14:57:46.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.745+0000 7f909ff51640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f909814f3a0 con 0x7f90800047d0 2024-03-23T14:57:46.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.745+0000 7f909e4ee640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f909814fd60 0x7f90981501e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:46.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.745+0000 7f909e4ee640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f909814fd60 0x7f90981501e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:44882/0 (socket says 172.21.15.164:44882) 2024-03-23T14:57:46.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.745+0000 7f909e4ee640 1 -- 172.21.15.164:0/4067826984 learned_addr learned my addr 172.21.15.164:0/4067826984 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:46.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.745+0000 7f909d4ec640 1 --2- 172.21.15.164:0/4067826984 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f90981e2b70 0x7f909814e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:46.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.745+0000 7f909dced640 1 --2- 172.21.15.164:0/4067826984 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f90800047d0 0x7f909813ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:46.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.745+0000 7f909e4ee640 1 -- 172.21.15.164:0/4067826984 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f90800047d0 msgr2=0x7f909813ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:46.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.745+0000 7f909e4ee640 1 --2- 172.21.15.164:0/4067826984 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f90800047d0 0x7f909813ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:46.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.745+0000 7f909e4ee640 1 -- 172.21.15.164:0/4067826984 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f90981e2b70 msgr2=0x7f909814e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:46.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.745+0000 7f909e4ee640 1 --2- 172.21.15.164:0/4067826984 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f90981e2b70 0x7f909814e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:46.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.745+0000 7f909e4ee640 1 -- 172.21.15.164:0/4067826984 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90981432e0 con 0x7f909814fd60 2024-03-23T14:57:46.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.745+0000 7f909e4ee640 1 --2- 172.21.15.164:0/4067826984 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f909814fd60 0x7f90981501e0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f9098070740 tx=0x7f909000e8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:46.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.745+0000 7f90967fc640 1 -- 172.21.15.164:0/4067826984 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90900052e0 con 0x7f909814fd60 2024-03-23T14:57:46.753 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.745+0000 7f90967fc640 1 -- 172.21.15.164:0/4067826984 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9090005460 con 0x7f909814fd60 2024-03-23T14:57:46.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.745+0000 7f909ff51640 1 -- 172.21.15.164:0/4067826984 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90981e3000 con 0x7f909814fd60 2024-03-23T14:57:46.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.745+0000 7f90967fc640 1 -- 172.21.15.164:0/4067826984 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9090003840 con 0x7f909814fd60 2024-03-23T14:57:46.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.745+0000 7f909ff51640 1 -- 172.21.15.164:0/4067826984 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90981e35c0 con 0x7f909814fd60 2024-03-23T14:57:46.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.745+0000 7f909ff51640 1 -- 172.21.15.164:0/4067826984 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90980690a0 con 0x7f909814fd60 2024-03-23T14:57:46.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.749+0000 7f90967fc640 1 -- 172.21.15.164:0/4067826984 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7f90900039a0 con 0x7f909814fd60 2024-03-23T14:57:46.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.749+0000 7f90967fc640 1 --2- 172.21.15.164:0/4067826984 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f90840672d0 0x7f9084069790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:46.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.749+0000 7f90967fc640 1 -- 172.21.15.164:0/4067826984 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5709+0+0 (secure 0 0 0) 0x7f90900849f0 con 0x7f909814fd60 2024-03-23T14:57:46.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.749+0000 7f909dced640 1 --2- 172.21.15.164:0/4067826984 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f90840672d0 0x7f9084069790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:46.754 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.749+0000 7f909dced640 1 --2- 172.21.15.164:0/4067826984 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f90840672d0 0x7f9084069790 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f90880074a0 tx=0x7f908800d040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:46.760 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:46.757+0000 7f90967fc640 1 -- 172.21.15.164:0/4067826984 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7f9090051710 con 0x7f909814fd60 2024-03-23T14:57:47.029 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:47.025+0000 7f909ff51640 1 -- 172.21.15.164:0/4067826984 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f909814f5f0 con 0x7f909814fd60 2024-03-23T14:57:47.029 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:47.025+0000 7f90967fc640 1 -- 172.21.15.164:0/4067826984 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v13) ==== 53+0+73 (secure 0 0 0) 0x7f9090019020 con 0x7f909814fd60 2024-03-23T14:57:47.030 INFO:teuthology.orchestra.run.smithi164.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-23T14:57:47.032 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:47.025+0000 7f909ff51640 1 -- 172.21.15.164:0/4067826984 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f90840672d0 msgr2=0x7f9084069790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:47.032 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:47.025+0000 7f909ff51640 1 --2- 172.21.15.164:0/4067826984 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f90840672d0 0x7f9084069790 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f90880074a0 tx=0x7f908800d040 comp rx=0 tx=0).stop 2024-03-23T14:57:47.032 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:47.025+0000 7f909ff51640 1 -- 172.21.15.164:0/4067826984 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f909814fd60 msgr2=0x7f90981501e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:47.032 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:47.025+0000 7f909ff51640 1 --2- 172.21.15.164:0/4067826984 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f909814fd60 0x7f90981501e0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f9098070740 tx=0x7f909000e8a0 comp rx=0 tx=0).stop 2024-03-23T14:57:47.032 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:47.029+0000 7f909ff51640 1 -- 172.21.15.164:0/4067826984 shutdown_connections 2024-03-23T14:57:47.032 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:47.029+0000 7f909ff51640 1 --2- 172.21.15.164:0/4067826984 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f909814fd60 0x7f90981501e0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:47.032 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:47.029+0000 7f909ff51640 1 --2- 172.21.15.164:0/4067826984 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f90840672d0 0x7f9084069790 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:47.032 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:47.029+0000 7f909ff51640 1 --2- 172.21.15.164:0/4067826984 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f90981e2b70 0x7f909814e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:47.032 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:47.029+0000 7f909ff51640 1 --2- 172.21.15.164:0/4067826984 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f90800047d0 0x7f909813ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:47.033 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:47.029+0000 7f909ff51640 1 -- 172.21.15.164:0/4067826984 >> 172.21.15.164:0/4067826984 conn(0x7f909806b930 msgr2=0x7f909814d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:47.033 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:47.029+0000 7f909ff51640 1 -- 172.21.15.164:0/4067826984 shutdown_connections 2024-03-23T14:57:47.033 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:47.029+0000 7f909ff51640 1 -- 172.21.15.164:0/4067826984 wait complete. 2024-03-23T14:57:47.047 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-23T14:57:47.047 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-23T14:57:47.047 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi164.front.sepia.ceph.com 2024-03-23T14:57:47.047 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-23T14:57:47.047 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-23T14:57:47.047 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-23T14:57:47.047 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-23T14:57:47.047 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-23T14:57:47.047 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-23T14:57:47.048 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T14:57:47.048 DEBUG:teuthology.orchestra.run.smithi164:> ip addr 2024-03-23T14:57:47.053 INFO:teuthology.orchestra.run.smithi164.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-23T14:57:47.053 INFO:teuthology.orchestra.run.smithi164.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-23T14:57:47.054 INFO:teuthology.orchestra.run.smithi164.stdout: inet 127.0.0.1/8 scope host lo 2024-03-23T14:57:47.054 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-03-23T14:57:47.054 INFO:teuthology.orchestra.run.smithi164.stdout: inet6 ::1/128 scope host 2024-03-23T14:57:47.054 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-03-23T14:57:47.054 INFO:teuthology.orchestra.run.smithi164.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-23T14:57:47.054 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether 0c:c4:7a:d9:34:f4 brd ff:ff:ff:ff:ff:ff 2024-03-23T14:57:47.054 INFO:teuthology.orchestra.run.smithi164.stdout: altname enp7s0f0 2024-03-23T14:57:47.054 INFO:teuthology.orchestra.run.smithi164.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-23T14:57:47.054 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether 0c:c4:7a:d9:34:f5 brd ff:ff:ff:ff:ff:ff 2024-03-23T14:57:47.054 INFO:teuthology.orchestra.run.smithi164.stdout: altname enp7s0f1 2024-03-23T14:57:47.054 INFO:teuthology.orchestra.run.smithi164.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-23T14:57:47.054 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether 0c:c4:7a:88:73:f2 brd ff:ff:ff:ff:ff:ff 2024-03-23T14:57:47.055 INFO:teuthology.orchestra.run.smithi164.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-23T14:57:47.055 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether 0c:c4:7a:88:73:f3 brd ff:ff:ff:ff:ff:ff 2024-03-23T14:57:47.055 INFO:teuthology.orchestra.run.smithi164.stdout: inet 172.21.15.164/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-03-23T14:57:47.055 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft 42480sec preferred_lft 42480sec 2024-03-23T14:57:47.055 INFO:teuthology.orchestra.run.smithi164.stdout: inet6 fe80::ec4:7aff:fe88:73f3/64 scope link 2024-03-23T14:57:47.055 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-03-23T14:57:47.055 INFO:teuthology.orchestra.run.smithi164.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-03-23T14:57:47.055 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether 02:42:9e:85:90:15 brd ff:ff:ff:ff:ff:ff 2024-03-23T14:57:47.055 INFO:teuthology.orchestra.run.smithi164.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-03-23T14:57:47.055 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-03-23T14:57:47.055 INFO:teuthology.orchestra.run.smithi164.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-23T14:57:47.055 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-03-23T14:57:47.055 INFO:teuthology.orchestra.run.smithi164.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-23T14:57:47.055 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-03-23T14:57:47.055 INFO:teuthology.orchestra.run.smithi164.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-03-23T14:57:47.056 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-03-23T14:57:47.056 INFO:teuthology.orchestra.run.smithi164.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-23T14:57:47.056 INFO:teuthology.orchestra.run.smithi164.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-23T14:57:47.056 INFO:teuthology.orchestra.run.smithi164.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-03-23T14:57:47.056 INFO:teuthology.orchestra.run.smithi164.stdout: valid_lft forever preferred_lft forever 2024-03-23T14:57:47.056 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T14:57:47.056 DEBUG:teuthology.orchestra.run.smithi164:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-23T14:57:47.105 INFO:teuthology.orchestra.run.smithi164.stdout:1 2024-03-23T14:57:47.106 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T14:57:47.107 DEBUG:teuthology.orchestra.run.smithi164:> route 2024-03-23T14:57:47.154 INFO:teuthology.orchestra.run.smithi164.stdout:Kernel IP routing table 2024-03-23T14:57:47.154 INFO:teuthology.orchestra.run.smithi164.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-23T14:57:47.154 INFO:teuthology.orchestra.run.smithi164.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-03-23T14:57:47.154 INFO:teuthology.orchestra.run.smithi164.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-23T14:57:47.154 INFO:teuthology.orchestra.run.smithi164.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-03-23T14:57:47.154 INFO:teuthology.orchestra.run.smithi164.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-03-23T14:57:47.154 INFO:teuthology.orchestra.run.smithi164.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-23T14:57:47.155 INFO:teuthology.orchestra.run.smithi164.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-23T14:57:47.155 INFO:teuthology.orchestra.run.smithi164.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-23T14:57:47.155 INFO:teuthology.orchestra.run.smithi164.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-23T14:57:47.155 INFO:teuthology.orchestra.run.smithi164.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-23T14:57:47.155 DEBUG:teuthology.orchestra.run.smithi164:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-23T14:57:47.155 DEBUG:teuthology.orchestra.run.smithi164:> set -e 2024-03-23T14:57:47.155 DEBUG:teuthology.orchestra.run.smithi164:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-23T14:57:47.155 DEBUG:teuthology.orchestra.run.smithi164:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-23T14:57:47.155 DEBUG:teuthology.orchestra.run.smithi164:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-23T14:57:47.156 DEBUG:teuthology.orchestra.run.smithi164:> ') 2024-03-23T14:57:47.237 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:47 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-23T14:57:47.259 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-23T14:57:47.266 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T14:57:47.266 DEBUG:teuthology.orchestra.run.smithi164:> ip netns list 2024-03-23T14:57:47.316 INFO:teuthology.orchestra.run.smithi164.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-23T14:57:47.316 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-23T14:57:47.316 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T14:57:47.316 DEBUG:teuthology.orchestra.run.smithi164:> sudo ip link set brx.0 up 2024-03-23T14:57:47.375 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-03-23T14:57:47.375 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-23T14:57:47.375 DEBUG:teuthology.orchestra.run.smithi164:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-23T14:57:47.420 INFO:teuthology.orchestra.run.smithi164.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-23T14:57:47.420 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-23T14:57:47.420 DEBUG:teuthology.orchestra.run.smithi164:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-23T14:57:47.467 DEBUG:teuthology.orchestra.run.smithi164:> sudo modprobe fuse 2024-03-23T14:57:47.524 DEBUG:teuthology.orchestra.run.smithi164:> cat /proc/self/mounts | awk '{print $2}' 2024-03-23T14:57:47.572 INFO:teuthology.orchestra.run.smithi164.stdout:/sys 2024-03-23T14:57:47.572 INFO:teuthology.orchestra.run.smithi164.stdout:/proc 2024-03-23T14:57:47.572 INFO:teuthology.orchestra.run.smithi164.stdout:/dev 2024-03-23T14:57:47.573 INFO:teuthology.orchestra.run.smithi164.stdout:/dev/pts 2024-03-23T14:57:47.573 INFO:teuthology.orchestra.run.smithi164.stdout:/run 2024-03-23T14:57:47.573 INFO:teuthology.orchestra.run.smithi164.stdout:/ 2024-03-23T14:57:47.573 INFO:teuthology.orchestra.run.smithi164.stdout:/sys/kernel/security 2024-03-23T14:57:47.573 INFO:teuthology.orchestra.run.smithi164.stdout:/dev/shm 2024-03-23T14:57:47.573 INFO:teuthology.orchestra.run.smithi164.stdout:/run/lock 2024-03-23T14:57:47.573 INFO:teuthology.orchestra.run.smithi164.stdout:/sys/fs/cgroup 2024-03-23T14:57:47.573 INFO:teuthology.orchestra.run.smithi164.stdout:/sys/fs/pstore 2024-03-23T14:57:47.573 INFO:teuthology.orchestra.run.smithi164.stdout:/sys/fs/bpf 2024-03-23T14:57:47.573 INFO:teuthology.orchestra.run.smithi164.stdout:/proc/sys/fs/binfmt_misc 2024-03-23T14:57:47.573 INFO:teuthology.orchestra.run.smithi164.stdout:/dev/hugepages 2024-03-23T14:57:47.573 INFO:teuthology.orchestra.run.smithi164.stdout:/dev/mqueue 2024-03-23T14:57:47.573 INFO:teuthology.orchestra.run.smithi164.stdout:/sys/kernel/debug 2024-03-23T14:57:47.573 INFO:teuthology.orchestra.run.smithi164.stdout:/sys/kernel/tracing 2024-03-23T14:57:47.573 INFO:teuthology.orchestra.run.smithi164.stdout:/proc/fs/nfsd 2024-03-23T14:57:47.573 INFO:teuthology.orchestra.run.smithi164.stdout:/sys/fs/fuse/connections 2024-03-23T14:57:47.574 INFO:teuthology.orchestra.run.smithi164.stdout:/sys/kernel/config 2024-03-23T14:57:47.574 INFO:teuthology.orchestra.run.smithi164.stdout:/run/credentials/systemd-sysusers.service 2024-03-23T14:57:47.574 INFO:teuthology.orchestra.run.smithi164.stdout:/run/rpc_pipefs 2024-03-23T14:57:47.574 INFO:teuthology.orchestra.run.smithi164.stdout:/run/user/1000 2024-03-23T14:57:47.574 INFO:teuthology.orchestra.run.smithi164.stdout:/var/lib/ceph 2024-03-23T14:57:47.574 INFO:teuthology.orchestra.run.smithi164.stdout:/var/lib/ceph/osd/ceph-0 2024-03-23T14:57:47.574 INFO:teuthology.orchestra.run.smithi164.stdout:/var/lib/ceph/osd/ceph-1 2024-03-23T14:57:47.574 INFO:teuthology.orchestra.run.smithi164.stdout:/var/lib/ceph/osd/ceph-2 2024-03-23T14:57:47.574 INFO:teuthology.orchestra.run.smithi164.stdout:/var/lib/ceph/osd/ceph-3 2024-03-23T14:57:47.574 INFO:teuthology.orchestra.run.smithi164.stdout:/run/netns 2024-03-23T14:57:47.574 INFO:teuthology.orchestra.run.smithi164.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-23T14:57:47.574 INFO:teuthology.orchestra.run.smithi164.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-23T14:57:47.575 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T14:57:47.575 DEBUG:teuthology.orchestra.run.smithi164:> ls /sys/fs/fuse/connections 2024-03-23T14:57:47.629 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-23T14:57:47.629 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-03-23T14:57:47.677 DEBUG:teuthology.orchestra.run.smithi164:> sudo modprobe fuse 2024-03-23T14:57:47.689 DEBUG:teuthology.orchestra.run.smithi164:> cat /proc/self/mounts | awk '{print $2}' 2024-03-23T14:57:47.737 INFO:teuthology.orchestra.run.smithi164.stdout:/sys 2024-03-23T14:57:47.737 INFO:teuthology.orchestra.run.smithi164.stdout:/proc 2024-03-23T14:57:47.737 INFO:teuthology.orchestra.run.smithi164.stdout:/dev 2024-03-23T14:57:47.737 INFO:teuthology.orchestra.run.smithi164.stdout:/dev/pts 2024-03-23T14:57:47.737 INFO:teuthology.orchestra.run.smithi164.stdout:/run 2024-03-23T14:57:47.737 INFO:teuthology.orchestra.run.smithi164.stdout:/ 2024-03-23T14:57:47.737 INFO:teuthology.orchestra.run.smithi164.stdout:/sys/kernel/security 2024-03-23T14:57:47.738 INFO:teuthology.orchestra.run.smithi164.stdout:/dev/shm 2024-03-23T14:57:47.738 INFO:teuthology.orchestra.run.smithi164.stdout:/run/lock 2024-03-23T14:57:47.738 INFO:teuthology.orchestra.run.smithi164.stdout:/sys/fs/cgroup 2024-03-23T14:57:47.738 INFO:teuthology.orchestra.run.smithi164.stdout:/sys/fs/pstore 2024-03-23T14:57:47.738 INFO:teuthology.orchestra.run.smithi164.stdout:/sys/fs/bpf 2024-03-23T14:57:47.738 INFO:teuthology.orchestra.run.smithi164.stdout:/proc/sys/fs/binfmt_misc 2024-03-23T14:57:47.738 INFO:teuthology.orchestra.run.smithi164.stdout:/dev/hugepages 2024-03-23T14:57:47.738 INFO:teuthology.orchestra.run.smithi164.stdout:/dev/mqueue 2024-03-23T14:57:47.738 INFO:teuthology.orchestra.run.smithi164.stdout:/sys/kernel/debug 2024-03-23T14:57:47.738 INFO:teuthology.orchestra.run.smithi164.stdout:/sys/kernel/tracing 2024-03-23T14:57:47.738 INFO:teuthology.orchestra.run.smithi164.stdout:/proc/fs/nfsd 2024-03-23T14:57:47.738 INFO:teuthology.orchestra.run.smithi164.stdout:/sys/fs/fuse/connections 2024-03-23T14:57:47.738 INFO:teuthology.orchestra.run.smithi164.stdout:/sys/kernel/config 2024-03-23T14:57:47.738 INFO:teuthology.orchestra.run.smithi164.stdout:/run/credentials/systemd-sysusers.service 2024-03-23T14:57:47.738 INFO:teuthology.orchestra.run.smithi164.stdout:/run/rpc_pipefs 2024-03-23T14:57:47.739 INFO:teuthology.orchestra.run.smithi164.stdout:/run/user/1000 2024-03-23T14:57:47.739 INFO:teuthology.orchestra.run.smithi164.stdout:/var/lib/ceph 2024-03-23T14:57:47.739 INFO:teuthology.orchestra.run.smithi164.stdout:/var/lib/ceph/osd/ceph-0 2024-03-23T14:57:47.739 INFO:teuthology.orchestra.run.smithi164.stdout:/var/lib/ceph/osd/ceph-1 2024-03-23T14:57:47.739 INFO:teuthology.orchestra.run.smithi164.stdout:/var/lib/ceph/osd/ceph-2 2024-03-23T14:57:47.739 INFO:teuthology.orchestra.run.smithi164.stdout:/var/lib/ceph/osd/ceph-3 2024-03-23T14:57:47.739 INFO:teuthology.orchestra.run.smithi164.stdout:/run/netns 2024-03-23T14:57:47.739 INFO:teuthology.orchestra.run.smithi164.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-23T14:57:47.739 INFO:teuthology.orchestra.run.smithi164.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-23T14:57:47.739 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T14:57:47.739 DEBUG:teuthology.orchestra.run.smithi164:> ls /sys/fs/fuse/connections 2024-03-23T14:57:47.747 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi164.stderr:2024-03-23T14:57:47.741+0000 7f4c713a1580 -1 init, newargv = 0x55b136595260 newargc=15 2024-03-23T14:57:47.747 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi164.stderr:ceph-fuse[37029]: starting ceph client 2024-03-23T14:57:47.752 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi164.stderr:ceph-fuse[37029]: starting fuse 2024-03-23T14:57:48.853 DEBUG:teuthology.orchestra.run.smithi164:> sudo modprobe fuse 2024-03-23T14:57:48.868 DEBUG:teuthology.orchestra.run.smithi164:> cat /proc/self/mounts | awk '{print $2}' 2024-03-23T14:57:48.917 INFO:teuthology.orchestra.run.smithi164.stdout:/sys 2024-03-23T14:57:48.917 INFO:teuthology.orchestra.run.smithi164.stdout:/proc 2024-03-23T14:57:48.917 INFO:teuthology.orchestra.run.smithi164.stdout:/dev 2024-03-23T14:57:48.917 INFO:teuthology.orchestra.run.smithi164.stdout:/dev/pts 2024-03-23T14:57:48.917 INFO:teuthology.orchestra.run.smithi164.stdout:/run 2024-03-23T14:57:48.917 INFO:teuthology.orchestra.run.smithi164.stdout:/ 2024-03-23T14:57:48.918 INFO:teuthology.orchestra.run.smithi164.stdout:/sys/kernel/security 2024-03-23T14:57:48.918 INFO:teuthology.orchestra.run.smithi164.stdout:/dev/shm 2024-03-23T14:57:48.918 INFO:teuthology.orchestra.run.smithi164.stdout:/run/lock 2024-03-23T14:57:48.918 INFO:teuthology.orchestra.run.smithi164.stdout:/sys/fs/cgroup 2024-03-23T14:57:48.918 INFO:teuthology.orchestra.run.smithi164.stdout:/sys/fs/pstore 2024-03-23T14:57:48.918 INFO:teuthology.orchestra.run.smithi164.stdout:/sys/fs/bpf 2024-03-23T14:57:48.918 INFO:teuthology.orchestra.run.smithi164.stdout:/proc/sys/fs/binfmt_misc 2024-03-23T14:57:48.918 INFO:teuthology.orchestra.run.smithi164.stdout:/dev/hugepages 2024-03-23T14:57:48.918 INFO:teuthology.orchestra.run.smithi164.stdout:/dev/mqueue 2024-03-23T14:57:48.918 INFO:teuthology.orchestra.run.smithi164.stdout:/sys/kernel/debug 2024-03-23T14:57:48.918 INFO:teuthology.orchestra.run.smithi164.stdout:/sys/kernel/tracing 2024-03-23T14:57:48.918 INFO:teuthology.orchestra.run.smithi164.stdout:/proc/fs/nfsd 2024-03-23T14:57:48.918 INFO:teuthology.orchestra.run.smithi164.stdout:/sys/fs/fuse/connections 2024-03-23T14:57:48.919 INFO:teuthology.orchestra.run.smithi164.stdout:/sys/kernel/config 2024-03-23T14:57:48.919 INFO:teuthology.orchestra.run.smithi164.stdout:/run/credentials/systemd-sysusers.service 2024-03-23T14:57:48.919 INFO:teuthology.orchestra.run.smithi164.stdout:/run/rpc_pipefs 2024-03-23T14:57:48.919 INFO:teuthology.orchestra.run.smithi164.stdout:/run/user/1000 2024-03-23T14:57:48.919 INFO:teuthology.orchestra.run.smithi164.stdout:/var/lib/ceph 2024-03-23T14:57:48.919 INFO:teuthology.orchestra.run.smithi164.stdout:/var/lib/ceph/osd/ceph-0 2024-03-23T14:57:48.919 INFO:teuthology.orchestra.run.smithi164.stdout:/var/lib/ceph/osd/ceph-1 2024-03-23T14:57:48.919 INFO:teuthology.orchestra.run.smithi164.stdout:/var/lib/ceph/osd/ceph-2 2024-03-23T14:57:48.919 INFO:teuthology.orchestra.run.smithi164.stdout:/var/lib/ceph/osd/ceph-3 2024-03-23T14:57:48.919 INFO:teuthology.orchestra.run.smithi164.stdout:/run/netns 2024-03-23T14:57:48.919 INFO:teuthology.orchestra.run.smithi164.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-23T14:57:48.919 INFO:teuthology.orchestra.run.smithi164.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-23T14:57:48.919 INFO:teuthology.orchestra.run.smithi164.stdout:/home/ubuntu/cephtest/mnt.0 2024-03-23T14:57:48.920 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T14:57:48.920 DEBUG:teuthology.orchestra.run.smithi164:> ls /sys/fs/fuse/connections 2024-03-23T14:57:48.968 INFO:teuthology.orchestra.run.smithi164.stdout:45 2024-03-23T14:57:48.968 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-03-23T14:57:48.968 DEBUG:teuthology.orchestra.run.smithi164:> sudo stdin-killer -- python3 -c ' 2024-03-23T14:57:48.968 DEBUG:teuthology.orchestra.run.smithi164:> import glob 2024-03-23T14:57:48.969 DEBUG:teuthology.orchestra.run.smithi164:> import re 2024-03-23T14:57:48.969 DEBUG:teuthology.orchestra.run.smithi164:> import os 2024-03-23T14:57:48.969 DEBUG:teuthology.orchestra.run.smithi164:> import subprocess 2024-03-23T14:57:48.969 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-23T14:57:48.969 DEBUG:teuthology.orchestra.run.smithi164:> def _find_admin_socket(client_name): 2024-03-23T14:57:48.969 DEBUG:teuthology.orchestra.run.smithi164:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-03-23T14:57:48.969 DEBUG:teuthology.orchestra.run.smithi164:> files = glob.glob(asok_path) 2024-03-23T14:57:48.969 DEBUG:teuthology.orchestra.run.smithi164:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-03-23T14:57:48.969 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-23T14:57:48.969 DEBUG:teuthology.orchestra.run.smithi164:> # Given a non-glob path, it better be there 2024-03-23T14:57:48.969 DEBUG:teuthology.orchestra.run.smithi164:> if "*" not in asok_path: 2024-03-23T14:57:48.969 DEBUG:teuthology.orchestra.run.smithi164:> assert(len(files) == 1) 2024-03-23T14:57:48.969 DEBUG:teuthology.orchestra.run.smithi164:> return files[0] 2024-03-23T14:57:48.969 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-23T14:57:48.969 DEBUG:teuthology.orchestra.run.smithi164:> for f in files: 2024-03-23T14:57:48.969 DEBUG:teuthology.orchestra.run.smithi164:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-23T14:57:48.969 DEBUG:teuthology.orchestra.run.smithi164:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-23T14:57:48.970 DEBUG:teuthology.orchestra.run.smithi164:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-23T14:57:48.970 DEBUG:teuthology.orchestra.run.smithi164:> contents = proc_f.read() 2024-03-23T14:57:48.970 DEBUG:teuthology.orchestra.run.smithi164:> if mountpoint in contents: 2024-03-23T14:57:48.970 DEBUG:teuthology.orchestra.run.smithi164:> return f 2024-03-23T14:57:48.970 DEBUG:teuthology.orchestra.run.smithi164:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-23T14:57:48.970 DEBUG:teuthology.orchestra.run.smithi164:> 2024-03-23T14:57:48.970 DEBUG:teuthology.orchestra.run.smithi164:> print(_find_admin_socket("client.0")) 2024-03-23T14:57:48.970 DEBUG:teuthology.orchestra.run.smithi164:> ' 2024-03-23T14:57:49.089 INFO:teuthology.orchestra.run.smithi164.stdout:/var/run/ceph/ceph-client.0.37029.asok 2024-03-23T14:57:49.093 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-23T14:57:49.100 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.37029.asok 2024-03-23T14:57:49.100 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T14:57:49.100 DEBUG:teuthology.orchestra.run.smithi164:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.37029.asok status 2024-03-23T14:57:49.280 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-03-23T14:57:49.280 INFO:teuthology.orchestra.run.smithi164.stdout: "metadata": { 2024-03-23T14:57:49.280 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_sha1": "21b3bd672163092b4e542f772797fcba02d3d4de", 2024-03-23T14:57:49.280 INFO:teuthology.orchestra.run.smithi164.stdout: "ceph_version": "ceph version 19.0.0-1855-g21b3bd67 (21b3bd672163092b4e542f772797fcba02d3d4de) squid (dev)", 2024-03-23T14:57:49.280 INFO:teuthology.orchestra.run.smithi164.stdout: "entity_id": "0", 2024-03-23T14:57:49.281 INFO:teuthology.orchestra.run.smithi164.stdout: "hostname": "smithi164", 2024-03-23T14:57:49.281 INFO:teuthology.orchestra.run.smithi164.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-03-23T14:57:49.281 INFO:teuthology.orchestra.run.smithi164.stdout: "pid": "37029", 2024-03-23T14:57:49.281 INFO:teuthology.orchestra.run.smithi164.stdout: "root": "/" 2024-03-23T14:57:49.281 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-03-23T14:57:49.281 INFO:teuthology.orchestra.run.smithi164.stdout: "dentry_count": 0, 2024-03-23T14:57:49.281 INFO:teuthology.orchestra.run.smithi164.stdout: "dentry_pinned_count": 0, 2024-03-23T14:57:49.281 INFO:teuthology.orchestra.run.smithi164.stdout: "id": 5133, 2024-03-23T14:57:49.281 INFO:teuthology.orchestra.run.smithi164.stdout: "inst": { 2024-03-23T14:57:49.281 INFO:teuthology.orchestra.run.smithi164.stdout: "name": { 2024-03-23T14:57:49.281 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "client", 2024-03-23T14:57:49.281 INFO:teuthology.orchestra.run.smithi164.stdout: "num": 5133 2024-03-23T14:57:49.281 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-03-23T14:57:49.281 INFO:teuthology.orchestra.run.smithi164.stdout: "addr": { 2024-03-23T14:57:49.281 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "v1", 2024-03-23T14:57:49.281 INFO:teuthology.orchestra.run.smithi164.stdout: "addr": "192.168.0.1:0", 2024-03-23T14:57:49.282 INFO:teuthology.orchestra.run.smithi164.stdout: "nonce": 2439772464 2024-03-23T14:57:49.282 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-03-23T14:57:49.282 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-03-23T14:57:49.282 INFO:teuthology.orchestra.run.smithi164.stdout: "addr": { 2024-03-23T14:57:49.282 INFO:teuthology.orchestra.run.smithi164.stdout: "type": "v1", 2024-03-23T14:57:49.282 INFO:teuthology.orchestra.run.smithi164.stdout: "addr": "192.168.0.1:0", 2024-03-23T14:57:49.282 INFO:teuthology.orchestra.run.smithi164.stdout: "nonce": 2439772464 2024-03-23T14:57:49.282 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-03-23T14:57:49.282 INFO:teuthology.orchestra.run.smithi164.stdout: "inst_str": "client.5133 192.168.0.1:0/2439772464", 2024-03-23T14:57:49.282 INFO:teuthology.orchestra.run.smithi164.stdout: "addr_str": "192.168.0.1:0/2439772464", 2024-03-23T14:57:49.282 INFO:teuthology.orchestra.run.smithi164.stdout: "inode_count": 1, 2024-03-23T14:57:49.282 INFO:teuthology.orchestra.run.smithi164.stdout: "mds_epoch": 13, 2024-03-23T14:57:49.282 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_epoch": 29, 2024-03-23T14:57:49.282 INFO:teuthology.orchestra.run.smithi164.stdout: "osd_epoch_barrier": 0, 2024-03-23T14:57:49.283 INFO:teuthology.orchestra.run.smithi164.stdout: "blocklisted": false, 2024-03-23T14:57:49.283 INFO:teuthology.orchestra.run.smithi164.stdout: "fs_name": "cephfs" 2024-03-23T14:57:49.283 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-03-23T14:57:49.294 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T14:57:49.294 DEBUG:teuthology.orchestra.run.smithi164:> stat --file-system '--printf=%T 2024-03-23T14:57:49.294 DEBUG:teuthology.orchestra.run.smithi164:> ' -- /home/ubuntu/cephtest/mnt.0 2024-03-23T14:57:49.300 INFO:teuthology.orchestra.run.smithi164.stdout:fuseblk 2024-03-23T14:57:49.300 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-03-23T14:57:49.300 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T14:57:49.300 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-23T14:57:49.355 DEBUG:teuthology.orchestra.run.smithi164:> mktemp 2024-03-23T14:57:49.399 INFO:teuthology.orchestra.run.smithi164.stdout:/tmp/tmp.5xlIB6ppRK 2024-03-23T14:57:49.400 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-23T14:57:49.400 DEBUG:teuthology.orchestra.run.smithi164:> dd of=/tmp/tmp.5xlIB6ppRK 2024-03-23T14:57:49.449 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'mkdir -p dir1/dir2/dir3') 2024-03-23T14:57:49.534 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:49 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-23T14:57:49.555 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-23T14:57:49.620 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'chmod 755 /tmp/test-cephfsai49c3po') 2024-03-23T14:57:49.670 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:49 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-23T14:57:49.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-23T14:57:49.680 INFO:tasks.cephfs.test_cephfs_shell:Running script "/tmp/test-cephfsai49c3po" 2024-03-23T14:57:49.680 DEBUG:teuthology.orchestra.run.smithi164:> cephfs-shell -b /tmp/test-cephfsai49c3po 2024-03-23T14:57:50.014 INFO:teuthology.orchestra.run.smithi164.stderr:Unrecognized command 2024-03-23T14:57:50.043 DEBUG:teuthology.orchestra.run:got remote process result: 127 2024-03-23T14:57:50.044 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-23T14:57:50.044 DEBUG:teuthology.orchestra.run.smithi164:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-23T14:57:50.085 DEBUG:teuthology.orchestra.run.smithi173:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-23T14:57:50.127 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-23T14:57:50.127 INFO:tasks.cephfs.mount:Terminating background process 2024-03-23T14:57:50.128 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-23T14:57:50.128 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T14:57:50.137 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T14:57:50.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.293+0000 7fa9bac67640 1 Processor -- start 2024-03-23T14:57:50.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.297+0000 7fa9bac67640 1 -- start start 2024-03-23T14:57:50.302 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.297+0000 7fa9bac67640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fa9b402c6e0 0x7fa9b4153330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:50.302 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.297+0000 7fa9bac67640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9b4141a90 con 0x7fa9b402c2a0 2024-03-23T14:57:50.302 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.297+0000 7fa9bac67640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9b40b9470 con 0x7fa9b406e8e0 2024-03-23T14:57:50.302 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.297+0000 7fa9bac67640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 -- 0x7fa9b4072af0 con 0x7fa9b402c6e0 2024-03-23T14:57:50.302 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.297+0000 7fa9b3fff640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fa9b402c2a0 0x7fa9b414fc00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:46796/0 (socket says 172.21.15.164:46796) 2024-03-23T14:57:50.302 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.297+0000 7fa9b3fff640 1 -- 172.21.15.164:0/3952456685 learned_addr learned my addr 172.21.15.164:0/3952456685 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-03-23T14:57:50.302 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.297+0000 7fa9b37fe640 1 -- 172.21.15.164:0/3952456685 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1940909267 0 0) 0x7fa9b4141a90 con 0x7fa9b402c2a0 2024-03-23T14:57:50.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.297+0000 7fa9b37fe640 1 -- 172.21.15.164:0/3952456685 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa9a4003530 con 0x7fa9b402c2a0 2024-03-23T14:57:50.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.297+0000 7fa9b9204640 1 --2- 172.21.15.164:0/3952456685 >> v2:172.21.15.173:3300/0 conn(0x7fa9b402c6e0 0x7fa9b4153330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:50.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.297+0000 7fa9b37fe640 1 -- 172.21.15.164:0/3952456685 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2460589132 0 0) 0x7fa9a4003530 con 0x7fa9b402c2a0 2024-03-23T14:57:50.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.297+0000 7fa9b37fe640 1 -- 172.21.15.164:0/3952456685 >> v2:172.21.15.173:3300/0 conn(0x7fa9b402c6e0 msgr2=0x7fa9b4153330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:50.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.297+0000 7fa9b37fe640 1 --2- 172.21.15.164:0/3952456685 >> v2:172.21.15.173:3300/0 conn(0x7fa9b402c6e0 0x7fa9b4153330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:50.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.297+0000 7fa9b37fe640 1 -- 172.21.15.164:0/3952456685 >> v1:172.21.15.173:6789/0 conn(0x7fa9b406e8e0 legacy=0x7fa9b4141380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:50.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.297+0000 7fa9b37fe640 1 -- 172.21.15.164:0/3952456685 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9b402cb30 con 0x7fa9b402c2a0 2024-03-23T14:57:50.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.297+0000 7fa9b37fe640 1 -- 172.21.15.164:0/3952456685 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2526390548 0 0) 0x7fa9a0001680 con 0x7fa9b402c2a0 2024-03-23T14:57:50.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.297+0000 7fa9b37fe640 1 -- 172.21.15.164:0/3952456685 >> v1:172.21.15.164:6789/0 conn(0x7fa9b402c2a0 legacy=0x7fa9b414fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:50.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.297+0000 7fa9b37fe640 1 --2- 172.21.15.164:0/3952456685 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa9a4003980 0x7fa9a4003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:50.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.297+0000 7fa9b37fe640 1 --2- 172.21.15.164:0/3952456685 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa9a40047d0 0x7fa9a4004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:50.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.297+0000 7fa9b37fe640 1 --2- 172.21.15.164:0/3952456685 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa9a40051e0 0x7fa9a400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:50.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.297+0000 7fa9b37fe640 1 -- 172.21.15.164:0/3952456685 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b4072af0 con 0x7fa9a4003980 2024-03-23T14:57:50.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.297+0000 7fa9b37fe640 1 -- 172.21.15.164:0/3952456685 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa9a400c0f0 con 0x7fa9a40047d0 2024-03-23T14:57:50.304 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.297+0000 7fa9b37fe640 1 -- 172.21.15.164:0/3952456685 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fa9a400c2b0 con 0x7fa9a40051e0 2024-03-23T14:57:50.304 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.297+0000 7fa9b8a03640 1 --2- 172.21.15.164:0/3952456685 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa9a4003980 0x7fa9a4003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:50.304 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.297+0000 7fa9b3fff640 1 --2- 172.21.15.164:0/3952456685 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa9a40047d0 0x7fa9a4004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:50.304 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.297+0000 7fa9b9204640 1 --2- 172.21.15.164:0/3952456685 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa9a40051e0 0x7fa9a400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:50.304 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.297+0000 7fa9b8a03640 1 -- 172.21.15.164:0/3952456685 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa9a40051e0 msgr2=0x7fa9a400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:50.304 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.297+0000 7fa9b8a03640 1 --2- 172.21.15.164:0/3952456685 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa9a40051e0 0x7fa9a400b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:50.304 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.297+0000 7fa9b8a03640 1 -- 172.21.15.164:0/3952456685 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa9a40047d0 msgr2=0x7fa9a4004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:50.304 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.297+0000 7fa9b8a03640 1 --2- 172.21.15.164:0/3952456685 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa9a40047d0 0x7fa9a4004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:50.304 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.297+0000 7fa9b8a03640 1 -- 172.21.15.164:0/3952456685 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9a4003530 con 0x7fa9a4003980 2024-03-23T14:57:50.304 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.297+0000 7fa9b9204640 1 -- 172.21.15.164:0/3952456685 reap_dead start 2024-03-23T14:57:50.305 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.301+0000 7fa9b8a03640 1 --2- 172.21.15.164:0/3952456685 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa9a4003980 0x7fa9a4003dc0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fa9a8001780 tx=0x7fa9a801c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dde56db28b9374e server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:50.306 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.301+0000 7fa9b37fe640 1 -- 172.21.15.164:0/3952456685 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9a80052d0 con 0x7fa9a4003980 2024-03-23T14:57:50.306 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.301+0000 7fa9b37fe640 1 -- 172.21.15.164:0/3952456685 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9a80a7050 con 0x7fa9a4003980 2024-03-23T14:57:50.306 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.301+0000 7fa9b37fe640 1 -- 172.21.15.164:0/3952456685 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9a8003600 con 0x7fa9a4003980 2024-03-23T14:57:50.306 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.301+0000 7fa9bac67640 1 -- 172.21.15.164:0/3952456685 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa9a4003980 msgr2=0x7fa9a4003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:50.306 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.301+0000 7fa9bac67640 1 --2- 172.21.15.164:0/3952456685 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa9a4003980 0x7fa9a4003dc0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fa9a8001780 tx=0x7fa9a801c420 comp rx=0 tx=0).stop 2024-03-23T14:57:50.306 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.301+0000 7fa9bac67640 1 -- 172.21.15.164:0/3952456685 shutdown_connections 2024-03-23T14:57:50.306 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.301+0000 7fa9bac67640 1 --2- 172.21.15.164:0/3952456685 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa9a4003980 0x7fa9a4003dc0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:50.306 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.301+0000 7fa9bac67640 1 -- 172.21.15.164:0/3952456685 >> 172.21.15.164:0/3952456685 conn(0x7fa9b40b7970 msgr2=0x7fa9b40b7d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:50.306 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.301+0000 7fa9bac67640 1 -- 172.21.15.164:0/3952456685 shutdown_connections 2024-03-23T14:57:50.306 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.301+0000 7fa9bac67640 1 -- 172.21.15.164:0/3952456685 wait complete. 2024-03-23T14:57:50.306 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.301+0000 7fa9bac67640 1 Processor -- start 2024-03-23T14:57:50.306 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.301+0000 7fa9bac67640 1 -- start start 2024-03-23T14:57:50.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.301+0000 7fa9bac67640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa9a4003980 0x7fa9b413fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:50.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.301+0000 7fa9bac67640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa9b41435c0 0x7fa9b4143a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:50.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.301+0000 7fa9bac67640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa9b413cce0 0x7fa9b413d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:50.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.301+0000 7fa9bac67640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b4144b60 con 0x7fa9b41435c0 2024-03-23T14:57:50.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.301+0000 7fa9bac67640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b40adf50 con 0x7fa9a4003980 2024-03-23T14:57:50.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.301+0000 7fa9bac67640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fa9b40aea10 con 0x7fa9b413cce0 2024-03-23T14:57:50.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.301+0000 7fa9b3fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa9b41435c0 0x7fa9b4143a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:50.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.301+0000 7fa9b3fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa9b41435c0 0x7fa9b4143a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:44904/0 (socket says 172.21.15.164:44904) 2024-03-23T14:57:50.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.301+0000 7fa9b3fff640 1 -- 172.21.15.164:0/1951883687 learned_addr learned my addr 172.21.15.164:0/1951883687 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:50.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.301+0000 7fa9b8a03640 1 --2- 172.21.15.164:0/1951883687 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa9a4003980 0x7fa9b413fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:50.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.301+0000 7fa9b9204640 1 --2- 172.21.15.164:0/1951883687 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa9b413cce0 0x7fa9b413d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:50.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.301+0000 7fa9b3fff640 1 -- 172.21.15.164:0/1951883687 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa9b413cce0 msgr2=0x7fa9b413d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:50.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.301+0000 7fa9b3fff640 1 --2- 172.21.15.164:0/1951883687 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa9b413cce0 0x7fa9b413d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:50.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.301+0000 7fa9b3fff640 1 -- 172.21.15.164:0/1951883687 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa9a4003980 msgr2=0x7fa9b413fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:50.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.301+0000 7fa9b3fff640 1 --2- 172.21.15.164:0/1951883687 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa9a4003980 0x7fa9b413fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:50.309 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.301+0000 7fa9b3fff640 1 -- 172.21.15.164:0/1951883687 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9b413d6a0 con 0x7fa9b41435c0 2024-03-23T14:57:50.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.301+0000 7fa9b3fff640 1 --2- 172.21.15.164:0/1951883687 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa9b41435c0 0x7fa9b4143a40 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fa9a0011790 tx=0x7fa9a0011c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:50.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.301+0000 7fa9b17fa640 1 -- 172.21.15.164:0/1951883687 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9a0011ea0 con 0x7fa9b41435c0 2024-03-23T14:57:50.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.301+0000 7fa9b17fa640 1 -- 172.21.15.164:0/1951883687 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9a000fb60 con 0x7fa9b41435c0 2024-03-23T14:57:50.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.301+0000 7fa9bac67640 1 -- 172.21.15.164:0/1951883687 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9b41448c0 con 0x7fa9b41435c0 2024-03-23T14:57:50.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.301+0000 7fa9b17fa640 1 -- 172.21.15.164:0/1951883687 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9a0015410 con 0x7fa9b41435c0 2024-03-23T14:57:50.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.301+0000 7fa9bac67640 1 -- 172.21.15.164:0/1951883687 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9b4140930 con 0x7fa9b41435c0 2024-03-23T14:57:50.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.305+0000 7fa9bac67640 1 -- 172.21.15.164:0/1951883687 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9b40ad8d0 con 0x7fa9b41435c0 2024-03-23T14:57:50.310 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.305+0000 7fa9b17fa640 1 -- 172.21.15.164:0/1951883687 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7fa9a0015570 con 0x7fa9b41435c0 2024-03-23T14:57:50.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.305+0000 7fa9b17fa640 1 --2- 172.21.15.164:0/1951883687 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fa9980672a0 0x7fa998069760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:50.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.305+0000 7fa9b17fa640 1 -- 172.21.15.164:0/1951883687 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5709+0+0 (secure 0 0 0) 0x7fa9a0088b10 con 0x7fa9b41435c0 2024-03-23T14:57:50.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.305+0000 7fa9b8a03640 1 --2- 172.21.15.164:0/1951883687 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fa9980672a0 0x7fa998069760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:50.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.305+0000 7fa9b8a03640 1 --2- 172.21.15.164:0/1951883687 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fa9980672a0 0x7fa998069760 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa9a80ac000 tx=0x7fa9a80011a0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:50.313 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.309+0000 7fa9b17fa640 1 -- 172.21.15.164:0/1951883687 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7fa9a0055830 con 0x7fa9b41435c0 2024-03-23T14:57:50.472 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.469+0000 7fa9bac67640 1 -- 172.21.15.164:0/1951883687 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa9b40adc70 con 0x7fa9b41435c0 2024-03-23T14:57:50.474 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.469+0000 7fa9b17fa640 1 -- 172.21.15.164:0/1951883687 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) ==== 74+0+16693 (secure 0 0 0) 0x7fa9b40adc70 con 0x7fa9b41435c0 2024-03-23T14:57:50.474 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-23T14:57:50.474 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":29,"fsid":"45c7911b-b69d-4e68-907b-fbfeb780aa3b","created":"2024-03-23T14:56:34.367633+0000","modified":"2024-03-23T14:57:42.610096+0000","last_up_change":"2024-03-23T14:56:46.297403+0000","last_in_change":"2024-03-23T14:56:36.629884+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T14:56:47.186157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-03-23T14:57:39.290666+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-03-23T14:57:39.934337+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"29","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"80abdc62-bd0d-4e60-a554-7ed6fd67cf54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":984716690},{"type":"v1","addr":"172.21.15.164:6801","nonce":984716690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":984716690},{"type":"v1","addr":"172.21.15.164:6803","nonce":984716690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":984716690},{"type":"v1","addr":"172.21.15.164:6807","nonce":984716690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":984716690},{"type":"v1","addr":"172.21.15.164:6805","nonce":984716690}]},"public_addr":"172.21.15.164:6801/984716690","cluster_addr":"172.21.15.164:6803/984716690","heartbeat_back_addr":"172.21.15.164:6807/984716690","heartbeat_front_addr":"172.21.15.164:6805/984716690","state":["exists","up"]},{"osd":1,"uuid":"f9001b06-be4d-4f42-a0e6-bcef2afa0d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":1349657239},{"type":"v1","addr":"172.21.15.164:6817","nonce":1349657239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":1349657239},{"type":"v1","addr":"172.21.15.164:6819","nonce":1349657239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":1349657239},{"type":"v1","addr":"172.21.15.164:6823","nonce":1349657239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":1349657239},{"type":"v1","addr":"172.21.15.164:6821","nonce":1349657239}]},"public_addr":"172.21.15.164:6817/1349657239","cluster_addr":"172.21.15.164:6819/1349657239","heartbeat_back_addr":"172.21.15.164:6823/1349657239","heartbeat_front_addr":"172.21.15.164:6821/1349657239","state":["exists","up"]},{"osd":2,"uuid":"fa0aeb9f-8b54-4831-a188-3e361c7980d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":887861252},{"type":"v1","addr":"172.21.15.164:6809","nonce":887861252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":887861252},{"type":"v1","addr":"172.21.15.164:6811","nonce":887861252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":887861252},{"type":"v1","addr":"172.21.15.164:6815","nonce":887861252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":887861252},{"type":"v1","addr":"172.21.15.164:6813","nonce":887861252}]},"public_addr":"172.21.15.164:6809/887861252","cluster_addr":"172.21.15.164:6811/887861252","heartbeat_back_addr":"172.21.15.164:6815/887861252","heartbeat_front_addr":"172.21.15.164:6813/887861252","state":["exists","up"]},{"osd":3,"uuid":"7e18c957-3091-46d5-8e17-8f3354c1726c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":812540271},{"type":"v1","addr":"172.21.15.164:6825","nonce":812540271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":812540271},{"type":"v1","addr":"172.21.15.164:6827","nonce":812540271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":812540271},{"type":"v1","addr":"172.21.15.164:6831","nonce":812540271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":812540271},{"type":"v1","addr":"172.21.15.164:6829","nonce":812540271}]},"public_addr":"172.21.15.164:6825/812540271","cluster_addr":"172.21.15.164:6827/812540271","heartbeat_back_addr":"172.21.15.164:6831/812540271","heartbeat_front_addr":"172.21.15.164:6829/812540271","state":["exists","up"]},{"osd":4,"uuid":"8e83f315-42bf-4471-9ce8-d81948da0f9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":2656857421},{"type":"v1","addr":"172.21.15.173:6825","nonce":2656857421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2656857421},{"type":"v1","addr":"172.21.15.173:6827","nonce":2656857421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2656857421},{"type":"v1","addr":"172.21.15.173:6831","nonce":2656857421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":2656857421},{"type":"v1","addr":"172.21.15.173:6829","nonce":2656857421}]},"public_addr":"172.21.15.173:6825/2656857421","cluster_addr":"172.21.15.173:6827/2656857421","heartbeat_back_addr":"172.21.15.173:6831/2656857421","heartbeat_front_addr":"172.21.15.173:6829/2656857421","state":["exists","up"]},{"osd":5,"uuid":"a230c872-71f9-40ab-8999-defd0162d4ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":3676483185},{"type":"v1","addr":"172.21.15.173:6801","nonce":3676483185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":3676483185},{"type":"v1","addr":"172.21.15.173:6803","nonce":3676483185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":3676483185},{"type":"v1","addr":"172.21.15.173:6807","nonce":3676483185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":3676483185},{"type":"v1","addr":"172.21.15.173:6805","nonce":3676483185}]},"public_addr":"172.21.15.173:6801/3676483185","cluster_addr":"172.21.15.173:6803/3676483185","heartbeat_back_addr":"172.21.15.173:6807/3676483185","heartbeat_front_addr":"172.21.15.173:6805/3676483185","state":["exists","up"]},{"osd":6,"uuid":"6193bfd7-8432-4f7f-a117-a3fcee21af99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":3631684087},{"type":"v1","addr":"172.21.15.173:6809","nonce":3631684087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":3631684087},{"type":"v1","addr":"172.21.15.173:6811","nonce":3631684087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":3631684087},{"type":"v1","addr":"172.21.15.173:6815","nonce":3631684087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":3631684087},{"type":"v1","addr":"172.21.15.173:6813","nonce":3631684087}]},"public_addr":"172.21.15.173:6809/3631684087","cluster_addr":"172.21.15.173:6811/3631684087","heartbeat_back_addr":"172.21.15.173:6815/3631684087","heartbeat_front_addr":"172.21.15.173:6813/3631684087","state":["exists","up"]},{"osd":7,"uuid":"b0f1b80a-afc4-4538-83b9-f081d2dcb8cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":3261114948},{"type":"v1","addr":"172.21.15.173:6817","nonce":3261114948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":3261114948},{"type":"v1","addr":"172.21.15.173:6819","nonce":3261114948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":3261114948},{"type":"v1","addr":"172.21.15.173:6823","nonce":3261114948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":3261114948},{"type":"v1","addr":"172.21.15.173:6821","nonce":3261114948}]},"public_addr":"172.21.15.173:6817/3261114948","cluster_addr":"172.21.15.173:6819/3261114948","heartbeat_back_addr":"172.21.15.173:6823/3261114948","heartbeat_front_addr":"172.21.15.173:6821/3261114948","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.188390+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.159932+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.191101+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.181776+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.224365+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.177515+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.153631+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.139972+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T14:57:50.478 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.473+0000 7fa9bac67640 1 -- 172.21.15.164:0/1951883687 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fa9980672a0 msgr2=0x7fa998069760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:50.478 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.473+0000 7fa9bac67640 1 --2- 172.21.15.164:0/1951883687 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fa9980672a0 0x7fa998069760 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa9a80ac000 tx=0x7fa9a80011a0 comp rx=0 tx=0).stop 2024-03-23T14:57:50.478 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.473+0000 7fa9bac67640 1 -- 172.21.15.164:0/1951883687 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa9b41435c0 msgr2=0x7fa9b4143a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:50.478 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.473+0000 7fa9bac67640 1 --2- 172.21.15.164:0/1951883687 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa9b41435c0 0x7fa9b4143a40 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fa9a0011790 tx=0x7fa9a0011c50 comp rx=0 tx=0).stop 2024-03-23T14:57:50.478 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.473+0000 7fa9bac67640 1 -- 172.21.15.164:0/1951883687 shutdown_connections 2024-03-23T14:57:50.478 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.473+0000 7fa9bac67640 1 --2- 172.21.15.164:0/1951883687 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa9b413cce0 0x7fa9b413d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:50.478 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.473+0000 7fa9bac67640 1 --2- 172.21.15.164:0/1951883687 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa9b41435c0 0x7fa9b4143a40 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:50.478 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.473+0000 7fa9bac67640 1 --2- 172.21.15.164:0/1951883687 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fa9980672a0 0x7fa998069760 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:50.478 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.473+0000 7fa9bac67640 1 --2- 172.21.15.164:0/1951883687 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa9a4003980 0x7fa9b413fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:50.478 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.473+0000 7fa9bac67640 1 -- 172.21.15.164:0/1951883687 >> 172.21.15.164:0/1951883687 conn(0x7fa9b40b7970 msgr2=0x7fa9b40b7d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:50.478 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.473+0000 7fa9bac67640 1 -- 172.21.15.164:0/1951883687 shutdown_connections 2024-03-23T14:57:50.478 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.473+0000 7fa9bac67640 1 -- 172.21.15.164:0/1951883687 wait complete. 2024-03-23T14:57:50.499 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T14:56:47.186157+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-23T14:57:39.290666+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '28', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-03-23T14:57:39.934337+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '29', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-23T14:57:50.499 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-23T14:57:50.635 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.629+0000 7f0b424a1640 1 Processor -- start 2024-03-23T14:57:50.635 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.629+0000 7f0b424a1640 1 -- start start 2024-03-23T14:57:50.636 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.633+0000 7f0b424a1640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f0b3c0add50 0x7f0b3c147770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:50.636 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.633+0000 7f0b424a1640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f0b3c147cb0 0x7f0b3c148090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:50.636 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.633+0000 7f0b424a1640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b3c06d650 con 0x7f0b3c09def0 2024-03-23T14:57:50.636 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.633+0000 7f0b424a1640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f0b3c1485d0 con 0x7f0b3c0add50 2024-03-23T14:57:50.636 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.633+0000 7f0b424a1640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 -- 0x7f0b3c0af4a0 con 0x7f0b3c147cb0 2024-03-23T14:57:50.636 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.633+0000 7f0b33fff640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f0b3c0add50 0x7f0b3c147770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:50.636 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.633+0000 7f0b3bfff640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f0b3c09def0 0x7f0b3c0b45e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:46798/0 (socket says 172.21.15.164:46798) 2024-03-23T14:57:50.637 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.633+0000 7f0b3bfff640 1 -- 172.21.15.164:0/1601604894 learned_addr learned my addr 172.21.15.164:0/1601604894 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-03-23T14:57:50.637 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.633+0000 7f0b33fff640 1 -- 172.21.15.164:0/1601604894 >> v2:172.21.15.173:3300/0 conn(0x7f0b3c147cb0 msgr2=0x7f0b3c148090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:50.637 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.633+0000 7f0b33fff640 1 --2- 172.21.15.164:0/1601604894 >> v2:172.21.15.173:3300/0 conn(0x7f0b3c147cb0 0x7f0b3c148090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:50.637 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.633+0000 7f0b33fff640 1 -- 172.21.15.164:0/1601604894 >> v1:172.21.15.164:6789/0 conn(0x7f0b3c09def0 legacy=0x7f0b3c0b45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:50.637 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.633+0000 7f0b33fff640 1 -- 172.21.15.164:0/1601604894 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b3c09e2d0 con 0x7f0b3c0add50 2024-03-23T14:57:50.637 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.633+0000 7f0b3b7fe640 1 -- 172.21.15.164:0/1601604894 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1409205331 0 0) 0x7f0b3c06d650 con 0x7f0b3c09def0 2024-03-23T14:57:50.638 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.633+0000 7f0b33fff640 1 --2- 172.21.15.164:0/1601604894 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0b3c0add50 0x7f0b3c147770 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c0b54c0 tx=0x7f0b2c01c360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8a57745abe068e4 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:50.638 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.633+0000 7f0b3b7fe640 1 -- 172.21.15.164:0/1601604894 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b2c0ad660 con 0x7f0b3c0add50 2024-03-23T14:57:50.638 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.633+0000 7f0b3b7fe640 1 -- 172.21.15.164:0/1601604894 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b2c0ad7c0 con 0x7f0b3c0add50 2024-03-23T14:57:50.638 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.633+0000 7f0b3b7fe640 1 -- 172.21.15.164:0/1601604894 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b2c0adb50 con 0x7f0b3c0add50 2024-03-23T14:57:50.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.633+0000 7f0b424a1640 1 -- 172.21.15.164:0/1601604894 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0b3c0add50 msgr2=0x7f0b3c147770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:50.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.633+0000 7f0b424a1640 1 --2- 172.21.15.164:0/1601604894 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0b3c0add50 0x7f0b3c147770 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c0b54c0 tx=0x7f0b2c01c360 comp rx=0 tx=0).stop 2024-03-23T14:57:50.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.633+0000 7f0b424a1640 1 -- 172.21.15.164:0/1601604894 shutdown_connections 2024-03-23T14:57:50.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.633+0000 7f0b424a1640 1 --2- 172.21.15.164:0/1601604894 >> v2:172.21.15.173:3300/0 conn(0x7f0b3c147cb0 0x7f0b3c148090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:50.640 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.633+0000 7f0b424a1640 1 --2- 172.21.15.164:0/1601604894 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0b3c0add50 0x7f0b3c147770 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:50.640 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.633+0000 7f0b424a1640 1 -- 172.21.15.164:0/1601604894 >> 172.21.15.164:0/1601604894 conn(0x7f0b3c0b8ca0 msgr2=0x7f0b3c0b90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:50.640 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.633+0000 7f0b424a1640 1 -- 172.21.15.164:0/1601604894 shutdown_connections 2024-03-23T14:57:50.640 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.633+0000 7f0b424a1640 1 -- 172.21.15.164:0/1601604894 wait complete. 2024-03-23T14:57:50.642 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.637+0000 7f0b424a1640 1 Processor -- start 2024-03-23T14:57:50.642 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.637+0000 7f0b424a1640 1 -- start start 2024-03-23T14:57:50.642 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.637+0000 7f0b424a1640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0b3c09def0 0x7f0b3c137e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:50.642 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.637+0000 7f0b424a1640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0b3c0add50 0x7f0b3c138350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:50.642 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.637+0000 7f0b424a1640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0b3c147cb0 0x7f0b3c1e0d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:50.642 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.637+0000 7f0b424a1640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0b3c070b10 con 0x7f0b3c147cb0 2024-03-23T14:57:50.642 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.637+0000 7f0b424a1640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f0b3c070170 con 0x7f0b3c0add50 2024-03-23T14:57:50.642 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.637+0000 7f0b424a1640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f0b3c141960 con 0x7f0b3c09def0 2024-03-23T14:57:50.643 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.637+0000 7f0b40a3e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0b3c147cb0 0x7f0b3c1e0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:50.643 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.637+0000 7f0b40a3e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0b3c147cb0 0x7f0b3c1e0d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:44930/0 (socket says 172.21.15.164:44930) 2024-03-23T14:57:50.643 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.637+0000 7f0b40a3e640 1 -- 172.21.15.164:0/3063127956 learned_addr learned my addr 172.21.15.164:0/3063127956 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:50.643 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.637+0000 7f0b33fff640 1 --2- 172.21.15.164:0/3063127956 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0b3c0add50 0x7f0b3c138350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:50.643 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.637+0000 7f0b40a3e640 1 -- 172.21.15.164:0/3063127956 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0b3c09def0 msgr2=0x7f0b3c137e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:50.643 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.637+0000 7f0b3bfff640 1 --2- 172.21.15.164:0/3063127956 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0b3c09def0 0x7f0b3c137e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:50.643 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.637+0000 7f0b40a3e640 1 --2- 172.21.15.164:0/3063127956 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0b3c09def0 0x7f0b3c137e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:50.643 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.637+0000 7f0b40a3e640 1 -- 172.21.15.164:0/3063127956 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0b3c0add50 msgr2=0x7f0b3c138350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:50.643 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.637+0000 7f0b40a3e640 1 --2- 172.21.15.164:0/3063127956 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0b3c0add50 0x7f0b3c138350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:50.643 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.637+0000 7f0b40a3e640 1 -- 172.21.15.164:0/3063127956 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b2c009420 con 0x7f0b3c147cb0 2024-03-23T14:57:50.643 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.637+0000 7f0b40a3e640 1 --2- 172.21.15.164:0/3063127956 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0b3c147cb0 0x7f0b3c1e0d70 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f0b3400b970 tx=0x7f0b3400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:50.643 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.637+0000 7f0b397fa640 1 -- 172.21.15.164:0/3063127956 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b3400cb40 con 0x7f0b3c147cb0 2024-03-23T14:57:50.643 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.637+0000 7f0b397fa640 1 -- 172.21.15.164:0/3063127956 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b3400ccc0 con 0x7f0b3c147cb0 2024-03-23T14:57:50.643 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.637+0000 7f0b424a1640 1 -- 172.21.15.164:0/3063127956 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b3c06f530 con 0x7f0b3c147cb0 2024-03-23T14:57:50.644 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.637+0000 7f0b397fa640 1 -- 172.21.15.164:0/3063127956 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b34014680 con 0x7f0b3c147cb0 2024-03-23T14:57:50.644 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.637+0000 7f0b424a1640 1 -- 172.21.15.164:0/3063127956 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b3c06f250 con 0x7f0b3c147cb0 2024-03-23T14:57:50.644 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.637+0000 7f0b424a1640 1 -- 172.21.15.164:0/3063127956 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b3c0690a0 con 0x7f0b3c147cb0 2024-03-23T14:57:50.645 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.637+0000 7f0b397fa640 1 -- 172.21.15.164:0/3063127956 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7f0b3401d3b0 con 0x7f0b3c147cb0 2024-03-23T14:57:50.645 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.637+0000 7f0b397fa640 1 --2- 172.21.15.164:0/3063127956 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f0b1c0672a0 0x7f0b1c069760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:50.645 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.641+0000 7f0b3bfff640 1 --2- 172.21.15.164:0/3063127956 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f0b1c0672a0 0x7f0b1c069760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:50.645 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.641+0000 7f0b397fa640 1 -- 172.21.15.164:0/3063127956 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5709+0+0 (secure 0 0 0) 0x7f0b34084ed0 con 0x7f0b3c147cb0 2024-03-23T14:57:50.645 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.641+0000 7f0b3bfff640 1 --2- 172.21.15.164:0/3063127956 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f0b1c0672a0 0x7f0b1c069760 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f0b24000f70 tx=0x7f0b24009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:50.648 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.645+0000 7f0b397fa640 1 -- 172.21.15.164:0/3063127956 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7f0b34051b10 con 0x7f0b3c147cb0 2024-03-23T14:57:50.796 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.793+0000 7f0b424a1640 1 -- 172.21.15.164:0/3063127956 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0b3c1483b0 con 0x7f0b3c147cb0 2024-03-23T14:57:50.796 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.793+0000 7f0b397fa640 1 -- 172.21.15.164:0/3063127956 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) ==== 93+0+10 (secure 0 0 0) 0x7f0b34019020 con 0x7f0b3c147cb0 2024-03-23T14:57:50.797 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 1 2024-03-23T14:57:50.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.793+0000 7f0b424a1640 1 -- 172.21.15.164:0/3063127956 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f0b1c0672a0 msgr2=0x7f0b1c069760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:50.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.793+0000 7f0b424a1640 1 --2- 172.21.15.164:0/3063127956 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f0b1c0672a0 0x7f0b1c069760 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f0b24000f70 tx=0x7f0b24009040 comp rx=0 tx=0).stop 2024-03-23T14:57:50.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.797+0000 7f0b424a1640 1 -- 172.21.15.164:0/3063127956 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0b3c147cb0 msgr2=0x7f0b3c1e0d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:50.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.797+0000 7f0b424a1640 1 --2- 172.21.15.164:0/3063127956 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0b3c147cb0 0x7f0b3c1e0d70 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f0b3400b970 tx=0x7f0b3400be30 comp rx=0 tx=0).stop 2024-03-23T14:57:50.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.797+0000 7f0b424a1640 1 -- 172.21.15.164:0/3063127956 shutdown_connections 2024-03-23T14:57:50.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.797+0000 7f0b424a1640 1 --2- 172.21.15.164:0/3063127956 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0b3c147cb0 0x7f0b3c1e0d70 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:50.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.797+0000 7f0b424a1640 1 --2- 172.21.15.164:0/3063127956 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f0b1c0672a0 0x7f0b1c069760 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:50.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.797+0000 7f0b424a1640 1 --2- 172.21.15.164:0/3063127956 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0b3c0add50 0x7f0b3c138350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:50.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.797+0000 7f0b424a1640 1 --2- 172.21.15.164:0/3063127956 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0b3c09def0 0x7f0b3c137e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:50.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.797+0000 7f0b424a1640 1 -- 172.21.15.164:0/3063127956 >> 172.21.15.164:0/3063127956 conn(0x7f0b3c0b8ca0 msgr2=0x7f0b3c13b940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:50.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.797+0000 7f0b424a1640 1 -- 172.21.15.164:0/3063127956 shutdown_connections 2024-03-23T14:57:50.801 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.797+0000 7f0b424a1640 1 -- 172.21.15.164:0/3063127956 wait complete. 2024-03-23T14:57:50.821 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-23T14:57:50.945 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.941+0000 7f250a5c8640 1 Processor -- start 2024-03-23T14:57:50.946 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.941+0000 7f250a5c8640 1 -- start start 2024-03-23T14:57:50.947 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.941+0000 7f250a5c8640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f25040b4a00 0x7f25040b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:50.947 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.941+0000 7f250a5c8640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f2504147420 0x7f2504147850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:50.947 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.941+0000 7f250a5c8640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f250409e2d0 con 0x7f250409def0 2024-03-23T14:57:50.947 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.941+0000 7f250a5c8640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f2504147d90 con 0x7f2504147420 2024-03-23T14:57:50.947 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.941+0000 7f250a5c8640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f2504146de0 con 0x7f25040b4a00 2024-03-23T14:57:50.947 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.941+0000 7f2508b65640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f2504147420 0x7f2504147850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:50.947 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.941+0000 7f2508b65640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f2504147420 0x7f2504147850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:44936/0 (socket says 172.21.15.164:44936) 2024-03-23T14:57:50.947 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.941+0000 7f2508b65640 1 -- 172.21.15.164:0/4125564104 learned_addr learned my addr 172.21.15.164:0/4125564104 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:50.948 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.941+0000 7f25037fe640 1 --2- 172.21.15.164:0/4125564104 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f25040b4a00 0x7f25040b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:50.948 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.941+0000 7f2508b65640 1 -- 172.21.15.164:0/4125564104 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f25040b4a00 msgr2=0x7f25040b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:50.948 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.941+0000 7f2508b65640 1 --2- 172.21.15.164:0/4125564104 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f25040b4a00 0x7f25040b4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:50.948 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.941+0000 7f2508b65640 1 -- 172.21.15.164:0/4125564104 >> v1:172.21.15.173:6789/0 conn(0x7f250409def0 legacy=0x7f25040ae620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:50.948 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.941+0000 7f2508b65640 1 -- 172.21.15.164:0/4125564104 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f250406cb70 con 0x7f2504147420 2024-03-23T14:57:50.950 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.945+0000 7f2508b65640 1 --2- 172.21.15.164:0/4125564104 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2504147420 0x7f2504147850 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f24fc009e80 tx=0x7f24fc01c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c2cdd611a41413d server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:50.950 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.945+0000 7f2502ffd640 1 -- 172.21.15.164:0/4125564104 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24fc0a55a0 con 0x7f2504147420 2024-03-23T14:57:50.950 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.945+0000 7f2502ffd640 1 -- 172.21.15.164:0/4125564104 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24fc0a5700 con 0x7f2504147420 2024-03-23T14:57:50.950 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.945+0000 7f2502ffd640 1 -- 172.21.15.164:0/4125564104 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24fc0a5a90 con 0x7f2504147420 2024-03-23T14:57:50.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.945+0000 7f250a5c8640 1 -- 172.21.15.164:0/4125564104 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2504147420 msgr2=0x7f2504147850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:50.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.945+0000 7f250a5c8640 1 --2- 172.21.15.164:0/4125564104 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2504147420 0x7f2504147850 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f24fc009e80 tx=0x7f24fc01c940 comp rx=0 tx=0).stop 2024-03-23T14:57:50.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.945+0000 7f250a5c8640 1 -- 172.21.15.164:0/4125564104 shutdown_connections 2024-03-23T14:57:50.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.945+0000 7f250a5c8640 1 --2- 172.21.15.164:0/4125564104 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2504147420 0x7f2504147850 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:50.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.945+0000 7f250a5c8640 1 --2- 172.21.15.164:0/4125564104 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f25040b4a00 0x7f25040b4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:50.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.945+0000 7f250a5c8640 1 -- 172.21.15.164:0/4125564104 >> 172.21.15.164:0/4125564104 conn(0x7f25040b8ca0 msgr2=0x7f25040b90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:50.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.945+0000 7f250a5c8640 1 -- 172.21.15.164:0/4125564104 shutdown_connections 2024-03-23T14:57:50.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.945+0000 7f250a5c8640 1 -- 172.21.15.164:0/4125564104 wait complete. 2024-03-23T14:57:50.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.949+0000 7f250a5c8640 1 Processor -- start 2024-03-23T14:57:50.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.949+0000 7f250a5c8640 1 -- start start 2024-03-23T14:57:50.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.949+0000 7f250a5c8640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f250409def0 0x7f250413c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:50.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.949+0000 7f250a5c8640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f25040b4a00 0x7f25041e2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:50.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.949+0000 7f250a5c8640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2504147420 0x7f250413b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:50.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.949+0000 7f250a5c8640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2504145240 con 0x7f2504147420 2024-03-23T14:57:50.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.949+0000 7f250a5c8640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f25041406a0 con 0x7f25040b4a00 2024-03-23T14:57:50.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.949+0000 7f250a5c8640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f250413aff0 con 0x7f250409def0 2024-03-23T14:57:50.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.949+0000 7f2508b65640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2504147420 0x7f250413b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:50.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.949+0000 7f2508b65640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2504147420 0x7f250413b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:44952/0 (socket says 172.21.15.164:44952) 2024-03-23T14:57:50.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.949+0000 7f2508b65640 1 -- 172.21.15.164:0/1839511901 learned_addr learned my addr 172.21.15.164:0/1839511901 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:50.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.949+0000 7f2503fff640 1 --2- 172.21.15.164:0/1839511901 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f250409def0 0x7f250413c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:50.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.949+0000 7f25037fe640 1 --2- 172.21.15.164:0/1839511901 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f25040b4a00 0x7f25041e2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:50.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.949+0000 7f2508b65640 1 -- 172.21.15.164:0/1839511901 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f250409def0 msgr2=0x7f250413c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:50.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.949+0000 7f2508b65640 1 --2- 172.21.15.164:0/1839511901 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f250409def0 0x7f250413c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:50.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.949+0000 7f2508b65640 1 -- 172.21.15.164:0/1839511901 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f25040b4a00 msgr2=0x7f25041e2fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:50.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.949+0000 7f2508b65640 1 --2- 172.21.15.164:0/1839511901 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f25040b4a00 0x7f25041e2fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:50.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.949+0000 7f2508b65640 1 -- 172.21.15.164:0/1839511901 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24fc009b30 con 0x7f2504147420 2024-03-23T14:57:50.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.949+0000 7f25037fe640 1 --2- 172.21.15.164:0/1839511901 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f25040b4a00 0x7f25041e2fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T14:57:50.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.949+0000 7f2503fff640 1 --2- 172.21.15.164:0/1839511901 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f250409def0 0x7f250413c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T14:57:50.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.949+0000 7f2508b65640 1 --2- 172.21.15.164:0/1839511901 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2504147420 0x7f250413b510 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f24fc01aff0 tx=0x7f24fc0196e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:50.956 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.949+0000 7f2500ff9640 1 -- 172.21.15.164:0/1839511901 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24fc0a55a0 con 0x7f2504147420 2024-03-23T14:57:50.956 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.949+0000 7f2500ff9640 1 -- 172.21.15.164:0/1839511901 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24fc0b8690 con 0x7f2504147420 2024-03-23T14:57:50.956 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.949+0000 7f250a5c8640 1 -- 172.21.15.164:0/1839511901 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f250413aab0 con 0x7f2504147420 2024-03-23T14:57:50.956 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.949+0000 7f2500ff9640 1 -- 172.21.15.164:0/1839511901 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24fc0b8a40 con 0x7f2504147420 2024-03-23T14:57:50.956 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.949+0000 7f250a5c8640 1 -- 172.21.15.164:0/1839511901 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f250413bf00 con 0x7f2504147420 2024-03-23T14:57:50.956 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.949+0000 7f250a5c8640 1 -- 172.21.15.164:0/1839511901 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f250413b210 con 0x7f2504147420 2024-03-23T14:57:50.956 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.949+0000 7f2500ff9640 1 -- 172.21.15.164:0/1839511901 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7f24fc0b8ba0 con 0x7f2504147420 2024-03-23T14:57:50.957 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.949+0000 7f2500ff9640 1 --2- 172.21.15.164:0/1839511901 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f24e40672a0 0x7f24e4069760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:50.957 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.949+0000 7f2500ff9640 1 -- 172.21.15.164:0/1839511901 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5709+0+0 (secure 0 0 0) 0x7f24fc1203e0 con 0x7f2504147420 2024-03-23T14:57:50.957 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.949+0000 7f2503fff640 1 --2- 172.21.15.164:0/1839511901 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f24e40672a0 0x7f24e4069760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:50.957 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.949+0000 7f2503fff640 1 --2- 172.21.15.164:0/1839511901 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f24e40672a0 0x7f24e4069760 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f24ec003e40 tx=0x7f24ec009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:50.959 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:50.953+0000 7f2500ff9640 1 -- 172.21.15.164:0/1839511901 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7f24fc0ed0b0 con 0x7f2504147420 2024-03-23T14:57:51.123 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.117+0000 7f250a5c8640 1 -- 172.21.15.164:0/1839511901 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f25040690a0 con 0x7f2504147420 2024-03-23T14:57:51.124 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.121+0000 7f2500ff9640 1 -- 172.21.15.164:0/1839511901 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v29) ==== 104+0+11 (secure 0 0 0) 0x7f24fc009510 con 0x7f2504147420 2024-03-23T14:57:51.124 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 32 2024-03-23T14:57:51.126 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.121+0000 7f250a5c8640 1 -- 172.21.15.164:0/1839511901 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f24e40672a0 msgr2=0x7f24e4069760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:51.127 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.121+0000 7f250a5c8640 1 --2- 172.21.15.164:0/1839511901 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f24e40672a0 0x7f24e4069760 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f24ec003e40 tx=0x7f24ec009040 comp rx=0 tx=0).stop 2024-03-23T14:57:51.127 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.121+0000 7f250a5c8640 1 -- 172.21.15.164:0/1839511901 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2504147420 msgr2=0x7f250413b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:51.127 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.121+0000 7f250a5c8640 1 --2- 172.21.15.164:0/1839511901 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2504147420 0x7f250413b510 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f24fc01aff0 tx=0x7f24fc0196e0 comp rx=0 tx=0).stop 2024-03-23T14:57:51.127 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.121+0000 7f250a5c8640 1 -- 172.21.15.164:0/1839511901 shutdown_connections 2024-03-23T14:57:51.127 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.121+0000 7f250a5c8640 1 --2- 172.21.15.164:0/1839511901 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2504147420 0x7f250413b510 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:51.127 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.121+0000 7f250a5c8640 1 --2- 172.21.15.164:0/1839511901 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f24e40672a0 0x7f24e4069760 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:51.127 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.121+0000 7f250a5c8640 1 --2- 172.21.15.164:0/1839511901 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f25040b4a00 0x7f25041e2fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:51.127 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.121+0000 7f250a5c8640 1 --2- 172.21.15.164:0/1839511901 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f250409def0 0x7f250413c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:51.127 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.121+0000 7f250a5c8640 1 -- 172.21.15.164:0/1839511901 >> 172.21.15.164:0/1839511901 conn(0x7f25040b8ca0 msgr2=0x7f25040b3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:51.127 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.121+0000 7f250a5c8640 1 -- 172.21.15.164:0/1839511901 shutdown_connections 2024-03-23T14:57:51.127 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.121+0000 7f250a5c8640 1 -- 172.21.15.164:0/1839511901 wait complete. 2024-03-23T14:57:51.141 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-23T14:57:51.264 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.261+0000 7f0a94caa640 1 Processor -- start 2024-03-23T14:57:51.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.261+0000 7f0a94caa640 1 -- start start 2024-03-23T14:57:51.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.261+0000 7f0a94caa640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0a900b9ac0 0x7f0a9013f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:51.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.261+0000 7f0a94caa640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f0a9006f8e0 0x7f0a901459b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:51.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.261+0000 7f0a94caa640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a90145240 con 0x7f0a90145ef0 2024-03-23T14:57:51.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.261+0000 7f0a94caa640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 -- 0x7f0a900b3b50 con 0x7f0a9006f8e0 2024-03-23T14:57:51.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.261+0000 7f0a94caa640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f0a90070050 con 0x7f0a900b9ac0 2024-03-23T14:57:51.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.261+0000 7f0a8e59c640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0a900b9ac0 0x7f0a9013f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:51.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.261+0000 7f0a8dd9b640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f0a9006f8e0 0x7f0a901459b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:51.267 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.261+0000 7f0a8e59c640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0a900b9ac0 0x7f0a9013f240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.164:41460/0 (socket says 172.21.15.164:41460) 2024-03-23T14:57:51.267 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.261+0000 7f0a8e59c640 1 -- 172.21.15.164:0/2482133925 learned_addr learned my addr 172.21.15.164:0/2482133925 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:51.267 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.261+0000 7f0a8e59c640 1 -- 172.21.15.164:0/2482133925 >> v2:172.21.15.173:3300/0 conn(0x7f0a9006f8e0 msgr2=0x7f0a901459b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:51.267 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.261+0000 7f0a8e59c640 1 --2- 172.21.15.164:0/2482133925 >> v2:172.21.15.173:3300/0 conn(0x7f0a9006f8e0 0x7f0a901459b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:51.267 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.261+0000 7f0a8e59c640 1 -- 172.21.15.164:0/2482133925 >> v1:172.21.15.173:6789/0 conn(0x7f0a90145ef0 legacy=0x7f0a90148810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:51.267 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.261+0000 7f0a8e59c640 1 -- 172.21.15.164:0/2482133925 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a900b9ea0 con 0x7f0a900b9ac0 2024-03-23T14:57:51.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.265+0000 7f0a8e59c640 1 --2- 172.21.15.164:0/2482133925 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0a900b9ac0 0x7f0a9013f240 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f0a900ba090 tx=0x7f0a8001c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8381220797bfff1a server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:51.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.265+0000 7f0a8d59a640 1 -- 172.21.15.164:0/2482133925 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a800ac600 con 0x7f0a900b9ac0 2024-03-23T14:57:51.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.265+0000 7f0a8d59a640 1 -- 172.21.15.164:0/2482133925 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a800a8050 con 0x7f0a900b9ac0 2024-03-23T14:57:51.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.265+0000 7f0a8d59a640 1 -- 172.21.15.164:0/2482133925 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a800ac980 con 0x7f0a900b9ac0 2024-03-23T14:57:51.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.265+0000 7f0a94caa640 1 -- 172.21.15.164:0/2482133925 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0a900b9ac0 msgr2=0x7f0a9013f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:51.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.265+0000 7f0a94caa640 1 --2- 172.21.15.164:0/2482133925 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0a900b9ac0 0x7f0a9013f240 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f0a900ba090 tx=0x7f0a8001c480 comp rx=0 tx=0).stop 2024-03-23T14:57:51.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.265+0000 7f0a94caa640 1 -- 172.21.15.164:0/2482133925 shutdown_connections 2024-03-23T14:57:51.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.265+0000 7f0a94caa640 1 --2- 172.21.15.164:0/2482133925 >> v2:172.21.15.173:3300/0 conn(0x7f0a9006f8e0 0x7f0a901459b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:51.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.265+0000 7f0a94caa640 1 --2- 172.21.15.164:0/2482133925 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0a900b9ac0 0x7f0a9013f240 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:51.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.265+0000 7f0a94caa640 1 -- 172.21.15.164:0/2482133925 >> 172.21.15.164:0/2482133925 conn(0x7f0a9006b930 msgr2=0x7f0a900b9150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:51.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.265+0000 7f0a94caa640 1 -- 172.21.15.164:0/2482133925 shutdown_connections 2024-03-23T14:57:51.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.265+0000 7f0a94caa640 1 -- 172.21.15.164:0/2482133925 wait complete. 2024-03-23T14:57:51.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.265+0000 7f0a94caa640 1 Processor -- start 2024-03-23T14:57:51.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.265+0000 7f0a94caa640 1 -- start start 2024-03-23T14:57:51.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.265+0000 7f0a94caa640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0a900b9ac0 0x7f0a900b70c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:51.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.265+0000 7f0a94caa640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0a90145ef0 0x7f0a901448e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:51.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.265+0000 7f0a94caa640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0a900b7ed0 0x7f0a9012c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:51.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.265+0000 7f0a94caa640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0a90147930 con 0x7f0a900b9ac0 2024-03-23T14:57:51.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.265+0000 7f0a94caa640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f0a900b7760 con 0x7f0a90145ef0 2024-03-23T14:57:51.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.265+0000 7f0a94caa640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f0a90072800 con 0x7f0a900b7ed0 2024-03-23T14:57:51.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.265+0000 7f0a8e59c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0a900b9ac0 0x7f0a900b70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:51.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.265+0000 7f0a8e59c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0a900b9ac0 0x7f0a900b70c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:44964/0 (socket says 172.21.15.164:44964) 2024-03-23T14:57:51.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.269+0000 7f0a8e59c640 1 -- 172.21.15.164:0/1711045411 learned_addr learned my addr 172.21.15.164:0/1711045411 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:51.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.269+0000 7f0a8dd9b640 1 --2- 172.21.15.164:0/1711045411 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0a90145ef0 0x7f0a901448e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:51.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.269+0000 7f0a8ed9d640 1 --2- 172.21.15.164:0/1711045411 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0a900b7ed0 0x7f0a9012c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:51.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.269+0000 7f0a8e59c640 1 -- 172.21.15.164:0/1711045411 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0a900b7ed0 msgr2=0x7f0a9012c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:51.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.269+0000 7f0a8e59c640 1 --2- 172.21.15.164:0/1711045411 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0a900b7ed0 0x7f0a9012c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:51.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.269+0000 7f0a8e59c640 1 -- 172.21.15.164:0/1711045411 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0a90145ef0 msgr2=0x7f0a901448e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:51.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.269+0000 7f0a8e59c640 1 --2- 172.21.15.164:0/1711045411 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0a90145ef0 0x7f0a901448e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:51.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.269+0000 7f0a8e59c640 1 -- 172.21.15.164:0/1711045411 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a800094d0 con 0x7f0a900b9ac0 2024-03-23T14:57:51.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.269+0000 7f0a8dd9b640 1 --2- 172.21.15.164:0/1711045411 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0a90145ef0 0x7f0a901448e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T14:57:51.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.269+0000 7f0a8e59c640 1 --2- 172.21.15.164:0/1711045411 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0a900b9ac0 0x7f0a900b70c0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f0a80000c10 tx=0x7f0a8001acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:51.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.269+0000 7f0a86ffd640 1 -- 172.21.15.164:0/1711045411 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a800b7420 con 0x7f0a900b9ac0 2024-03-23T14:57:51.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.269+0000 7f0a86ffd640 1 -- 172.21.15.164:0/1711045411 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a800b7580 con 0x7f0a900b9ac0 2024-03-23T14:57:51.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.269+0000 7f0a94caa640 1 -- 172.21.15.164:0/1711045411 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a90070af0 con 0x7f0a900b9ac0 2024-03-23T14:57:51.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.269+0000 7f0a86ffd640 1 -- 172.21.15.164:0/1711045411 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a800b7930 con 0x7f0a900b9ac0 2024-03-23T14:57:51.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.269+0000 7f0a94caa640 1 -- 172.21.15.164:0/1711045411 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a901484d0 con 0x7f0a900b9ac0 2024-03-23T14:57:51.275 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.269+0000 7f0a94caa640 1 -- 172.21.15.164:0/1711045411 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a90070930 con 0x7f0a900b9ac0 2024-03-23T14:57:51.275 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.269+0000 7f0a86ffd640 1 -- 172.21.15.164:0/1711045411 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7f0a800b3040 con 0x7f0a900b9ac0 2024-03-23T14:57:51.275 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.269+0000 7f0a86ffd640 1 --2- 172.21.15.164:0/1711045411 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f0a700672a0 0x7f0a70069760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:51.275 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.269+0000 7f0a86ffd640 1 -- 172.21.15.164:0/1711045411 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5709+0+0 (secure 0 0 0) 0x7f0a8011f080 con 0x7f0a900b9ac0 2024-03-23T14:57:51.275 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.269+0000 7f0a8dd9b640 1 --2- 172.21.15.164:0/1711045411 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f0a700672a0 0x7f0a70069760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:51.276 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.269+0000 7f0a8dd9b640 1 --2- 172.21.15.164:0/1711045411 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f0a700672a0 0x7f0a70069760 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f0a90147620 tx=0x7f0a78008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:51.283 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.277+0000 7f0a86ffd640 1 -- 172.21.15.164:0/1711045411 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7f0a800ebcf0 con 0x7f0a900b9ac0 2024-03-23T14:57:51.529 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.525+0000 7f0a94caa640 1 -- 172.21.15.164:0/1711045411 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f0a900690a0 con 0x7f0a900b9ac0 2024-03-23T14:57:51.530 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.525+0000 7f0a86ffd640 1 -- 172.21.15.164:0/1711045411 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v29) ==== 100+0+11 (secure 0 0 0) 0x7f0a800ef3b0 con 0x7f0a900b9ac0 2024-03-23T14:57:51.530 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 64 2024-03-23T14:57:51.532 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.529+0000 7f0a94caa640 1 -- 172.21.15.164:0/1711045411 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f0a700672a0 msgr2=0x7f0a70069760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:51.532 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.529+0000 7f0a94caa640 1 --2- 172.21.15.164:0/1711045411 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f0a700672a0 0x7f0a70069760 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f0a90147620 tx=0x7f0a78008040 comp rx=0 tx=0).stop 2024-03-23T14:57:51.532 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.529+0000 7f0a94caa640 1 -- 172.21.15.164:0/1711045411 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0a900b9ac0 msgr2=0x7f0a900b70c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:51.532 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.529+0000 7f0a94caa640 1 --2- 172.21.15.164:0/1711045411 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0a900b9ac0 0x7f0a900b70c0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f0a80000c10 tx=0x7f0a8001acc0 comp rx=0 tx=0).stop 2024-03-23T14:57:51.532 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.529+0000 7f0a94caa640 1 -- 172.21.15.164:0/1711045411 shutdown_connections 2024-03-23T14:57:51.532 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.529+0000 7f0a94caa640 1 --2- 172.21.15.164:0/1711045411 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0a900b7ed0 0x7f0a9012c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:51.532 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.529+0000 7f0a94caa640 1 --2- 172.21.15.164:0/1711045411 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f0a700672a0 0x7f0a70069760 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:51.532 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.529+0000 7f0a94caa640 1 --2- 172.21.15.164:0/1711045411 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0a90145ef0 0x7f0a901448e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:51.532 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.529+0000 7f0a94caa640 1 --2- 172.21.15.164:0/1711045411 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0a900b9ac0 0x7f0a900b70c0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:51.533 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.529+0000 7f0a94caa640 1 -- 172.21.15.164:0/1711045411 >> 172.21.15.164:0/1711045411 conn(0x7f0a9006b930 msgr2=0x7f0a9013b100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:51.533 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.529+0000 7f0a94caa640 1 -- 172.21.15.164:0/1711045411 shutdown_connections 2024-03-23T14:57:51.533 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.529+0000 7f0a94caa640 1 -- 172.21.15.164:0/1711045411 wait complete. 2024-03-23T14:57:51.548 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-23T14:57:51.660 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.657+0000 7fba389d9640 1 Processor -- start 2024-03-23T14:57:51.660 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.657+0000 7fba389d9640 1 -- start start 2024-03-23T14:57:51.661 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.657+0000 7fba389d9640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fba34147890 0x7fba340b0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:51.661 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.657+0000 7fba389d9640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fba34070c00 0x7fba3414e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:51.661 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.657+0000 7fba389d9640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba341480c0 con 0x7fba340707c0 2024-03-23T14:57:51.661 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.657+0000 7fba389d9640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fba340705c0 con 0x7fba34070c00 2024-03-23T14:57:51.661 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.657+0000 7fba389d9640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fba34071070 con 0x7fba34147890 2024-03-23T14:57:51.661 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.657+0000 7fba32d9d640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fba34070c00 0x7fba3414e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:51.661 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.657+0000 7fba32d9d640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fba34070c00 0x7fba3414e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:44968/0 (socket says 172.21.15.164:44968) 2024-03-23T14:57:51.661 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.657+0000 7fba32d9d640 1 -- 172.21.15.164:0/1097067486 learned_addr learned my addr 172.21.15.164:0/1097067486 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:51.662 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.657+0000 7fba3259c640 1 --2- 172.21.15.164:0/1097067486 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fba34147890 0x7fba340b0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:51.662 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.657+0000 7fba32d9d640 1 -- 172.21.15.164:0/1097067486 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fba34147890 msgr2=0x7fba340b0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:51.662 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.657+0000 7fba32d9d640 1 --2- 172.21.15.164:0/1097067486 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fba34147890 0x7fba340b0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:51.662 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.657+0000 7fba32d9d640 1 -- 172.21.15.164:0/1097067486 >> v1:172.21.15.173:6789/0 conn(0x7fba340707c0 legacy=0x7fba3413df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:51.662 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.657+0000 7fba32d9d640 1 -- 172.21.15.164:0/1097067486 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba340725c0 con 0x7fba34070c00 2024-03-23T14:57:51.663 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.657+0000 7fba32d9d640 1 --2- 172.21.15.164:0/1097067486 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fba34070c00 0x7fba3414e980 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fba2c009e50 tx=0x7fba2c01c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d974c8c79ea734e server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:51.663 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.657+0000 7fba31d9b640 1 -- 172.21.15.164:0/1097067486 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba2c0a5550 con 0x7fba34070c00 2024-03-23T14:57:51.663 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.657+0000 7fba31d9b640 1 -- 172.21.15.164:0/1097067486 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba2c0a56b0 con 0x7fba34070c00 2024-03-23T14:57:51.663 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.657+0000 7fba31d9b640 1 -- 172.21.15.164:0/1097067486 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba2c0a5a40 con 0x7fba34070c00 2024-03-23T14:57:51.664 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.661+0000 7fba389d9640 1 -- 172.21.15.164:0/1097067486 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fba34070c00 msgr2=0x7fba3414e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:51.664 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.661+0000 7fba389d9640 1 --2- 172.21.15.164:0/1097067486 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fba34070c00 0x7fba3414e980 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fba2c009e50 tx=0x7fba2c01c8d0 comp rx=0 tx=0).stop 2024-03-23T14:57:51.664 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.661+0000 7fba389d9640 1 -- 172.21.15.164:0/1097067486 shutdown_connections 2024-03-23T14:57:51.664 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.661+0000 7fba389d9640 1 --2- 172.21.15.164:0/1097067486 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fba34070c00 0x7fba3414e980 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:51.664 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.661+0000 7fba389d9640 1 --2- 172.21.15.164:0/1097067486 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fba34147890 0x7fba340b0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:51.665 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.661+0000 7fba389d9640 1 -- 172.21.15.164:0/1097067486 >> 172.21.15.164:0/1097067486 conn(0x7fba340b7970 msgr2=0x7fba340b7d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:51.665 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.661+0000 7fba389d9640 1 -- 172.21.15.164:0/1097067486 shutdown_connections 2024-03-23T14:57:51.665 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.661+0000 7fba389d9640 1 -- 172.21.15.164:0/1097067486 wait complete. 2024-03-23T14:57:51.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.661+0000 7fba389d9640 1 Processor -- start 2024-03-23T14:57:51.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.661+0000 7fba389d9640 1 -- start start 2024-03-23T14:57:51.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.661+0000 7fba389d9640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fba340707c0 0x7fba3414ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:51.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.661+0000 7fba389d9640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fba34070c00 0x7fba34143c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:51.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.661+0000 7fba389d9640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fba34147890 0x7fba341e0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:51.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.661+0000 7fba389d9640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fba341dfd10 con 0x7fba34070c00 2024-03-23T14:57:51.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.661+0000 7fba389d9640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fba341df670 con 0x7fba340707c0 2024-03-23T14:57:51.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.661+0000 7fba389d9640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fba341df930 con 0x7fba34147890 2024-03-23T14:57:51.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.661+0000 7fba2bfff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fba34070c00 0x7fba34143c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:51.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.661+0000 7fba2bfff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fba34070c00 0x7fba34143c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:44982/0 (socket says 172.21.15.164:44982) 2024-03-23T14:57:51.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.661+0000 7fba2bfff640 1 -- 172.21.15.164:0/967867655 learned_addr learned my addr 172.21.15.164:0/967867655 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:51.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.661+0000 7fba3259c640 1 --2- 172.21.15.164:0/967867655 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fba340707c0 0x7fba3414ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:51.667 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.661+0000 7fba32d9d640 1 --2- 172.21.15.164:0/967867655 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fba34147890 0x7fba341e0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:51.668 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.661+0000 7fba2bfff640 1 -- 172.21.15.164:0/967867655 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fba34147890 msgr2=0x7fba341e0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:51.668 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.661+0000 7fba2bfff640 1 --2- 172.21.15.164:0/967867655 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fba34147890 0x7fba341e0130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:51.668 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.661+0000 7fba2bfff640 1 -- 172.21.15.164:0/967867655 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fba340707c0 msgr2=0x7fba3414ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:51.668 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.661+0000 7fba2bfff640 1 --2- 172.21.15.164:0/967867655 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fba340707c0 0x7fba3414ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:51.668 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.661+0000 7fba2bfff640 1 -- 172.21.15.164:0/967867655 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba2c009b30 con 0x7fba34070c00 2024-03-23T14:57:51.668 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.661+0000 7fba32d9d640 1 --2- 172.21.15.164:0/967867655 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fba34147890 0x7fba341e0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T14:57:51.668 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.661+0000 7fba2bfff640 1 --2- 172.21.15.164:0/967867655 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fba34070c00 0x7fba34143c10 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fba1c00c450 tx=0x7fba1c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:51.668 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.661+0000 7fba2affd640 1 -- 172.21.15.164:0/967867655 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba1c012830 con 0x7fba34070c00 2024-03-23T14:57:51.668 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.661+0000 7fba2affd640 1 -- 172.21.15.164:0/967867655 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba1c0129b0 con 0x7fba34070c00 2024-03-23T14:57:51.668 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.661+0000 7fba2affd640 1 -- 172.21.15.164:0/967867655 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba1c012d60 con 0x7fba34070c00 2024-03-23T14:57:51.668 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.661+0000 7fba389d9640 1 -- 172.21.15.164:0/967867655 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba340b1f30 con 0x7fba34070c00 2024-03-23T14:57:51.668 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.661+0000 7fba389d9640 1 -- 172.21.15.164:0/967867655 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba340b0e70 con 0x7fba34070c00 2024-03-23T14:57:51.669 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.661+0000 7fba389d9640 1 -- 172.21.15.164:0/967867655 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba341dfe50 con 0x7fba34070c00 2024-03-23T14:57:51.669 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.665+0000 7fba2affd640 1 -- 172.21.15.164:0/967867655 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7fba1c01c460 con 0x7fba34070c00 2024-03-23T14:57:51.669 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.665+0000 7fba2affd640 1 --2- 172.21.15.164:0/967867655 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fba140672a0 0x7fba14069760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:51.669 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.665+0000 7fba3259c640 1 --2- 172.21.15.164:0/967867655 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fba140672a0 0x7fba14069760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:51.669 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.665+0000 7fba2affd640 1 -- 172.21.15.164:0/967867655 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5709+0+0 (secure 0 0 0) 0x7fba1c085830 con 0x7fba34070c00 2024-03-23T14:57:51.669 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.665+0000 7fba3259c640 1 --2- 172.21.15.164:0/967867655 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fba140672a0 0x7fba14069760 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fba2400b930 tx=0x7fba24008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:51.672 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.669+0000 7fba2affd640 1 -- 172.21.15.164:0/967867655 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7fba1c052450 con 0x7fba34070c00 2024-03-23T14:57:51.934 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.929+0000 7fba389d9640 1 -- 172.21.15.164:0/967867655 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fba341dfe50 con 0x7fba34070c00 2024-03-23T14:57:51.935 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.929+0000 7fba2affd640 1 -- 172.21.15.164:0/967867655 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v29) ==== 80+0+0 (secure 0 0 0) 0x7fba341dfe50 con 0x7fba34070c00 2024-03-23T14:57:51.935 INFO:teuthology.orchestra.run.smithi164.stderr:listed 0 entries 2024-03-23T14:57:51.937 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.933+0000 7fba389d9640 1 -- 172.21.15.164:0/967867655 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fba140672a0 msgr2=0x7fba14069760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:51.938 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.933+0000 7fba389d9640 1 --2- 172.21.15.164:0/967867655 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fba140672a0 0x7fba14069760 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fba2400b930 tx=0x7fba24008040 comp rx=0 tx=0).stop 2024-03-23T14:57:51.938 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.933+0000 7fba389d9640 1 -- 172.21.15.164:0/967867655 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fba34070c00 msgr2=0x7fba34143c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:51.938 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.933+0000 7fba389d9640 1 --2- 172.21.15.164:0/967867655 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fba34070c00 0x7fba34143c10 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fba1c00c450 tx=0x7fba1c00c910 comp rx=0 tx=0).stop 2024-03-23T14:57:51.938 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.933+0000 7fba389d9640 1 -- 172.21.15.164:0/967867655 shutdown_connections 2024-03-23T14:57:51.938 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.933+0000 7fba389d9640 1 --2- 172.21.15.164:0/967867655 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fba34147890 0x7fba341e0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:51.938 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.933+0000 7fba389d9640 1 --2- 172.21.15.164:0/967867655 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fba34070c00 0x7fba34143c10 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:51.938 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.933+0000 7fba389d9640 1 --2- 172.21.15.164:0/967867655 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fba140672a0 0x7fba14069760 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:51.938 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.933+0000 7fba389d9640 1 --2- 172.21.15.164:0/967867655 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fba340707c0 0x7fba3414ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:51.938 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.933+0000 7fba389d9640 1 -- 172.21.15.164:0/967867655 >> 172.21.15.164:0/967867655 conn(0x7fba340b7970 msgr2=0x7fba34144240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:51.938 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.933+0000 7fba389d9640 1 -- 172.21.15.164:0/967867655 shutdown_connections 2024-03-23T14:57:51.938 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:51.933+0000 7fba389d9640 1 -- 172.21.15.164:0/967867655 wait complete. 2024-03-23T14:57:51.954 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi164.front.sepia.ceph.com... 2024-03-23T14:57:51.954 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T14:57:51.954 DEBUG:teuthology.orchestra.run.smithi164:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-03-23T14:57:51.965 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi164.stderr:ceph-fuse[37029]: fuse finished with error 0 and tester_r 0 2024-03-23T14:57:51.966 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi164.front.sepia.ceph.com 2024-03-23T14:57:51.966 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T14:57:51.966 DEBUG:teuthology.orchestra.run.smithi164:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-23T14:57:52.129 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T14:57:52.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.293+0000 7f5cf07b1640 1 Processor -- start 2024-03-23T14:57:52.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.293+0000 7f5cf07b1640 1 -- start start 2024-03-23T14:57:52.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.293+0000 7f5cf07b1640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f5ce80b9ac0 0x7f5ce813f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:52.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.293+0000 7f5cf07b1640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5ce806f8e0 0x7f5ce81459b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:52.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.293+0000 7f5cf07b1640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ce8145240 con 0x7f5ce8145ef0 2024-03-23T14:57:52.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.293+0000 7f5cf07b1640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f5ce80b3b50 con 0x7f5ce80b9ac0 2024-03-23T14:57:52.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.293+0000 7f5cf07b1640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f5ce8070050 con 0x7f5ce806f8e0 2024-03-23T14:57:52.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.293+0000 7f5cee54d640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f5ce80b9ac0 0x7f5ce813f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:52.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.293+0000 7f5cee54d640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f5ce80b9ac0 0x7f5ce813f240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:44994/0 (socket says 172.21.15.164:44994) 2024-03-23T14:57:52.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.293+0000 7f5cee54d640 1 -- 172.21.15.164:0/3998252155 learned_addr learned my addr 172.21.15.164:0/3998252155 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:52.300 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.297+0000 7f5cedd4c640 1 --2- 172.21.15.164:0/3998252155 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5ce806f8e0 0x7f5ce81459b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:52.301 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.297+0000 7f5cee54d640 1 -- 172.21.15.164:0/3998252155 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5ce806f8e0 msgr2=0x7f5ce81459b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:52.301 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.297+0000 7f5cee54d640 1 --2- 172.21.15.164:0/3998252155 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5ce806f8e0 0x7f5ce81459b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:52.301 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.297+0000 7f5cee54d640 1 -- 172.21.15.164:0/3998252155 >> v1:172.21.15.173:6789/0 conn(0x7f5ce8145ef0 legacy=0x7f5ce8148810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:52.301 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.297+0000 7f5cee54d640 1 -- 172.21.15.164:0/3998252155 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ce80b9ea0 con 0x7f5ce80b9ac0 2024-03-23T14:57:52.301 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.297+0000 7f5cedd4c640 1 --2- 172.21.15.164:0/3998252155 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5ce806f8e0 0x7f5ce81459b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T14:57:52.301 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.297+0000 7f5cee54d640 1 --2- 172.21.15.164:0/3998252155 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5ce80b9ac0 0x7f5ce813f240 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f5ce80ba090 tx=0x7f5cd801c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9ffffc203e3a42e server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:52.302 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.297+0000 7f5ced54b640 1 -- 172.21.15.164:0/3998252155 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cd80ab600 con 0x7f5ce80b9ac0 2024-03-23T14:57:52.302 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.297+0000 7f5ced54b640 1 -- 172.21.15.164:0/3998252155 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5cd80a7050 con 0x7f5ce80b9ac0 2024-03-23T14:57:52.302 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.297+0000 7f5ced54b640 1 -- 172.21.15.164:0/3998252155 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cd80ab980 con 0x7f5ce80b9ac0 2024-03-23T14:57:52.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.297+0000 7f5cf07b1640 1 -- 172.21.15.164:0/3998252155 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5ce80b9ac0 msgr2=0x7f5ce813f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:52.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.297+0000 7f5cf07b1640 1 --2- 172.21.15.164:0/3998252155 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5ce80b9ac0 0x7f5ce813f240 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f5ce80ba090 tx=0x7f5cd801c440 comp rx=0 tx=0).stop 2024-03-23T14:57:52.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.297+0000 7f5cf07b1640 1 -- 172.21.15.164:0/3998252155 shutdown_connections 2024-03-23T14:57:52.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.297+0000 7f5cf07b1640 1 --2- 172.21.15.164:0/3998252155 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5ce806f8e0 0x7f5ce81459b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:52.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.297+0000 7f5cf07b1640 1 --2- 172.21.15.164:0/3998252155 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5ce80b9ac0 0x7f5ce813f240 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:52.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.297+0000 7f5cf07b1640 1 -- 172.21.15.164:0/3998252155 >> 172.21.15.164:0/3998252155 conn(0x7f5ce806b930 msgr2=0x7f5ce80b9150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:52.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.297+0000 7f5cf07b1640 1 -- 172.21.15.164:0/3998252155 shutdown_connections 2024-03-23T14:57:52.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.297+0000 7f5cf07b1640 1 -- 172.21.15.164:0/3998252155 wait complete. 2024-03-23T14:57:52.305 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.301+0000 7f5cf07b1640 1 Processor -- start 2024-03-23T14:57:52.305 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.301+0000 7f5cf07b1640 1 -- start start 2024-03-23T14:57:52.305 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.301+0000 7f5cf07b1640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5ce80b9ac0 0x7f5ce812f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:52.306 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.301+0000 7f5cf07b1640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5ce8145ef0 0x7f5ce81e0d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:52.306 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.301+0000 7f5cf07b1640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5ce81e12c0 0x7f5ce80727e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:52.306 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.301+0000 7f5cf07b1640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5ce81478d0 con 0x7f5ce8145ef0 2024-03-23T14:57:52.306 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.301+0000 7f5cf07b1640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f5ce8143220 con 0x7f5ce80b9ac0 2024-03-23T14:57:52.306 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.301+0000 7f5cf07b1640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f5ce81e1750 con 0x7f5ce81e12c0 2024-03-23T14:57:52.306 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.301+0000 7f5cedd4c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5ce8145ef0 0x7f5ce81e0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:52.306 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.301+0000 7f5cedd4c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5ce8145ef0 0x7f5ce81e0d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:45006/0 (socket says 172.21.15.164:45006) 2024-03-23T14:57:52.306 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.301+0000 7f5cedd4c640 1 -- 172.21.15.164:0/3760068535 learned_addr learned my addr 172.21.15.164:0/3760068535 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:52.306 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.301+0000 7f5cee54d640 1 --2- 172.21.15.164:0/3760068535 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5ce80b9ac0 0x7f5ce812f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:52.306 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.301+0000 7f5ceed4e640 1 --2- 172.21.15.164:0/3760068535 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5ce81e12c0 0x7f5ce80727e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:52.306 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.301+0000 7f5cedd4c640 1 -- 172.21.15.164:0/3760068535 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5ce81e12c0 msgr2=0x7f5ce80727e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:52.306 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.301+0000 7f5cedd4c640 1 --2- 172.21.15.164:0/3760068535 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5ce81e12c0 0x7f5ce80727e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:52.306 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.301+0000 7f5cedd4c640 1 -- 172.21.15.164:0/3760068535 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5ce80b9ac0 msgr2=0x7f5ce812f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:52.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.301+0000 7f5cedd4c640 1 --2- 172.21.15.164:0/3760068535 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5ce80b9ac0 0x7f5ce812f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:52.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.301+0000 7f5cedd4c640 1 -- 172.21.15.164:0/3760068535 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cd80094d0 con 0x7f5ce8145ef0 2024-03-23T14:57:52.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.301+0000 7f5cedd4c640 1 --2- 172.21.15.164:0/3760068535 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5ce8145ef0 0x7f5ce81e0d80 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f5ce400cf90 tx=0x7f5ce400a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:52.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.301+0000 7f5cdeffd640 1 -- 172.21.15.164:0/3760068535 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ce400d840 con 0x7f5ce8145ef0 2024-03-23T14:57:52.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.301+0000 7f5cdeffd640 1 -- 172.21.15.164:0/3760068535 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ce400d9a0 con 0x7f5ce8145ef0 2024-03-23T14:57:52.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.301+0000 7f5cf07b1640 1 -- 172.21.15.164:0/3760068535 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ce8070af0 con 0x7f5ce8145ef0 2024-03-23T14:57:52.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.301+0000 7f5cdeffd640 1 -- 172.21.15.164:0/3760068535 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ce400dd30 con 0x7f5ce8145ef0 2024-03-23T14:57:52.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.301+0000 7f5cf07b1640 1 -- 172.21.15.164:0/3760068535 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ce8146bd0 con 0x7f5ce8145ef0 2024-03-23T14:57:52.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.301+0000 7f5cdeffd640 1 -- 172.21.15.164:0/3760068535 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7f5ce4015440 con 0x7f5ce8145ef0 2024-03-23T14:57:52.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.301+0000 7f5cf07b1640 1 -- 172.21.15.164:0/3760068535 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ce80690a0 con 0x7f5ce8145ef0 2024-03-23T14:57:52.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.301+0000 7f5cdeffd640 1 --2- 172.21.15.164:0/3760068535 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f5cd40672a0 0x7f5cd4069760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:52.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.301+0000 7f5cdeffd640 1 -- 172.21.15.164:0/3760068535 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5709+0+0 (secure 0 0 0) 0x7f5ce4085980 con 0x7f5ce8145ef0 2024-03-23T14:57:52.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.301+0000 7f5cee54d640 1 --2- 172.21.15.164:0/3760068535 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f5cd40672a0 0x7f5cd4069760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:52.308 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.305+0000 7f5cee54d640 1 --2- 172.21.15.164:0/3760068535 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f5cd40672a0 0x7f5cd4069760 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f5cd80ad000 tx=0x7f5cd8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:52.311 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.305+0000 7f5cdeffd640 1 -- 172.21.15.164:0/3760068535 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7f5ce4052650 con 0x7f5ce8145ef0 2024-03-23T14:57:52.492 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.489+0000 7f5cf07b1640 1 -- 172.21.15.164:0/3760068535 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5ce8147b20 con 0x7f5ce8145ef0 2024-03-23T14:57:52.492 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.489+0000 7f5cdeffd640 1 -- 172.21.15.164:0/3760068535 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+3537 (secure 0 0 0) 0x7f5ce401a020 con 0x7f5ce8145ef0 2024-03-23T14:57:52.493 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-23T14:57:52.493 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":13,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4619,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/1401585102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":1401585102},{"type":"v1","addr":"172.21.15.173:6833","nonce":1401585102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-23T14:57:40.961416+0000","modified":"2024-03-23T14:57:43.045871+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4935},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4935":{"gid":4935,"name":"a","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.164:6835/3261092607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":3261092607},{"type":"v1","addr":"172.21.15.164:6835","nonce":3261092607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4935,"qdb_cluster":[4935]},"id":2}]} 2024-03-23T14:57:52.493 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 13 2024-03-23T14:57:52.495 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.489+0000 7f5cf07b1640 1 -- 172.21.15.164:0/3760068535 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f5cd40672a0 msgr2=0x7f5cd4069760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:52.495 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.489+0000 7f5cf07b1640 1 --2- 172.21.15.164:0/3760068535 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f5cd40672a0 0x7f5cd4069760 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f5cd80ad000 tx=0x7f5cd8008040 comp rx=0 tx=0).stop 2024-03-23T14:57:52.495 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.489+0000 7f5cf07b1640 1 -- 172.21.15.164:0/3760068535 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5ce8145ef0 msgr2=0x7f5ce81e0d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:52.495 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.489+0000 7f5cf07b1640 1 --2- 172.21.15.164:0/3760068535 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5ce8145ef0 0x7f5ce81e0d80 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f5ce400cf90 tx=0x7f5ce400a7d0 comp rx=0 tx=0).stop 2024-03-23T14:57:52.495 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.489+0000 7f5cf07b1640 1 -- 172.21.15.164:0/3760068535 shutdown_connections 2024-03-23T14:57:52.495 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.489+0000 7f5cf07b1640 1 --2- 172.21.15.164:0/3760068535 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5ce81e12c0 0x7f5ce80727e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:52.495 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.489+0000 7f5cf07b1640 1 --2- 172.21.15.164:0/3760068535 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5ce8145ef0 0x7f5ce81e0d80 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:52.495 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.489+0000 7f5cf07b1640 1 --2- 172.21.15.164:0/3760068535 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f5cd40672a0 0x7f5cd4069760 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:52.495 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.489+0000 7f5cf07b1640 1 --2- 172.21.15.164:0/3760068535 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5ce80b9ac0 0x7f5ce812f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:52.495 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.489+0000 7f5cf07b1640 1 -- 172.21.15.164:0/3760068535 >> 172.21.15.164:0/3760068535 conn(0x7f5ce806b930 msgr2=0x7f5ce813b2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:52.495 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.489+0000 7f5cf07b1640 1 -- 172.21.15.164:0/3760068535 shutdown_connections 2024-03-23T14:57:52.495 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.489+0000 7f5cf07b1640 1 -- 172.21.15.164:0/3760068535 wait complete. 2024-03-23T14:57:52.510 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T14:57:52.624 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.621+0000 7fe956aa6640 1 Processor -- start 2024-03-23T14:57:52.624 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.621+0000 7fe956aa6640 1 -- start start 2024-03-23T14:57:52.627 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.621+0000 7fe956aa6640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fe9500b56f0 0x7fe95014a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:52.627 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.621+0000 7fe956aa6640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe9500b4a70 0x7fe950140f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:52.627 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.621+0000 7fe956aa6640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9500b4300 con 0x7fe950141470 2024-03-23T14:57:52.627 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.621+0000 7fe956aa6640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 -- 0x7fe9500b5fa0 con 0x7fe9500b56f0 2024-03-23T14:57:52.627 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.621+0000 7fe956aa6640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fe9500b45e0 con 0x7fe9500b4a70 2024-03-23T14:57:52.627 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.621+0000 7fe955043640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fe950141470 0x7fe9500b75f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:43054/0 (socket says 172.21.15.164:43054) 2024-03-23T14:57:52.627 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.621+0000 7fe955043640 1 -- 172.21.15.164:0/2952550394 learned_addr learned my addr 172.21.15.164:0/2952550394 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-03-23T14:57:52.627 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.621+0000 7fe94f7fe640 1 -- 172.21.15.164:0/2952550394 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2746288214 0 0) 0x7fe9500b4300 con 0x7fe950141470 2024-03-23T14:57:52.627 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.621+0000 7fe94ffff640 1 --2- 172.21.15.164:0/2952550394 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe9500b4a70 0x7fe950140f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:52.627 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.621+0000 7fe954842640 1 --2- 172.21.15.164:0/2952550394 >> v2:172.21.15.173:3300/0 conn(0x7fe9500b56f0 0x7fe95014a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:52.627 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.621+0000 7fe94f7fe640 1 -- 172.21.15.164:0/2952550394 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe940003530 con 0x7fe950141470 2024-03-23T14:57:52.628 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.621+0000 7fe94f7fe640 1 -- 172.21.15.164:0/2952550394 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2005447135 0 0) 0x7fe940003530 con 0x7fe950141470 2024-03-23T14:57:52.628 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.621+0000 7fe94f7fe640 1 -- 172.21.15.164:0/2952550394 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe9500b4a70 msgr2=0x7fe950140f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:52.628 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.621+0000 7fe94f7fe640 1 --2- 172.21.15.164:0/2952550394 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe9500b4a70 0x7fe950140f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:52.628 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.621+0000 7fe94f7fe640 1 -- 172.21.15.164:0/2952550394 >> v2:172.21.15.173:3300/0 conn(0x7fe9500b56f0 msgr2=0x7fe95014a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:52.628 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.621+0000 7fe94f7fe640 1 --2- 172.21.15.164:0/2952550394 >> v2:172.21.15.173:3300/0 conn(0x7fe9500b56f0 0x7fe95014a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:52.628 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.621+0000 7fe94f7fe640 1 -- 172.21.15.164:0/2952550394 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe95014ade0 con 0x7fe950141470 2024-03-23T14:57:52.628 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.621+0000 7fe94f7fe640 1 -- 172.21.15.164:0/2952550394 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2526390548 0 0) 0x7fe948001a00 con 0x7fe950141470 2024-03-23T14:57:52.628 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.621+0000 7fe94f7fe640 1 -- 172.21.15.164:0/2952550394 >> v1:172.21.15.164:6789/0 conn(0x7fe950141470 legacy=0x7fe9500b75f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:52.628 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.621+0000 7fe94f7fe640 1 --2- 172.21.15.164:0/2952550394 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe9500b4a70 0x7fe940003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:52.628 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.621+0000 7fe94f7fe640 1 --2- 172.21.15.164:0/2952550394 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe940003f00 0x7fe9400047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:52.628 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.621+0000 7fe94f7fe640 1 --2- 172.21.15.164:0/2952550394 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe940004d10 0x7fe940005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:52.628 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.621+0000 7fe94f7fe640 1 -- 172.21.15.164:0/2952550394 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe9500b45e0 con 0x7fe940003f00 2024-03-23T14:57:52.628 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.621+0000 7fe94f7fe640 1 -- 172.21.15.164:0/2952550394 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fe94000b990 con 0x7fe940004d10 2024-03-23T14:57:52.628 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.621+0000 7fe94f7fe640 1 -- 172.21.15.164:0/2952550394 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fe94000bad0 con 0x7fe9500b4a70 2024-03-23T14:57:52.628 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.621+0000 7fe94ffff640 1 --2- 172.21.15.164:0/2952550394 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe940003f00 0x7fe9400047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:52.629 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.621+0000 7fe954842640 1 --2- 172.21.15.164:0/2952550394 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe9500b4a70 0x7fe940003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:52.629 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.621+0000 7fe955043640 1 --2- 172.21.15.164:0/2952550394 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe940004d10 0x7fe940005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:52.629 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.621+0000 7fe94ffff640 1 -- 172.21.15.164:0/2952550394 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe9500b4a70 msgr2=0x7fe940003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:52.629 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.621+0000 7fe94ffff640 1 --2- 172.21.15.164:0/2952550394 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe9500b4a70 0x7fe940003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:52.629 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.621+0000 7fe94ffff640 1 -- 172.21.15.164:0/2952550394 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe940004d10 msgr2=0x7fe940005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:52.629 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.621+0000 7fe94ffff640 1 --2- 172.21.15.164:0/2952550394 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe940004d10 0x7fe940005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:52.629 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.621+0000 7fe94ffff640 1 -- 172.21.15.164:0/2952550394 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe940003530 con 0x7fe940003f00 2024-03-23T14:57:52.629 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.621+0000 7fe954842640 1 --2- 172.21.15.164:0/2952550394 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe9500b4a70 0x7fe940003980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T14:57:52.629 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.621+0000 7fe955043640 1 --2- 172.21.15.164:0/2952550394 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe940004d10 0x7fe940005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T14:57:52.630 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.625+0000 7fe94ffff640 1 --2- 172.21.15.164:0/2952550394 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe940003f00 0x7fe9400047d0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fe93c009ae0 tx=0x7fe93c01ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=695746981c5be43c server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:52.630 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.625+0000 7fe94f7fe640 1 -- 172.21.15.164:0/2952550394 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe93c0a5660 con 0x7fe940003f00 2024-03-23T14:57:52.630 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.625+0000 7fe94f7fe640 1 -- 172.21.15.164:0/2952550394 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe93c0a57c0 con 0x7fe940003f00 2024-03-23T14:57:52.630 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.625+0000 7fe94f7fe640 1 -- 172.21.15.164:0/2952550394 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe93c0a5b70 con 0x7fe940003f00 2024-03-23T14:57:52.630 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.625+0000 7fe956aa6640 1 -- 172.21.15.164:0/2952550394 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe940003f00 msgr2=0x7fe9400047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:52.630 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.625+0000 7fe956aa6640 1 --2- 172.21.15.164:0/2952550394 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe940003f00 0x7fe9400047d0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fe93c009ae0 tx=0x7fe93c01ca10 comp rx=0 tx=0).stop 2024-03-23T14:57:52.631 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.625+0000 7fe955043640 1 -- 172.21.15.164:0/2952550394 reap_dead start 2024-03-23T14:57:52.631 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.625+0000 7fe956aa6640 1 -- 172.21.15.164:0/2952550394 shutdown_connections 2024-03-23T14:57:52.631 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.625+0000 7fe956aa6640 1 -- 172.21.15.164:0/2952550394 >> 172.21.15.164:0/2952550394 conn(0x7fe95002c3c0 msgr2=0x7fe95002c7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:52.631 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.625+0000 7fe956aa6640 1 -- 172.21.15.164:0/2952550394 shutdown_connections 2024-03-23T14:57:52.631 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.625+0000 7fe956aa6640 1 -- 172.21.15.164:0/2952550394 wait complete. 2024-03-23T14:57:52.631 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.625+0000 7fe956aa6640 1 Processor -- start 2024-03-23T14:57:52.631 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.625+0000 7fe956aa6640 1 -- start start 2024-03-23T14:57:52.633 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.629+0000 7fe956aa6640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe940003f00 0x7fe95013d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:52.633 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.629+0000 7fe956aa6640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe95013bb80 0x7fe9501dc960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:52.633 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.629+0000 7fe956aa6640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe9501d6720 0x7fe9501d6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:52.633 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.629+0000 7fe956aa6640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe95013c9a0 con 0x7fe95013bb80 2024-03-23T14:57:52.634 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.629+0000 7fe956aa6640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fe9501d7110 con 0x7fe9501d6720 2024-03-23T14:57:52.634 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.629+0000 7fe956aa6640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fe95013ced0 con 0x7fe940003f00 2024-03-23T14:57:52.634 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.629+0000 7fe94ffff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe95013bb80 0x7fe9501dc960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:52.634 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.629+0000 7fe94ffff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe95013bb80 0x7fe9501dc960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:48586/0 (socket says 172.21.15.164:48586) 2024-03-23T14:57:52.634 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.629+0000 7fe94ffff640 1 -- 172.21.15.164:0/3014893960 learned_addr learned my addr 172.21.15.164:0/3014893960 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:52.634 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.629+0000 7fe954842640 1 --2- 172.21.15.164:0/3014893960 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe940003f00 0x7fe95013d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:52.634 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.629+0000 7fe955043640 1 --2- 172.21.15.164:0/3014893960 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe9501d6720 0x7fe9501d6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:52.634 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.629+0000 7fe94ffff640 1 -- 172.21.15.164:0/3014893960 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe940003f00 msgr2=0x7fe95013d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:52.634 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.629+0000 7fe94ffff640 1 --2- 172.21.15.164:0/3014893960 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe940003f00 0x7fe95013d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:52.634 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.629+0000 7fe94ffff640 1 -- 172.21.15.164:0/3014893960 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe9501d6720 msgr2=0x7fe9501d6bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:52.634 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.629+0000 7fe94ffff640 1 --2- 172.21.15.164:0/3014893960 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe9501d6720 0x7fe9501d6bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:52.634 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.629+0000 7fe94ffff640 1 -- 172.21.15.164:0/3014893960 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe950144350 con 0x7fe95013bb80 2024-03-23T14:57:52.634 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.629+0000 7fe955043640 1 --2- 172.21.15.164:0/3014893960 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe9501d6720 0x7fe9501d6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T14:57:52.634 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.629+0000 7fe94ffff640 1 --2- 172.21.15.164:0/3014893960 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe95013bb80 0x7fe9501dc960 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fe93c01c840 tx=0x7fe940005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:52.635 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.629+0000 7fe94d7fa640 1 -- 172.21.15.164:0/3014893960 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe93c004220 con 0x7fe95013bb80 2024-03-23T14:57:52.635 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.629+0000 7fe94d7fa640 1 -- 172.21.15.164:0/3014893960 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe93c0043e0 con 0x7fe95013bb80 2024-03-23T14:57:52.635 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.629+0000 7fe956aa6640 1 -- 172.21.15.164:0/3014893960 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe95013c9a0 con 0x7fe95013bb80 2024-03-23T14:57:52.635 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.629+0000 7fe94d7fa640 1 -- 172.21.15.164:0/3014893960 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe93c003860 con 0x7fe95013bb80 2024-03-23T14:57:52.635 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.629+0000 7fe956aa6640 1 -- 172.21.15.164:0/3014893960 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe950143710 con 0x7fe95013bb80 2024-03-23T14:57:52.636 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.629+0000 7fe956aa6640 1 -- 172.21.15.164:0/3014893960 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe95013b8a0 con 0x7fe95013bb80 2024-03-23T14:57:52.636 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.629+0000 7fe94d7fa640 1 -- 172.21.15.164:0/3014893960 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7fe93c0b4040 con 0x7fe95013bb80 2024-03-23T14:57:52.636 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.629+0000 7fe94d7fa640 1 --2- 172.21.15.164:0/3014893960 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fe9340672a0 0x7fe934069760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:52.636 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.629+0000 7fe954842640 1 --2- 172.21.15.164:0/3014893960 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fe9340672a0 0x7fe934069760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:52.636 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.629+0000 7fe94d7fa640 1 -- 172.21.15.164:0/3014893960 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5709+0+0 (secure 0 0 0) 0x7fe93c11eb30 con 0x7fe95013bb80 2024-03-23T14:57:52.636 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.633+0000 7fe954842640 1 --2- 172.21.15.164:0/3014893960 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fe9340672a0 0x7fe934069760 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe950148290 tx=0x7fe944008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:52.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.633+0000 7fe94d7fa640 1 -- 172.21.15.164:0/3014893960 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7fe95013b8a0 con 0x7fe95013bb80 2024-03-23T14:57:52.793 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.789+0000 7fe956aa6640 1 -- 172.21.15.164:0/3014893960 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe95013b850 con 0x7fe95013bb80 2024-03-23T14:57:52.794 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.789+0000 7fe94d7fa640 1 -- 172.21.15.164:0/3014893960 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) ==== 74+0+16693 (secure 0 0 0) 0x7fe93c0eb870 con 0x7fe95013bb80 2024-03-23T14:57:52.795 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-23T14:57:52.795 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":29,"fsid":"45c7911b-b69d-4e68-907b-fbfeb780aa3b","created":"2024-03-23T14:56:34.367633+0000","modified":"2024-03-23T14:57:42.610096+0000","last_up_change":"2024-03-23T14:56:46.297403+0000","last_in_change":"2024-03-23T14:56:36.629884+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T14:56:47.186157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-03-23T14:57:39.290666+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-03-23T14:57:39.934337+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"29","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"80abdc62-bd0d-4e60-a554-7ed6fd67cf54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":984716690},{"type":"v1","addr":"172.21.15.164:6801","nonce":984716690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":984716690},{"type":"v1","addr":"172.21.15.164:6803","nonce":984716690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":984716690},{"type":"v1","addr":"172.21.15.164:6807","nonce":984716690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":984716690},{"type":"v1","addr":"172.21.15.164:6805","nonce":984716690}]},"public_addr":"172.21.15.164:6801/984716690","cluster_addr":"172.21.15.164:6803/984716690","heartbeat_back_addr":"172.21.15.164:6807/984716690","heartbeat_front_addr":"172.21.15.164:6805/984716690","state":["exists","up"]},{"osd":1,"uuid":"f9001b06-be4d-4f42-a0e6-bcef2afa0d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":1349657239},{"type":"v1","addr":"172.21.15.164:6817","nonce":1349657239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":1349657239},{"type":"v1","addr":"172.21.15.164:6819","nonce":1349657239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":1349657239},{"type":"v1","addr":"172.21.15.164:6823","nonce":1349657239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":1349657239},{"type":"v1","addr":"172.21.15.164:6821","nonce":1349657239}]},"public_addr":"172.21.15.164:6817/1349657239","cluster_addr":"172.21.15.164:6819/1349657239","heartbeat_back_addr":"172.21.15.164:6823/1349657239","heartbeat_front_addr":"172.21.15.164:6821/1349657239","state":["exists","up"]},{"osd":2,"uuid":"fa0aeb9f-8b54-4831-a188-3e361c7980d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":887861252},{"type":"v1","addr":"172.21.15.164:6809","nonce":887861252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":887861252},{"type":"v1","addr":"172.21.15.164:6811","nonce":887861252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":887861252},{"type":"v1","addr":"172.21.15.164:6815","nonce":887861252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":887861252},{"type":"v1","addr":"172.21.15.164:6813","nonce":887861252}]},"public_addr":"172.21.15.164:6809/887861252","cluster_addr":"172.21.15.164:6811/887861252","heartbeat_back_addr":"172.21.15.164:6815/887861252","heartbeat_front_addr":"172.21.15.164:6813/887861252","state":["exists","up"]},{"osd":3,"uuid":"7e18c957-3091-46d5-8e17-8f3354c1726c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":812540271},{"type":"v1","addr":"172.21.15.164:6825","nonce":812540271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":812540271},{"type":"v1","addr":"172.21.15.164:6827","nonce":812540271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":812540271},{"type":"v1","addr":"172.21.15.164:6831","nonce":812540271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":812540271},{"type":"v1","addr":"172.21.15.164:6829","nonce":812540271}]},"public_addr":"172.21.15.164:6825/812540271","cluster_addr":"172.21.15.164:6827/812540271","heartbeat_back_addr":"172.21.15.164:6831/812540271","heartbeat_front_addr":"172.21.15.164:6829/812540271","state":["exists","up"]},{"osd":4,"uuid":"8e83f315-42bf-4471-9ce8-d81948da0f9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":2656857421},{"type":"v1","addr":"172.21.15.173:6825","nonce":2656857421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2656857421},{"type":"v1","addr":"172.21.15.173:6827","nonce":2656857421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2656857421},{"type":"v1","addr":"172.21.15.173:6831","nonce":2656857421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":2656857421},{"type":"v1","addr":"172.21.15.173:6829","nonce":2656857421}]},"public_addr":"172.21.15.173:6825/2656857421","cluster_addr":"172.21.15.173:6827/2656857421","heartbeat_back_addr":"172.21.15.173:6831/2656857421","heartbeat_front_addr":"172.21.15.173:6829/2656857421","state":["exists","up"]},{"osd":5,"uuid":"a230c872-71f9-40ab-8999-defd0162d4ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":3676483185},{"type":"v1","addr":"172.21.15.173:6801","nonce":3676483185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":3676483185},{"type":"v1","addr":"172.21.15.173:6803","nonce":3676483185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":3676483185},{"type":"v1","addr":"172.21.15.173:6807","nonce":3676483185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":3676483185},{"type":"v1","addr":"172.21.15.173:6805","nonce":3676483185}]},"public_addr":"172.21.15.173:6801/3676483185","cluster_addr":"172.21.15.173:6803/3676483185","heartbeat_back_addr":"172.21.15.173:6807/3676483185","heartbeat_front_addr":"172.21.15.173:6805/3676483185","state":["exists","up"]},{"osd":6,"uuid":"6193bfd7-8432-4f7f-a117-a3fcee21af99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":3631684087},{"type":"v1","addr":"172.21.15.173:6809","nonce":3631684087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":3631684087},{"type":"v1","addr":"172.21.15.173:6811","nonce":3631684087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":3631684087},{"type":"v1","addr":"172.21.15.173:6815","nonce":3631684087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":3631684087},{"type":"v1","addr":"172.21.15.173:6813","nonce":3631684087}]},"public_addr":"172.21.15.173:6809/3631684087","cluster_addr":"172.21.15.173:6811/3631684087","heartbeat_back_addr":"172.21.15.173:6815/3631684087","heartbeat_front_addr":"172.21.15.173:6813/3631684087","state":["exists","up"]},{"osd":7,"uuid":"b0f1b80a-afc4-4538-83b9-f081d2dcb8cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":3261114948},{"type":"v1","addr":"172.21.15.173:6817","nonce":3261114948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":3261114948},{"type":"v1","addr":"172.21.15.173:6819","nonce":3261114948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":3261114948},{"type":"v1","addr":"172.21.15.173:6823","nonce":3261114948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":3261114948},{"type":"v1","addr":"172.21.15.173:6821","nonce":3261114948}]},"public_addr":"172.21.15.173:6817/3261114948","cluster_addr":"172.21.15.173:6819/3261114948","heartbeat_back_addr":"172.21.15.173:6823/3261114948","heartbeat_front_addr":"172.21.15.173:6821/3261114948","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.188390+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.159932+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.191101+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.181776+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.224365+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.177515+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.153631+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.139972+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T14:57:52.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.793+0000 7fe956aa6640 1 -- 172.21.15.164:0/3014893960 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fe9340672a0 msgr2=0x7fe934069760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:52.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.793+0000 7fe956aa6640 1 --2- 172.21.15.164:0/3014893960 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fe9340672a0 0x7fe934069760 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe950148290 tx=0x7fe944008040 comp rx=0 tx=0).stop 2024-03-23T14:57:52.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.793+0000 7fe956aa6640 1 -- 172.21.15.164:0/3014893960 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe95013bb80 msgr2=0x7fe9501dc960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:52.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.793+0000 7fe956aa6640 1 --2- 172.21.15.164:0/3014893960 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe95013bb80 0x7fe9501dc960 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fe93c01c840 tx=0x7fe940005800 comp rx=0 tx=0).stop 2024-03-23T14:57:52.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.793+0000 7fe956aa6640 1 -- 172.21.15.164:0/3014893960 shutdown_connections 2024-03-23T14:57:52.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.793+0000 7fe956aa6640 1 --2- 172.21.15.164:0/3014893960 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fe9340672a0 0x7fe934069760 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:52.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.793+0000 7fe956aa6640 1 --2- 172.21.15.164:0/3014893960 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe9501d6720 0x7fe9501d6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:52.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.793+0000 7fe956aa6640 1 --2- 172.21.15.164:0/3014893960 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe95013bb80 0x7fe9501dc960 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:52.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.793+0000 7fe956aa6640 1 --2- 172.21.15.164:0/3014893960 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe940003f00 0x7fe95013d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:52.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.793+0000 7fe956aa6640 1 -- 172.21.15.164:0/3014893960 >> 172.21.15.164:0/3014893960 conn(0x7fe95002c3c0 msgr2=0x7fe95014b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:52.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.793+0000 7fe956aa6640 1 -- 172.21.15.164:0/3014893960 shutdown_connections 2024-03-23T14:57:52.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.793+0000 7fe956aa6640 1 -- 172.21.15.164:0/3014893960 wait complete. 2024-03-23T14:57:52.812 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-23T14:56:47.186157+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-23T14:57:39.290666+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '28', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-03-23T14:57:39.934337+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '29', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-23T14:57:52.812 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-23T14:57:52.931 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.925+0000 7f896c2d4640 1 Processor -- start 2024-03-23T14:57:52.931 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.925+0000 7f896c2d4640 1 -- start start 2024-03-23T14:57:52.932 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.929+0000 7f896c2d4640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f89640b9ac0 0x7f896413f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:52.932 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.929+0000 7f896c2d4640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f896406f8e0 0x7f89641459b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:52.932 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.929+0000 7f896c2d4640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f8964145ef0 0x7f8964148810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:52.932 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.929+0000 7f896c2d4640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f89640b3b50 con 0x7f8964145ef0 2024-03-23T14:57:52.933 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.929+0000 7f896c2d4640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 -- 0x7f8964145240 con 0x7f89640b9ac0 2024-03-23T14:57:52.933 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.929+0000 7f896c2d4640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f8964070050 con 0x7f896406f8e0 2024-03-23T14:57:52.933 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.929+0000 7f896a871640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f8964145ef0 0x7f8964148810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:52.933 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.929+0000 7f896a871640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f8964145ef0 0x7f8964148810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:48588/0 (socket says 172.21.15.164:48588) 2024-03-23T14:57:52.933 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.929+0000 7f896a871640 1 -- 172.21.15.164:0/105329502 learned_addr learned my addr 172.21.15.164:0/105329502 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:52.933 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.929+0000 7f896a070640 1 --2- 172.21.15.164:0/105329502 >> v2:172.21.15.173:3300/0 conn(0x7f89640b9ac0 0x7f896413f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:52.933 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.929+0000 7f896986f640 1 --2- 172.21.15.164:0/105329502 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f896406f8e0 0x7f89641459b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:52.933 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.929+0000 7f896a871640 1 -- 172.21.15.164:0/105329502 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f896406f8e0 msgr2=0x7f89641459b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:52.933 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.929+0000 7f896a871640 1 --2- 172.21.15.164:0/105329502 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f896406f8e0 0x7f89641459b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:52.933 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.929+0000 7f896a871640 1 -- 172.21.15.164:0/105329502 >> v2:172.21.15.173:3300/0 conn(0x7f89640b9ac0 msgr2=0x7f896413f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:52.933 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.929+0000 7f896a871640 1 --2- 172.21.15.164:0/105329502 >> v2:172.21.15.173:3300/0 conn(0x7f89640b9ac0 0x7f896413f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:52.933 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.929+0000 7f896a871640 1 -- 172.21.15.164:0/105329502 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89640b9ea0 con 0x7f8964145ef0 2024-03-23T14:57:52.934 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.929+0000 7f896a871640 1 --2- 172.21.15.164:0/105329502 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8964145ef0 0x7f8964148810 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f89640ba090 tx=0x7f895c01c670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a575b6c710aa830 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:52.934 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.929+0000 7f896906e640 1 -- 172.21.15.164:0/105329502 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f895c0a6500 con 0x7f8964145ef0 2024-03-23T14:57:52.934 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.929+0000 7f896906e640 1 -- 172.21.15.164:0/105329502 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f895c0a6660 con 0x7f8964145ef0 2024-03-23T14:57:52.934 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.929+0000 7f896906e640 1 -- 172.21.15.164:0/105329502 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f895c0a69f0 con 0x7f8964145ef0 2024-03-23T14:57:52.935 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.929+0000 7f896c2d4640 1 -- 172.21.15.164:0/105329502 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8964145ef0 msgr2=0x7f8964148810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:52.935 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.929+0000 7f896c2d4640 1 --2- 172.21.15.164:0/105329502 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8964145ef0 0x7f8964148810 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f89640ba090 tx=0x7f895c01c670 comp rx=0 tx=0).stop 2024-03-23T14:57:52.936 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.929+0000 7f896c2d4640 1 -- 172.21.15.164:0/105329502 shutdown_connections 2024-03-23T14:57:52.936 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.929+0000 7f896c2d4640 1 --2- 172.21.15.164:0/105329502 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f896406f8e0 0x7f89641459b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:52.936 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.929+0000 7f896c2d4640 1 --2- 172.21.15.164:0/105329502 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8964145ef0 0x7f8964148810 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:52.936 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.929+0000 7f896c2d4640 1 --2- 172.21.15.164:0/105329502 >> v2:172.21.15.173:3300/0 conn(0x7f89640b9ac0 0x7f896413f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:52.936 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.929+0000 7f896c2d4640 1 -- 172.21.15.164:0/105329502 >> 172.21.15.164:0/105329502 conn(0x7f896406b930 msgr2=0x7f89640b9150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:52.936 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.929+0000 7f896c2d4640 1 -- 172.21.15.164:0/105329502 shutdown_connections 2024-03-23T14:57:52.936 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.929+0000 7f896c2d4640 1 -- 172.21.15.164:0/105329502 wait complete. 2024-03-23T14:57:52.938 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.933+0000 7f896c2d4640 1 Processor -- start 2024-03-23T14:57:52.938 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.933+0000 7f896c2d4640 1 -- start start 2024-03-23T14:57:52.938 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.933+0000 7f896c2d4640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f89640b9ac0 0x7f89641e0570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:52.938 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.933+0000 7f896c2d4640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8964145ef0 0x7f89640725f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:52.938 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.933+0000 7f896c2d4640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f896413d3d0 0x7f89641417c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:52.938 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.933+0000 7f896c2d4640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f89641dfb60 con 0x7f8964145ef0 2024-03-23T14:57:52.938 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.933+0000 7f896c2d4640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f89641e0ab0 con 0x7f89640b9ac0 2024-03-23T14:57:52.938 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.933+0000 7f896c2d4640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f896413f210 con 0x7f896413d3d0 2024-03-23T14:57:52.938 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.933+0000 7f896986f640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8964145ef0 0x7f89640725f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:52.939 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.933+0000 7f896986f640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8964145ef0 0x7f89640725f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:48590/0 (socket says 172.21.15.164:48590) 2024-03-23T14:57:52.939 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.933+0000 7f896986f640 1 -- 172.21.15.164:0/1539483546 learned_addr learned my addr 172.21.15.164:0/1539483546 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:52.939 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.933+0000 7f896a070640 1 --2- 172.21.15.164:0/1539483546 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f89640b9ac0 0x7f89641e0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:52.939 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.933+0000 7f896a871640 1 --2- 172.21.15.164:0/1539483546 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f896413d3d0 0x7f89641417c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:52.939 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.933+0000 7f896986f640 1 -- 172.21.15.164:0/1539483546 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f896413d3d0 msgr2=0x7f89641417c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:52.939 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.933+0000 7f896986f640 1 --2- 172.21.15.164:0/1539483546 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f896413d3d0 0x7f89641417c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:52.939 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.933+0000 7f896986f640 1 -- 172.21.15.164:0/1539483546 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f89640b9ac0 msgr2=0x7f89641e0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:52.939 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.933+0000 7f896986f640 1 --2- 172.21.15.164:0/1539483546 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f89640b9ac0 0x7f89641e0570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:52.939 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.933+0000 7f896986f640 1 -- 172.21.15.164:0/1539483546 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f895c009b30 con 0x7f8964145ef0 2024-03-23T14:57:52.939 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.933+0000 7f896986f640 1 --2- 172.21.15.164:0/1539483546 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8964145ef0 0x7f89640725f0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f895400d800 tx=0x7f895400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:52.939 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.933+0000 7f895affd640 1 -- 172.21.15.164:0/1539483546 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89540096c0 con 0x7f8964145ef0 2024-03-23T14:57:52.939 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.933+0000 7f895affd640 1 -- 172.21.15.164:0/1539483546 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8954009820 con 0x7f8964145ef0 2024-03-23T14:57:52.939 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.933+0000 7f895affd640 1 -- 172.21.15.164:0/1539483546 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8954009bd0 con 0x7f8964145ef0 2024-03-23T14:57:52.939 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.933+0000 7f896c2d4640 1 -- 172.21.15.164:0/1539483546 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89641dfd70 con 0x7f8964145ef0 2024-03-23T14:57:52.939 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.933+0000 7f896c2d4640 1 -- 172.21.15.164:0/1539483546 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8964140060 con 0x7f8964145ef0 2024-03-23T14:57:52.940 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.933+0000 7f896c2d4640 1 -- 172.21.15.164:0/1539483546 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89640690a0 con 0x7f8964145ef0 2024-03-23T14:57:52.940 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.933+0000 7f895affd640 1 -- 172.21.15.164:0/1539483546 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7f8954009d30 con 0x7f8964145ef0 2024-03-23T14:57:52.940 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.937+0000 7f895affd640 1 --2- 172.21.15.164:0/1539483546 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f89480672a0 0x7f8948069760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:52.941 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.937+0000 7f896a070640 1 --2- 172.21.15.164:0/1539483546 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f89480672a0 0x7f8948069760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:52.941 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.937+0000 7f895affd640 1 -- 172.21.15.164:0/1539483546 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5709+0+0 (secure 0 0 0) 0x7f8954085750 con 0x7f8964145ef0 2024-03-23T14:57:52.941 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.937+0000 7f896a070640 1 --2- 172.21.15.164:0/1539483546 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f89480672a0 0x7f8948069760 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f89641e4190 tx=0x7f8960008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:52.943 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:52.941+0000 7f895affd640 1 -- 172.21.15.164:0/1539483546 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7f8954052390 con 0x7f8964145ef0 2024-03-23T14:57:53.091 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.085+0000 7f896c2d4640 1 -- 172.21.15.164:0/1539483546 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f896414f080 con 0x7f8964145ef0 2024-03-23T14:57:53.091 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.085+0000 7f895affd640 1 -- 172.21.15.164:0/1539483546 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) ==== 93+0+10 (secure 0 0 0) 0x7f895400b440 con 0x7f8964145ef0 2024-03-23T14:57:53.091 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 1 2024-03-23T14:57:53.095 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.089+0000 7f896c2d4640 1 -- 172.21.15.164:0/1539483546 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f89480672a0 msgr2=0x7f8948069760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:53.095 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.089+0000 7f896c2d4640 1 --2- 172.21.15.164:0/1539483546 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f89480672a0 0x7f8948069760 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f89641e4190 tx=0x7f8960008040 comp rx=0 tx=0).stop 2024-03-23T14:57:53.095 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.089+0000 7f896c2d4640 1 -- 172.21.15.164:0/1539483546 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8964145ef0 msgr2=0x7f89640725f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:53.095 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.089+0000 7f896c2d4640 1 --2- 172.21.15.164:0/1539483546 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8964145ef0 0x7f89640725f0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f895400d800 tx=0x7f895400dcc0 comp rx=0 tx=0).stop 2024-03-23T14:57:53.095 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.089+0000 7f896c2d4640 1 -- 172.21.15.164:0/1539483546 shutdown_connections 2024-03-23T14:57:53.095 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.089+0000 7f896c2d4640 1 --2- 172.21.15.164:0/1539483546 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f896413d3d0 0x7f89641417c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:53.095 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.089+0000 7f896c2d4640 1 --2- 172.21.15.164:0/1539483546 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8964145ef0 0x7f89640725f0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:53.095 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.089+0000 7f896c2d4640 1 --2- 172.21.15.164:0/1539483546 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f89480672a0 0x7f8948069760 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:53.095 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.089+0000 7f896c2d4640 1 --2- 172.21.15.164:0/1539483546 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f89640b9ac0 0x7f89641e0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:53.095 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.089+0000 7f896c2d4640 1 -- 172.21.15.164:0/1539483546 >> 172.21.15.164:0/1539483546 conn(0x7f896406b930 msgr2=0x7f8964139850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:53.095 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.089+0000 7f896c2d4640 1 -- 172.21.15.164:0/1539483546 shutdown_connections 2024-03-23T14:57:53.095 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.089+0000 7f896c2d4640 1 -- 172.21.15.164:0/1539483546 wait complete. 2024-03-23T14:57:53.116 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-23T14:57:53.263 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.257+0000 7f3d929f6640 1 Processor -- start 2024-03-23T14:57:53.264 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.257+0000 7f3d929f6640 1 -- start start 2024-03-23T14:57:53.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.261+0000 7f3d929f6640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3d8c0b56f0 0x7f3d8c14a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:53.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.261+0000 7f3d929f6640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f3d8c0b4a70 0x7f3d8c140f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:53.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.261+0000 7f3d929f6640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f3d8c141470 0x7f3d8c0b75f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:53.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.261+0000 7f3d929f6640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f3d8c0b5fa0 con 0x7f3d8c0b4a70 2024-03-23T14:57:53.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.261+0000 7f3d929f6640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 -- 0x7f3d8c0b4300 con 0x7f3d8c141470 2024-03-23T14:57:53.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.261+0000 7f3d929f6640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f3d8c0b45e0 con 0x7f3d8c0b56f0 2024-03-23T14:57:53.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.261+0000 7f3d83fff640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f3d8c0b4a70 0x7f3d8c140f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:53.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.261+0000 7f3d83fff640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f3d8c0b4a70 0x7f3d8c140f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:48592/0 (socket says 172.21.15.164:48592) 2024-03-23T14:57:53.265 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.261+0000 7f3d83fff640 1 -- 172.21.15.164:0/1443939899 learned_addr learned my addr 172.21.15.164:0/1443939899 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:53.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.261+0000 7f3d90f93640 1 --2- 172.21.15.164:0/1443939899 >> v2:172.21.15.173:3300/0 conn(0x7f3d8c141470 0x7f3d8c0b75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:53.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.261+0000 7f3d83fff640 1 -- 172.21.15.164:0/1443939899 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3d8c0b56f0 msgr2=0x7f3d8c14a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:53.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.261+0000 7f3d83fff640 1 --2- 172.21.15.164:0/1443939899 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3d8c0b56f0 0x7f3d8c14a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:53.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.261+0000 7f3d83fff640 1 -- 172.21.15.164:0/1443939899 >> v2:172.21.15.173:3300/0 conn(0x7f3d8c141470 msgr2=0x7f3d8c0b75f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:53.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.261+0000 7f3d83fff640 1 --2- 172.21.15.164:0/1443939899 >> v2:172.21.15.173:3300/0 conn(0x7f3d8c141470 0x7f3d8c0b75f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:53.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.261+0000 7f3d83fff640 1 -- 172.21.15.164:0/1443939899 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d8c14ade0 con 0x7f3d8c0b4a70 2024-03-23T14:57:53.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.261+0000 7f3d90f93640 1 --2- 172.21.15.164:0/1443939899 >> v2:172.21.15.173:3300/0 conn(0x7f3d8c141470 0x7f3d8c0b75f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T14:57:53.267 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.261+0000 7f3d83fff640 1 --2- 172.21.15.164:0/1443939899 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3d8c0b4a70 0x7f3d8c140f30 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f3d8c0bab00 tx=0x7f3d7401c240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee4f89b8967aff6a server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:53.267 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.261+0000 7f3d8b7fe640 1 -- 172.21.15.164:0/1443939899 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d740ab600 con 0x7f3d8c0b4a70 2024-03-23T14:57:53.267 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.261+0000 7f3d8b7fe640 1 -- 172.21.15.164:0/1443939899 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d740a7050 con 0x7f3d8c0b4a70 2024-03-23T14:57:53.267 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.261+0000 7f3d8b7fe640 1 -- 172.21.15.164:0/1443939899 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d740ab980 con 0x7f3d8c0b4a70 2024-03-23T14:57:53.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.265+0000 7f3d929f6640 1 -- 172.21.15.164:0/1443939899 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3d8c0b4a70 msgr2=0x7f3d8c140f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:53.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.265+0000 7f3d929f6640 1 --2- 172.21.15.164:0/1443939899 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3d8c0b4a70 0x7f3d8c140f30 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f3d8c0bab00 tx=0x7f3d7401c240 comp rx=0 tx=0).stop 2024-03-23T14:57:53.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.265+0000 7f3d929f6640 1 -- 172.21.15.164:0/1443939899 shutdown_connections 2024-03-23T14:57:53.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.265+0000 7f3d929f6640 1 --2- 172.21.15.164:0/1443939899 >> v2:172.21.15.173:3300/0 conn(0x7f3d8c141470 0x7f3d8c0b75f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:53.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.265+0000 7f3d929f6640 1 --2- 172.21.15.164:0/1443939899 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3d8c0b4a70 0x7f3d8c140f30 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:53.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.265+0000 7f3d929f6640 1 --2- 172.21.15.164:0/1443939899 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3d8c0b56f0 0x7f3d8c14a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:53.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.265+0000 7f3d929f6640 1 -- 172.21.15.164:0/1443939899 >> 172.21.15.164:0/1443939899 conn(0x7f3d8c02c3c0 msgr2=0x7f3d8c02c7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:53.269 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.265+0000 7f3d929f6640 1 -- 172.21.15.164:0/1443939899 shutdown_connections 2024-03-23T14:57:53.269 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.265+0000 7f3d929f6640 1 -- 172.21.15.164:0/1443939899 wait complete. 2024-03-23T14:57:53.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.265+0000 7f3d929f6640 1 Processor -- start 2024-03-23T14:57:53.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.265+0000 7f3d929f6640 1 -- start start 2024-03-23T14:57:53.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.265+0000 7f3d929f6640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3d8c0b4a70 0x7f3d8c1472f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:53.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.265+0000 7f3d929f6640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3d8c0b56f0 0x7f3d8c149590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:53.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.265+0000 7f3d929f6640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3d8c141470 0x7f3d8c146410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:53.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.265+0000 7f3d929f6640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f3d8c141ed0 con 0x7f3d8c141470 2024-03-23T14:57:53.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.265+0000 7f3d929f6640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f3d8c1e4090 con 0x7f3d8c0b56f0 2024-03-23T14:57:53.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.265+0000 7f3d929f6640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f3d8c145c10 con 0x7f3d8c0b4a70 2024-03-23T14:57:53.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.265+0000 7f3d90f93640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3d8c141470 0x7f3d8c146410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:53.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.265+0000 7f3d90f93640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3d8c141470 0x7f3d8c146410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:48606/0 (socket says 172.21.15.164:48606) 2024-03-23T14:57:53.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.265+0000 7f3d90f93640 1 -- 172.21.15.164:0/2368612535 learned_addr learned my addr 172.21.15.164:0/2368612535 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:53.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.265+0000 7f3d83fff640 1 --2- 172.21.15.164:0/2368612535 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3d8c0b56f0 0x7f3d8c149590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:53.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.265+0000 7f3d8bfff640 1 --2- 172.21.15.164:0/2368612535 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3d8c0b4a70 0x7f3d8c1472f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:53.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.265+0000 7f3d90f93640 1 -- 172.21.15.164:0/2368612535 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3d8c0b4a70 msgr2=0x7f3d8c1472f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:53.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.265+0000 7f3d90f93640 1 --2- 172.21.15.164:0/2368612535 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3d8c0b4a70 0x7f3d8c1472f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:53.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.265+0000 7f3d90f93640 1 -- 172.21.15.164:0/2368612535 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3d8c0b56f0 msgr2=0x7f3d8c149590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:53.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.265+0000 7f3d90f93640 1 --2- 172.21.15.164:0/2368612535 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3d8c0b56f0 0x7f3d8c149590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:53.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.265+0000 7f3d90f93640 1 -- 172.21.15.164:0/2368612535 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d740094d0 con 0x7f3d8c141470 2024-03-23T14:57:53.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.265+0000 7f3d8bfff640 1 --2- 172.21.15.164:0/2368612535 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3d8c0b4a70 0x7f3d8c1472f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T14:57:53.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.265+0000 7f3d83fff640 1 --2- 172.21.15.164:0/2368612535 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3d8c0b56f0 0x7f3d8c149590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T14:57:53.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.265+0000 7f3d90f93640 1 --2- 172.21.15.164:0/2368612535 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3d8c141470 0x7f3d8c146410 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f3d8400cd80 tx=0x7f3d8400a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:53.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.265+0000 7f3d897fa640 1 -- 172.21.15.164:0/2368612535 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d8400f680 con 0x7f3d8c141470 2024-03-23T14:57:53.272 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.265+0000 7f3d897fa640 1 -- 172.21.15.164:0/2368612535 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d8400f800 con 0x7f3d8c141470 2024-03-23T14:57:53.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.265+0000 7f3d929f6640 1 -- 172.21.15.164:0/2368612535 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d8c145ef0 con 0x7f3d8c141470 2024-03-23T14:57:53.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.265+0000 7f3d897fa640 1 -- 172.21.15.164:0/2368612535 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d8400fbb0 con 0x7f3d8c141470 2024-03-23T14:57:53.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.265+0000 7f3d929f6640 1 -- 172.21.15.164:0/2368612535 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d8c146950 con 0x7f3d8c141470 2024-03-23T14:57:53.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.269+0000 7f3d929f6640 1 -- 172.21.15.164:0/2368612535 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d8c146130 con 0x7f3d8c141470 2024-03-23T14:57:53.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.269+0000 7f3d897fa640 1 -- 172.21.15.164:0/2368612535 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7f3d8400b040 con 0x7f3d8c141470 2024-03-23T14:57:53.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.269+0000 7f3d897fa640 1 --2- 172.21.15.164:0/2368612535 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f3d6c0672d0 0x7f3d6c069790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:53.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.269+0000 7f3d8bfff640 1 --2- 172.21.15.164:0/2368612535 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f3d6c0672d0 0x7f3d6c069790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:53.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.269+0000 7f3d897fa640 1 -- 172.21.15.164:0/2368612535 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5709+0+0 (secure 0 0 0) 0x7f3d84085990 con 0x7f3d8c141470 2024-03-23T14:57:53.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.269+0000 7f3d8bfff640 1 --2- 172.21.15.164:0/2368612535 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f3d6c0672d0 0x7f3d6c069790 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f3d7c00a8b0 tx=0x7f3d7c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:53.276 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.273+0000 7f3d897fa640 1 -- 172.21.15.164:0/2368612535 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7f3d8c146130 con 0x7f3d8c141470 2024-03-23T14:57:53.425 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.421+0000 7f3d929f6640 1 -- 172.21.15.164:0/2368612535 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f3d8c146130 con 0x7f3d8c141470 2024-03-23T14:57:53.426 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.421+0000 7f3d897fa640 1 -- 172.21.15.164:0/2368612535 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v29) ==== 104+0+11 (secure 0 0 0) 0x7f3d8c146130 con 0x7f3d8c141470 2024-03-23T14:57:53.426 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 32 2024-03-23T14:57:53.428 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.425+0000 7f3d929f6640 1 -- 172.21.15.164:0/2368612535 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f3d6c0672d0 msgr2=0x7f3d6c069790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:53.428 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.425+0000 7f3d929f6640 1 --2- 172.21.15.164:0/2368612535 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f3d6c0672d0 0x7f3d6c069790 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f3d7c00a8b0 tx=0x7f3d7c008040 comp rx=0 tx=0).stop 2024-03-23T14:57:53.428 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.425+0000 7f3d929f6640 1 -- 172.21.15.164:0/2368612535 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3d8c141470 msgr2=0x7f3d8c146410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:53.428 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.425+0000 7f3d929f6640 1 --2- 172.21.15.164:0/2368612535 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3d8c141470 0x7f3d8c146410 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f3d8400cd80 tx=0x7f3d8400a650 comp rx=0 tx=0).stop 2024-03-23T14:57:53.428 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.425+0000 7f3d929f6640 1 -- 172.21.15.164:0/2368612535 shutdown_connections 2024-03-23T14:57:53.428 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.425+0000 7f3d929f6640 1 --2- 172.21.15.164:0/2368612535 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3d8c141470 0x7f3d8c146410 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:53.428 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.425+0000 7f3d929f6640 1 --2- 172.21.15.164:0/2368612535 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f3d6c0672d0 0x7f3d6c069790 secure :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f3d7c00a8b0 tx=0x7f3d7c008040 comp rx=0 tx=0).stop 2024-03-23T14:57:53.428 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.425+0000 7f3d929f6640 1 --2- 172.21.15.164:0/2368612535 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3d8c0b56f0 0x7f3d8c149590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:53.428 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.425+0000 7f3d929f6640 1 --2- 172.21.15.164:0/2368612535 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3d8c0b4a70 0x7f3d8c1472f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:53.428 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.425+0000 7f3d929f6640 1 -- 172.21.15.164:0/2368612535 >> 172.21.15.164:0/2368612535 conn(0x7f3d8c02c3c0 msgr2=0x7f3d8c14b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:53.429 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.425+0000 7f3d929f6640 1 -- 172.21.15.164:0/2368612535 shutdown_connections 2024-03-23T14:57:53.429 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.425+0000 7f3d929f6640 1 -- 172.21.15.164:0/2368612535 wait complete. 2024-03-23T14:57:53.451 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-23T14:57:53.636 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.633+0000 7fc661076640 1 Processor -- start 2024-03-23T14:57:53.636 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.633+0000 7fc661076640 1 -- start start 2024-03-23T14:57:53.637 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.633+0000 7fc661076640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fc65c0b9ac0 0x7fc65c0ae920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:53.637 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.633+0000 7fc661076640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc65c0b9ea0 con 0x7fc65c144cf0 2024-03-23T14:57:53.637 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.633+0000 7fc661076640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc65c0b53e0 con 0x7fc65c0adc70 2024-03-23T14:57:53.637 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.633+0000 7fc661076640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fc65c0af400 con 0x7fc65c0b9ac0 2024-03-23T14:57:53.637 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.633+0000 7fc65ad9d640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fc65c0b9ac0 0x7fc65c0ae920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:53.637 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.633+0000 7fc65b59e640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fc65c144cf0 0x7fc65c1477b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:43066/0 (socket says 172.21.15.164:43066) 2024-03-23T14:57:53.637 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.633+0000 7fc65b59e640 1 -- 172.21.15.164:0/1154228979 learned_addr learned my addr 172.21.15.164:0/1154228979 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-03-23T14:57:53.637 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.633+0000 7fc65ad9d640 1 -- 172.21.15.164:0/1154228979 >> v1:172.21.15.173:6789/0 conn(0x7fc65c0adc70 legacy=0x7fc65c1445e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:53.638 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.633+0000 7fc65ad9d640 1 -- 172.21.15.164:0/1154228979 >> v1:172.21.15.164:6789/0 conn(0x7fc65c144cf0 legacy=0x7fc65c1477b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:53.638 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.633+0000 7fc65ad9d640 1 -- 172.21.15.164:0/1154228979 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc65c0b56c0 con 0x7fc65c0b9ac0 2024-03-23T14:57:53.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.633+0000 7fc65ad9d640 1 --2- 172.21.15.164:0/1154228979 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc65c0b9ac0 0x7fc65c0ae920 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fc64c0097c0 tx=0x7fc64c01c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d66c83590297ef29 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:53.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.633+0000 7fc65a59c640 1 -- 172.21.15.164:0/1154228979 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc64c0ac620 con 0x7fc65c0b9ac0 2024-03-23T14:57:53.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.633+0000 7fc65a59c640 1 -- 172.21.15.164:0/1154228979 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc64c0ac780 con 0x7fc65c0b9ac0 2024-03-23T14:57:53.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.633+0000 7fc65a59c640 1 -- 172.21.15.164:0/1154228979 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc64c0acb10 con 0x7fc65c0b9ac0 2024-03-23T14:57:53.640 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.637+0000 7fc661076640 1 -- 172.21.15.164:0/1154228979 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc65c0b9ac0 msgr2=0x7fc65c0ae920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:53.640 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.637+0000 7fc661076640 1 --2- 172.21.15.164:0/1154228979 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc65c0b9ac0 0x7fc65c0ae920 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fc64c0097c0 tx=0x7fc64c01c3a0 comp rx=0 tx=0).stop 2024-03-23T14:57:53.640 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.637+0000 7fc661076640 1 -- 172.21.15.164:0/1154228979 shutdown_connections 2024-03-23T14:57:53.640 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.637+0000 7fc661076640 1 --2- 172.21.15.164:0/1154228979 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc65c0b9ac0 0x7fc65c0ae920 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:53.640 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.637+0000 7fc661076640 1 -- 172.21.15.164:0/1154228979 >> 172.21.15.164:0/1154228979 conn(0x7fc65c06b930 msgr2=0x7fc65c0b9150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:53.641 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.637+0000 7fc661076640 1 -- 172.21.15.164:0/1154228979 shutdown_connections 2024-03-23T14:57:53.641 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.637+0000 7fc661076640 1 -- 172.21.15.164:0/1154228979 wait complete. 2024-03-23T14:57:53.641 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.637+0000 7fc661076640 1 Processor -- start 2024-03-23T14:57:53.641 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.637+0000 7fc661076640 1 -- start start 2024-03-23T14:57:53.643 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.637+0000 7fc661076640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc65c0adc70 0x7fc65c13cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:53.643 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.637+0000 7fc661076640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc65c0b9ac0 0x7fc65c071f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:53.643 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.637+0000 7fc661076640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc65c144cf0 0x7fc65c1e3f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:53.643 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.637+0000 7fc661076640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc65c13df30 con 0x7fc65c0adc70 2024-03-23T14:57:53.643 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.637+0000 7fc661076640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fc65c1e4460 con 0x7fc65c144cf0 2024-03-23T14:57:53.643 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.637+0000 7fc661076640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fc65c1e3720 con 0x7fc65c0b9ac0 2024-03-23T14:57:53.643 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.637+0000 7fc65ad9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc65c0adc70 0x7fc65c13cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:53.643 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.637+0000 7fc65ad9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc65c0adc70 0x7fc65c13cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:48624/0 (socket says 172.21.15.164:48624) 2024-03-23T14:57:53.644 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.637+0000 7fc65ad9d640 1 -- 172.21.15.164:0/1736313382 learned_addr learned my addr 172.21.15.164:0/1736313382 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:53.644 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.637+0000 7fc653fff640 1 --2- 172.21.15.164:0/1736313382 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc65c0b9ac0 0x7fc65c071f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:53.644 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.637+0000 7fc65b59e640 1 --2- 172.21.15.164:0/1736313382 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc65c144cf0 0x7fc65c1e3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:53.644 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.637+0000 7fc65ad9d640 1 -- 172.21.15.164:0/1736313382 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc65c0b9ac0 msgr2=0x7fc65c071f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:53.644 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.637+0000 7fc65ad9d640 1 --2- 172.21.15.164:0/1736313382 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc65c0b9ac0 0x7fc65c071f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:53.644 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.637+0000 7fc65ad9d640 1 -- 172.21.15.164:0/1736313382 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc65c144cf0 msgr2=0x7fc65c1e3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:53.644 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.637+0000 7fc65ad9d640 1 --2- 172.21.15.164:0/1736313382 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc65c144cf0 0x7fc65c1e3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:53.644 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.637+0000 7fc65ad9d640 1 -- 172.21.15.164:0/1736313382 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc64c009420 con 0x7fc65c0adc70 2024-03-23T14:57:53.644 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.637+0000 7fc65ad9d640 1 --2- 172.21.15.164:0/1736313382 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc65c0adc70 0x7fc65c13cb00 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fc64c018fb0 tx=0x7fc64c01abe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:53.644 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.637+0000 7fc6537fe640 1 -- 172.21.15.164:0/1736313382 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc64c0b8450 con 0x7fc65c0adc70 2024-03-23T14:57:53.644 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.637+0000 7fc6537fe640 1 -- 172.21.15.164:0/1736313382 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc64c0b85b0 con 0x7fc65c0adc70 2024-03-23T14:57:53.644 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.637+0000 7fc661076640 1 -- 172.21.15.164:0/1736313382 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc65c1e3a00 con 0x7fc65c0adc70 2024-03-23T14:57:53.644 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.637+0000 7fc6537fe640 1 -- 172.21.15.164:0/1736313382 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc64c0b8960 con 0x7fc65c0adc70 2024-03-23T14:57:53.644 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.637+0000 7fc661076640 1 -- 172.21.15.164:0/1736313382 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc65c143dd0 con 0x7fc65c0adc70 2024-03-23T14:57:53.645 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.641+0000 7fc661076640 1 -- 172.21.15.164:0/1736313382 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc65c02c9b0 con 0x7fc65c0adc70 2024-03-23T14:57:53.645 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.641+0000 7fc6537fe640 1 -- 172.21.15.164:0/1736313382 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7fc64c0aa070 con 0x7fc65c0adc70 2024-03-23T14:57:53.645 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.641+0000 7fc6537fe640 1 --2- 172.21.15.164:0/1736313382 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fc63c0672a0 0x7fc63c069760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:53.645 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.641+0000 7fc653fff640 1 --2- 172.21.15.164:0/1736313382 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fc63c0672a0 0x7fc63c069760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:53.646 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.641+0000 7fc6537fe640 1 -- 172.21.15.164:0/1736313382 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5709+0+0 (secure 0 0 0) 0x7fc64c11f780 con 0x7fc65c0adc70 2024-03-23T14:57:53.646 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.641+0000 7fc653fff640 1 --2- 172.21.15.164:0/1736313382 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fc63c0672a0 0x7fc63c069760 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc65c0b1ee0 tx=0x7fc644009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:53.649 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.645+0000 7fc6537fe640 1 -- 172.21.15.164:0/1736313382 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7fc64c0ec4e0 con 0x7fc65c0adc70 2024-03-23T14:57:53.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.793+0000 7fc661076640 1 -- 172.21.15.164:0/1736313382 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc65c1e3580 con 0x7fc65c0adc70 2024-03-23T14:57:53.798 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.793+0000 7fc6537fe640 1 -- 172.21.15.164:0/1736313382 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v29) ==== 100+0+11 (secure 0 0 0) 0x7fc64c0bc020 con 0x7fc65c0adc70 2024-03-23T14:57:53.798 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 64 2024-03-23T14:57:53.801 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.797+0000 7fc661076640 1 -- 172.21.15.164:0/1736313382 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fc63c0672a0 msgr2=0x7fc63c069760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:53.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.797+0000 7fc661076640 1 --2- 172.21.15.164:0/1736313382 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fc63c0672a0 0x7fc63c069760 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc65c0b1ee0 tx=0x7fc644009040 comp rx=0 tx=0).stop 2024-03-23T14:57:53.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.797+0000 7fc661076640 1 -- 172.21.15.164:0/1736313382 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc65c0adc70 msgr2=0x7fc65c13cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:53.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.797+0000 7fc661076640 1 --2- 172.21.15.164:0/1736313382 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc65c0adc70 0x7fc65c13cb00 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fc64c018fb0 tx=0x7fc64c01abe0 comp rx=0 tx=0).stop 2024-03-23T14:57:53.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.797+0000 7fc661076640 1 -- 172.21.15.164:0/1736313382 shutdown_connections 2024-03-23T14:57:53.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.797+0000 7fc661076640 1 --2- 172.21.15.164:0/1736313382 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fc63c0672a0 0x7fc63c069760 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:53.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.797+0000 7fc661076640 1 --2- 172.21.15.164:0/1736313382 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc65c144cf0 0x7fc65c1e3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:53.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.797+0000 7fc661076640 1 --2- 172.21.15.164:0/1736313382 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc65c0b9ac0 0x7fc65c071f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:53.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.797+0000 7fc661076640 1 --2- 172.21.15.164:0/1736313382 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc65c0adc70 0x7fc65c13cb00 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:53.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.797+0000 7fc661076640 1 -- 172.21.15.164:0/1736313382 >> 172.21.15.164:0/1736313382 conn(0x7fc65c06b930 msgr2=0x7fc65c1443b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:53.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.797+0000 7fc661076640 1 -- 172.21.15.164:0/1736313382 shutdown_connections 2024-03-23T14:57:53.802 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.797+0000 7fc661076640 1 -- 172.21.15.164:0/1736313382 wait complete. 2024-03-23T14:57:53.818 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T14:57:53.928 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.925+0000 7f32190fc640 1 Processor -- start 2024-03-23T14:57:53.928 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.925+0000 7f32190fc640 1 -- start start 2024-03-23T14:57:53.930 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.925+0000 7f32190fc640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f321406f8e0 0x7f32141459b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:53.930 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.925+0000 7f32190fc640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f3214145ef0 0x7f321414fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:53.930 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.925+0000 7f32190fc640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3214145240 con 0x7f32140b9ac0 2024-03-23T14:57:53.930 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.925+0000 7f32190fc640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f32140b3b50 con 0x7f321406f8e0 2024-03-23T14:57:53.930 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.925+0000 7f32190fc640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 -- 0x7f3214070050 con 0x7f3214145ef0 2024-03-23T14:57:53.930 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.925+0000 7f321259c640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f321406f8e0 0x7f32141459b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:53.930 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.925+0000 7f3212d9d640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f32140b9ac0 0x7f321413f240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:43072/0 (socket says 172.21.15.164:43072) 2024-03-23T14:57:53.930 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.925+0000 7f3212d9d640 1 -- 172.21.15.164:0/2339787470 learned_addr learned my addr 172.21.15.164:0/2339787470 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-03-23T14:57:53.930 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.925+0000 7f321259c640 1 -- 172.21.15.164:0/2339787470 >> v2:172.21.15.173:3300/0 conn(0x7f3214145ef0 msgr2=0x7f321414fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:53.930 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.925+0000 7f321259c640 1 --2- 172.21.15.164:0/2339787470 >> v2:172.21.15.173:3300/0 conn(0x7f3214145ef0 0x7f321414fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:53.930 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.925+0000 7f321259c640 1 -- 172.21.15.164:0/2339787470 >> v1:172.21.15.164:6789/0 conn(0x7f32140b9ac0 legacy=0x7f321413f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:53.930 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.925+0000 7f3211d9b640 1 -- 172.21.15.164:0/2339787470 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3035661476 0 0) 0x7f3214145240 con 0x7f32140b9ac0 2024-03-23T14:57:53.930 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.925+0000 7f321259c640 1 -- 172.21.15.164:0/2339787470 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f321413f950 con 0x7f321406f8e0 2024-03-23T14:57:53.932 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.925+0000 7f321259c640 1 --2- 172.21.15.164:0/2339787470 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f321406f8e0 0x7f32141459b0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f3214073710 tx=0x7f320801c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cb0c32b5ef964cf server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:53.932 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.925+0000 7f3211d9b640 1 -- 172.21.15.164:0/2339787470 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32080ab5f0 con 0x7f321406f8e0 2024-03-23T14:57:53.932 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.925+0000 7f3211d9b640 1 -- 172.21.15.164:0/2339787470 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32080ab750 con 0x7f321406f8e0 2024-03-23T14:57:53.932 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.925+0000 7f3211d9b640 1 -- 172.21.15.164:0/2339787470 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32080abae0 con 0x7f321406f8e0 2024-03-23T14:57:53.933 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.929+0000 7f32190fc640 1 -- 172.21.15.164:0/2339787470 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f321406f8e0 msgr2=0x7f32141459b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:53.933 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.929+0000 7f32190fc640 1 --2- 172.21.15.164:0/2339787470 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f321406f8e0 0x7f32141459b0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f3214073710 tx=0x7f320801c3f0 comp rx=0 tx=0).stop 2024-03-23T14:57:53.933 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.929+0000 7f32190fc640 1 -- 172.21.15.164:0/2339787470 shutdown_connections 2024-03-23T14:57:53.933 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.929+0000 7f32190fc640 1 --2- 172.21.15.164:0/2339787470 >> v2:172.21.15.173:3300/0 conn(0x7f3214145ef0 0x7f321414fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:53.933 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.929+0000 7f32190fc640 1 --2- 172.21.15.164:0/2339787470 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f321406f8e0 0x7f32141459b0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:53.933 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.929+0000 7f32190fc640 1 -- 172.21.15.164:0/2339787470 >> 172.21.15.164:0/2339787470 conn(0x7f321406b930 msgr2=0x7f32140b9150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:53.933 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.929+0000 7f32190fc640 1 -- 172.21.15.164:0/2339787470 shutdown_connections 2024-03-23T14:57:53.933 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.929+0000 7f32190fc640 1 -- 172.21.15.164:0/2339787470 wait complete. 2024-03-23T14:57:53.936 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.929+0000 7f32190fc640 1 Processor -- start 2024-03-23T14:57:53.936 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.929+0000 7f32190fc640 1 -- start start 2024-03-23T14:57:53.936 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.929+0000 7f32190fc640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f32140b9ac0 0x7f3214071800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:53.936 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.929+0000 7f32190fc640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3214145ef0 0x7f32140b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:53.936 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.929+0000 7f32190fc640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f32140728d0 0x7f32140b7770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:53.936 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.929+0000 7f32190fc640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f3214132140 con 0x7f32140b9ac0 2024-03-23T14:57:53.936 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.929+0000 7f32190fc640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f321412eaf0 con 0x7f3214145ef0 2024-03-23T14:57:53.937 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.929+0000 7f32190fc640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f3214071d40 con 0x7f32140728d0 2024-03-23T14:57:53.937 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.929+0000 7f3212d9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f32140b9ac0 0x7f3214071800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:53.937 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.929+0000 7f3212d9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f32140b9ac0 0x7f3214071800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:48642/0 (socket says 172.21.15.164:48642) 2024-03-23T14:57:53.937 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.929+0000 7f3212d9d640 1 -- 172.21.15.164:0/1821097041 learned_addr learned my addr 172.21.15.164:0/1821097041 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:53.937 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.929+0000 7f321259c640 1 --2- 172.21.15.164:0/1821097041 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3214145ef0 0x7f32140b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:53.937 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.929+0000 7f3212d9d640 1 -- 172.21.15.164:0/1821097041 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f32140728d0 msgr2=0x7f32140b7770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:53.937 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.929+0000 7f3212d9d640 1 --2- 172.21.15.164:0/1821097041 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f32140728d0 0x7f32140b7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:53.937 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.929+0000 7f3212d9d640 1 -- 172.21.15.164:0/1821097041 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3214145ef0 msgr2=0x7f32140b7230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:53.937 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.929+0000 7f3212d9d640 1 --2- 172.21.15.164:0/1821097041 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3214145ef0 0x7f32140b7230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:53.937 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.929+0000 7f3212d9d640 1 -- 172.21.15.164:0/1821097041 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3208009420 con 0x7f32140b9ac0 2024-03-23T14:57:53.937 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.929+0000 7f321259c640 1 --2- 172.21.15.164:0/1821097041 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3214145ef0 0x7f32140b7230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T14:57:53.937 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.929+0000 7f3212d9d640 1 --2- 172.21.15.164:0/1821097041 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f32140b9ac0 0x7f3214071800 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f32040011f0 tx=0x7f320400c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:53.937 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.929+0000 7f31fb7fe640 1 -- 172.21.15.164:0/1821097041 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32040074e0 con 0x7f32140b9ac0 2024-03-23T14:57:53.937 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.929+0000 7f31fb7fe640 1 -- 172.21.15.164:0/1821097041 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3204007640 con 0x7f32140b9ac0 2024-03-23T14:57:53.937 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.929+0000 7f32190fc640 1 -- 172.21.15.164:0/1821097041 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3214070af0 con 0x7f32140b9ac0 2024-03-23T14:57:53.937 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.929+0000 7f31fb7fe640 1 -- 172.21.15.164:0/1821097041 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3204004a10 con 0x7f32140b9ac0 2024-03-23T14:57:53.937 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.929+0000 7f32190fc640 1 -- 172.21.15.164:0/1821097041 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f321412d280 con 0x7f32140b9ac0 2024-03-23T14:57:53.938 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.929+0000 7f32190fc640 1 -- 172.21.15.164:0/1821097041 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3214131f30 con 0x7f32140b9ac0 2024-03-23T14:57:53.938 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.933+0000 7f31fb7fe640 1 -- 172.21.15.164:0/1821097041 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7f3204004b70 con 0x7f32140b9ac0 2024-03-23T14:57:53.938 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.933+0000 7f31fb7fe640 1 --2- 172.21.15.164:0/1821097041 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f32000672a0 0x7f3200069760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:53.938 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.933+0000 7f31fb7fe640 1 -- 172.21.15.164:0/1821097041 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5709+0+0 (secure 0 0 0) 0x7f3204084550 con 0x7f32140b9ac0 2024-03-23T14:57:53.938 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.933+0000 7f321259c640 1 --2- 172.21.15.164:0/1821097041 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f32000672a0 0x7f3200069760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:53.938 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.933+0000 7f321259c640 1 --2- 172.21.15.164:0/1821097041 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f32000672a0 0x7f3200069760 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f32080af000 tx=0x7f3208008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:53.940 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:53.937+0000 7f31fb7fe640 1 -- 172.21.15.164:0/1821097041 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7f3204051190 con 0x7f32140b9ac0 2024-03-23T14:57:54.120 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.117+0000 7f32190fc640 1 -- 172.21.15.164:0/1821097041 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f32140690a0 con 0x7f32140b9ac0 2024-03-23T14:57:54.120 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.117+0000 7f31fb7fe640 1 -- 172.21.15.164:0/1821097041 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+3537 (secure 0 0 0) 0x7f3204018df0 con 0x7f32140b9ac0 2024-03-23T14:57:54.120 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-23T14:57:54.121 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":13,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4619,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/1401585102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":1401585102},{"type":"v1","addr":"172.21.15.173:6833","nonce":1401585102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-23T14:57:40.961416+0000","modified":"2024-03-23T14:57:43.045871+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4935},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4935":{"gid":4935,"name":"a","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.164:6835/3261092607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":3261092607},{"type":"v1","addr":"172.21.15.164:6835","nonce":3261092607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4935,"qdb_cluster":[4935]},"id":2}]} 2024-03-23T14:57:54.121 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 13 2024-03-23T14:57:54.122 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.117+0000 7f32190fc640 1 -- 172.21.15.164:0/1821097041 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f32000672a0 msgr2=0x7f3200069760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:54.122 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.117+0000 7f32190fc640 1 --2- 172.21.15.164:0/1821097041 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f32000672a0 0x7f3200069760 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f32080af000 tx=0x7f3208008040 comp rx=0 tx=0).stop 2024-03-23T14:57:54.123 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.117+0000 7f32190fc640 1 -- 172.21.15.164:0/1821097041 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f32140b9ac0 msgr2=0x7f3214071800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:54.123 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.117+0000 7f32190fc640 1 --2- 172.21.15.164:0/1821097041 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f32140b9ac0 0x7f3214071800 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f32040011f0 tx=0x7f320400c9b0 comp rx=0 tx=0).stop 2024-03-23T14:57:54.123 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.117+0000 7f32190fc640 1 -- 172.21.15.164:0/1821097041 shutdown_connections 2024-03-23T14:57:54.123 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.117+0000 7f32190fc640 1 --2- 172.21.15.164:0/1821097041 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f32140728d0 0x7f32140b7770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:54.123 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.117+0000 7f32190fc640 1 --2- 172.21.15.164:0/1821097041 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f32000672a0 0x7f3200069760 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:54.123 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.117+0000 7f32190fc640 1 --2- 172.21.15.164:0/1821097041 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3214145ef0 0x7f32140b7230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:54.123 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.117+0000 7f32190fc640 1 --2- 172.21.15.164:0/1821097041 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f32140b9ac0 0x7f3214071800 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:54.123 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.117+0000 7f32190fc640 1 -- 172.21.15.164:0/1821097041 >> 172.21.15.164:0/1821097041 conn(0x7f321406b930 msgr2=0x7f321413caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:54.123 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.117+0000 7f32190fc640 1 -- 172.21.15.164:0/1821097041 shutdown_connections 2024-03-23T14:57:54.123 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.117+0000 7f32190fc640 1 -- 172.21.15.164:0/1821097041 wait complete. 2024-03-23T14:57:54.138 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T14:57:54.263 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.257+0000 7fceff604640 1 Processor -- start 2024-03-23T14:57:54.263 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.257+0000 7fceff604640 1 -- start start 2024-03-23T14:57:54.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.261+0000 7fceff604640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fcef806e8e0 0x7fcef8141380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:54.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.261+0000 7fceff604640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcef802c2a0 0x7fcef802c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:54.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.261+0000 7fceff604640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcef80b9470 con 0x7fcef814a880 2024-03-23T14:57:54.266 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.261+0000 7fceff604640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 -- 0x7fcef8072af0 con 0x7fcef806e8e0 2024-03-23T14:57:54.267 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.261+0000 7fceff604640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fcef806ecc0 con 0x7fcef802c2a0 2024-03-23T14:57:54.267 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.261+0000 7fcefdba1640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fcef814a880 0x7fcef814ad10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:43080/0 (socket says 172.21.15.164:43080) 2024-03-23T14:57:54.267 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.261+0000 7fcefdba1640 1 -- 172.21.15.164:0/2314242401 learned_addr learned my addr 172.21.15.164:0/2314242401 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-03-23T14:57:54.267 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.261+0000 7fcefd3a0640 1 --2- 172.21.15.164:0/2314242401 >> v2:172.21.15.173:3300/0 conn(0x7fcef806e8e0 0x7fcef8141380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:54.267 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.261+0000 7fcefcb9f640 1 --2- 172.21.15.164:0/2314242401 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcef802c2a0 0x7fcef802c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:54.267 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.261+0000 7fceebfff640 1 -- 172.21.15.164:0/2314242401 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3521959461 0 0) 0x7fcef80b9470 con 0x7fcef814a880 2024-03-23T14:57:54.267 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.261+0000 7fceebfff640 1 -- 172.21.15.164:0/2314242401 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcee4003530 con 0x7fcef814a880 2024-03-23T14:57:54.267 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.261+0000 7fceebfff640 1 -- 172.21.15.164:0/2314242401 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 737979349 0 0) 0x7fcee4003530 con 0x7fcef814a880 2024-03-23T14:57:54.267 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.261+0000 7fceebfff640 1 -- 172.21.15.164:0/2314242401 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcef802c2a0 msgr2=0x7fcef802c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:54.267 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.261+0000 7fceebfff640 1 --2- 172.21.15.164:0/2314242401 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcef802c2a0 0x7fcef802c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:54.267 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.261+0000 7fceebfff640 1 -- 172.21.15.164:0/2314242401 >> v2:172.21.15.173:3300/0 conn(0x7fcef806e8e0 msgr2=0x7fcef8141380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:54.267 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.261+0000 7fceebfff640 1 --2- 172.21.15.164:0/2314242401 >> v2:172.21.15.173:3300/0 conn(0x7fcef806e8e0 0x7fcef8141380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:54.267 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.261+0000 7fceebfff640 1 -- 172.21.15.164:0/2314242401 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcef8072040 con 0x7fcef814a880 2024-03-23T14:57:54.267 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.261+0000 7fceebfff640 1 -- 172.21.15.164:0/2314242401 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2526390548 0 0) 0x7fcef0001a00 con 0x7fcef814a880 2024-03-23T14:57:54.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.261+0000 7fceebfff640 1 -- 172.21.15.164:0/2314242401 >> v1:172.21.15.164:6789/0 conn(0x7fcef814a880 legacy=0x7fcef814ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:54.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.261+0000 7fceebfff640 1 --2- 172.21.15.164:0/2314242401 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcee4003980 0x7fcee4003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:54.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.261+0000 7fceebfff640 1 --2- 172.21.15.164:0/2314242401 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcef802c2a0 0x7fcee40047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:54.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.261+0000 7fceebfff640 1 --2- 172.21.15.164:0/2314242401 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcee4004d10 0x7fcee4005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:54.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.261+0000 7fceebfff640 1 -- 172.21.15.164:0/2314242401 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fcef8072af0 con 0x7fcee4004d10 2024-03-23T14:57:54.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.261+0000 7fceebfff640 1 -- 172.21.15.164:0/2314242401 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fcef806ecc0 con 0x7fcee4003980 2024-03-23T14:57:54.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.261+0000 7fceebfff640 1 -- 172.21.15.164:0/2314242401 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fcee400b990 con 0x7fcef802c2a0 2024-03-23T14:57:54.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.261+0000 7fcefdba1640 1 --2- 172.21.15.164:0/2314242401 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcee4004d10 0x7fcee4005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:54.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.261+0000 7fcefd3a0640 1 --2- 172.21.15.164:0/2314242401 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcee4003980 0x7fcee4003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:54.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.261+0000 7fcefcb9f640 1 --2- 172.21.15.164:0/2314242401 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcef802c2a0 0x7fcee40047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:54.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.261+0000 7fcefdba1640 1 -- 172.21.15.164:0/2314242401 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcef802c2a0 msgr2=0x7fcee40047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:54.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.261+0000 7fcefdba1640 1 --2- 172.21.15.164:0/2314242401 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcef802c2a0 0x7fcee40047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:54.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.261+0000 7fcefdba1640 1 -- 172.21.15.164:0/2314242401 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcee4003980 msgr2=0x7fcee4003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:54.268 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.261+0000 7fcefdba1640 1 --2- 172.21.15.164:0/2314242401 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcee4003980 0x7fcee4003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:54.269 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.261+0000 7fcefdba1640 1 -- 172.21.15.164:0/2314242401 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcee4003530 con 0x7fcee4004d10 2024-03-23T14:57:54.269 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.261+0000 7fcefcb9f640 1 --2- 172.21.15.164:0/2314242401 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcef802c2a0 0x7fcee40047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T14:57:54.269 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.261+0000 7fcefd3a0640 1 --2- 172.21.15.164:0/2314242401 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcee4003980 0x7fcee4003da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T14:57:54.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.265+0000 7fcefdba1640 1 --2- 172.21.15.164:0/2314242401 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcee4004d10 0x7fcee4005190 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fcef000a830 tx=0x7fcef001d360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fe9b9529366d672 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:54.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.265+0000 7fceebfff640 1 -- 172.21.15.164:0/2314242401 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcef00ad630 con 0x7fcee4004d10 2024-03-23T14:57:54.270 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.265+0000 7fceebfff640 1 -- 172.21.15.164:0/2314242401 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcef00a8070 con 0x7fcee4004d10 2024-03-23T14:57:54.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.265+0000 7fceebfff640 1 -- 172.21.15.164:0/2314242401 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcef00ad9d0 con 0x7fcee4004d10 2024-03-23T14:57:54.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.265+0000 7fceff604640 1 -- 172.21.15.164:0/2314242401 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcee4004d10 msgr2=0x7fcee4005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:54.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.265+0000 7fceff604640 1 --2- 172.21.15.164:0/2314242401 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcee4004d10 0x7fcee4005190 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fcef000a830 tx=0x7fcef001d360 comp rx=0 tx=0).stop 2024-03-23T14:57:54.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.265+0000 7fcefdba1640 1 -- 172.21.15.164:0/2314242401 reap_dead start 2024-03-23T14:57:54.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.265+0000 7fceff604640 1 -- 172.21.15.164:0/2314242401 shutdown_connections 2024-03-23T14:57:54.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.265+0000 7fceff604640 1 -- 172.21.15.164:0/2314242401 >> 172.21.15.164:0/2314242401 conn(0x7fcef80b7970 msgr2=0x7fcef80b7d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:54.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.265+0000 7fceff604640 1 -- 172.21.15.164:0/2314242401 shutdown_connections 2024-03-23T14:57:54.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.265+0000 7fceff604640 1 -- 172.21.15.164:0/2314242401 wait complete. 2024-03-23T14:57:54.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.265+0000 7fceff604640 1 Processor -- start 2024-03-23T14:57:54.271 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.265+0000 7fceff604640 1 -- start start 2024-03-23T14:57:54.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.265+0000 7fceff604640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcef80b33e0 0x7fcef80b1e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:54.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.265+0000 7fceff604640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcef80b2370 0x7fcef81d1e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:54.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.265+0000 7fceff604640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcef80b37c0 0x7fcef81d23a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:54.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.265+0000 7fceff604640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fcef81dceb0 con 0x7fcef80b37c0 2024-03-23T14:57:54.273 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.265+0000 7fceff604640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fcef81d0a10 con 0x7fcef80b2370 2024-03-23T14:57:54.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.265+0000 7fceff604640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fcef80b3cc0 con 0x7fcef80b33e0 2024-03-23T14:57:54.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.265+0000 7fcefdba1640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcef80b37c0 0x7fcef81d23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:54.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.265+0000 7fcefdba1640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcef80b37c0 0x7fcef81d23a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:48668/0 (socket says 172.21.15.164:48668) 2024-03-23T14:57:54.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.265+0000 7fcefdba1640 1 -- 172.21.15.164:0/3293813959 learned_addr learned my addr 172.21.15.164:0/3293813959 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:54.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.265+0000 7fcefd3a0640 1 --2- 172.21.15.164:0/3293813959 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcef80b33e0 0x7fcef80b1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:54.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.265+0000 7fcefdba1640 1 -- 172.21.15.164:0/3293813959 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcef80b33e0 msgr2=0x7fcef80b1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:54.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.265+0000 7fcefcb9f640 1 --2- 172.21.15.164:0/3293813959 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcef80b2370 0x7fcef81d1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:54.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.265+0000 7fcefdba1640 1 --2- 172.21.15.164:0/3293813959 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcef80b33e0 0x7fcef80b1e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:54.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.265+0000 7fcefdba1640 1 -- 172.21.15.164:0/3293813959 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcef80b2370 msgr2=0x7fcef81d1e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:54.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.265+0000 7fcefdba1640 1 --2- 172.21.15.164:0/3293813959 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcef80b2370 0x7fcef81d1e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:54.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.265+0000 7fcefdba1640 1 -- 172.21.15.164:0/3293813959 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcef81dd8a0 con 0x7fcef80b37c0 2024-03-23T14:57:54.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.269+0000 7fcefdba1640 1 --2- 172.21.15.164:0/3293813959 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcef80b37c0 0x7fcef81d23a0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fcef001cdd0 tx=0x7fcee4005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:54.274 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.269+0000 7fcee9ffb640 1 -- 172.21.15.164:0/3293813959 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcef00019d0 con 0x7fcef80b37c0 2024-03-23T14:57:54.275 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.269+0000 7fcee9ffb640 1 -- 172.21.15.164:0/3293813959 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcef0002000 con 0x7fcef80b37c0 2024-03-23T14:57:54.275 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.269+0000 7fceff604640 1 -- 172.21.15.164:0/3293813959 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcef81dd0a0 con 0x7fcef80b37c0 2024-03-23T14:57:54.275 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.269+0000 7fcee9ffb640 1 -- 172.21.15.164:0/3293813959 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcef00ad630 con 0x7fcef80b37c0 2024-03-23T14:57:54.275 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.269+0000 7fceff604640 1 -- 172.21.15.164:0/3293813959 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcef80b1290 con 0x7fcef80b37c0 2024-03-23T14:57:54.275 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.269+0000 7fceff604640 1 -- 172.21.15.164:0/3293813959 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcef80690a0 con 0x7fcef80b37c0 2024-03-23T14:57:54.275 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.269+0000 7fcee9ffb640 1 -- 172.21.15.164:0/3293813959 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7fcef00aa030 con 0x7fcef80b37c0 2024-03-23T14:57:54.276 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.269+0000 7fcee9ffb640 1 --2- 172.21.15.164:0/3293813959 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fcedc0672a0 0x7fcedc069760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:54.276 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.269+0000 7fcefd3a0640 1 --2- 172.21.15.164:0/3293813959 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fcedc0672a0 0x7fcedc069760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:54.276 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.269+0000 7fcee9ffb640 1 -- 172.21.15.164:0/3293813959 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5709+0+0 (secure 0 0 0) 0x7fcef011e9a0 con 0x7fcef80b37c0 2024-03-23T14:57:54.276 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.269+0000 7fcefd3a0640 1 --2- 172.21.15.164:0/3293813959 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fcedc0672a0 0x7fcedc069760 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fcef80b1770 tx=0x7fcef4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:54.278 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.273+0000 7fcee9ffb640 1 -- 172.21.15.164:0/3293813959 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7fcef00ef110 con 0x7fcef80b37c0 2024-03-23T14:57:54.427 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.421+0000 7fceff604640 1 -- 172.21.15.164:0/3293813959 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcef81dd550 con 0x7fcef80b37c0 2024-03-23T14:57:54.428 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.425+0000 7fcee9ffb640 1 -- 172.21.15.164:0/3293813959 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) ==== 74+0+16693 (secure 0 0 0) 0x7fcef00ba090 con 0x7fcef80b37c0 2024-03-23T14:57:54.428 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-23T14:57:54.428 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":29,"fsid":"45c7911b-b69d-4e68-907b-fbfeb780aa3b","created":"2024-03-23T14:56:34.367633+0000","modified":"2024-03-23T14:57:42.610096+0000","last_up_change":"2024-03-23T14:56:46.297403+0000","last_in_change":"2024-03-23T14:56:36.629884+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T14:56:47.186157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-03-23T14:57:39.290666+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-03-23T14:57:39.934337+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"29","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"80abdc62-bd0d-4e60-a554-7ed6fd67cf54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":984716690},{"type":"v1","addr":"172.21.15.164:6801","nonce":984716690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":984716690},{"type":"v1","addr":"172.21.15.164:6803","nonce":984716690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":984716690},{"type":"v1","addr":"172.21.15.164:6807","nonce":984716690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":984716690},{"type":"v1","addr":"172.21.15.164:6805","nonce":984716690}]},"public_addr":"172.21.15.164:6801/984716690","cluster_addr":"172.21.15.164:6803/984716690","heartbeat_back_addr":"172.21.15.164:6807/984716690","heartbeat_front_addr":"172.21.15.164:6805/984716690","state":["exists","up"]},{"osd":1,"uuid":"f9001b06-be4d-4f42-a0e6-bcef2afa0d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":1349657239},{"type":"v1","addr":"172.21.15.164:6817","nonce":1349657239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":1349657239},{"type":"v1","addr":"172.21.15.164:6819","nonce":1349657239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":1349657239},{"type":"v1","addr":"172.21.15.164:6823","nonce":1349657239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":1349657239},{"type":"v1","addr":"172.21.15.164:6821","nonce":1349657239}]},"public_addr":"172.21.15.164:6817/1349657239","cluster_addr":"172.21.15.164:6819/1349657239","heartbeat_back_addr":"172.21.15.164:6823/1349657239","heartbeat_front_addr":"172.21.15.164:6821/1349657239","state":["exists","up"]},{"osd":2,"uuid":"fa0aeb9f-8b54-4831-a188-3e361c7980d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":887861252},{"type":"v1","addr":"172.21.15.164:6809","nonce":887861252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":887861252},{"type":"v1","addr":"172.21.15.164:6811","nonce":887861252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":887861252},{"type":"v1","addr":"172.21.15.164:6815","nonce":887861252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":887861252},{"type":"v1","addr":"172.21.15.164:6813","nonce":887861252}]},"public_addr":"172.21.15.164:6809/887861252","cluster_addr":"172.21.15.164:6811/887861252","heartbeat_back_addr":"172.21.15.164:6815/887861252","heartbeat_front_addr":"172.21.15.164:6813/887861252","state":["exists","up"]},{"osd":3,"uuid":"7e18c957-3091-46d5-8e17-8f3354c1726c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":812540271},{"type":"v1","addr":"172.21.15.164:6825","nonce":812540271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":812540271},{"type":"v1","addr":"172.21.15.164:6827","nonce":812540271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":812540271},{"type":"v1","addr":"172.21.15.164:6831","nonce":812540271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":812540271},{"type":"v1","addr":"172.21.15.164:6829","nonce":812540271}]},"public_addr":"172.21.15.164:6825/812540271","cluster_addr":"172.21.15.164:6827/812540271","heartbeat_back_addr":"172.21.15.164:6831/812540271","heartbeat_front_addr":"172.21.15.164:6829/812540271","state":["exists","up"]},{"osd":4,"uuid":"8e83f315-42bf-4471-9ce8-d81948da0f9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":2656857421},{"type":"v1","addr":"172.21.15.173:6825","nonce":2656857421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2656857421},{"type":"v1","addr":"172.21.15.173:6827","nonce":2656857421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2656857421},{"type":"v1","addr":"172.21.15.173:6831","nonce":2656857421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":2656857421},{"type":"v1","addr":"172.21.15.173:6829","nonce":2656857421}]},"public_addr":"172.21.15.173:6825/2656857421","cluster_addr":"172.21.15.173:6827/2656857421","heartbeat_back_addr":"172.21.15.173:6831/2656857421","heartbeat_front_addr":"172.21.15.173:6829/2656857421","state":["exists","up"]},{"osd":5,"uuid":"a230c872-71f9-40ab-8999-defd0162d4ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":3676483185},{"type":"v1","addr":"172.21.15.173:6801","nonce":3676483185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":3676483185},{"type":"v1","addr":"172.21.15.173:6803","nonce":3676483185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":3676483185},{"type":"v1","addr":"172.21.15.173:6807","nonce":3676483185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":3676483185},{"type":"v1","addr":"172.21.15.173:6805","nonce":3676483185}]},"public_addr":"172.21.15.173:6801/3676483185","cluster_addr":"172.21.15.173:6803/3676483185","heartbeat_back_addr":"172.21.15.173:6807/3676483185","heartbeat_front_addr":"172.21.15.173:6805/3676483185","state":["exists","up"]},{"osd":6,"uuid":"6193bfd7-8432-4f7f-a117-a3fcee21af99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":3631684087},{"type":"v1","addr":"172.21.15.173:6809","nonce":3631684087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":3631684087},{"type":"v1","addr":"172.21.15.173:6811","nonce":3631684087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":3631684087},{"type":"v1","addr":"172.21.15.173:6815","nonce":3631684087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":3631684087},{"type":"v1","addr":"172.21.15.173:6813","nonce":3631684087}]},"public_addr":"172.21.15.173:6809/3631684087","cluster_addr":"172.21.15.173:6811/3631684087","heartbeat_back_addr":"172.21.15.173:6815/3631684087","heartbeat_front_addr":"172.21.15.173:6813/3631684087","state":["exists","up"]},{"osd":7,"uuid":"b0f1b80a-afc4-4538-83b9-f081d2dcb8cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":3261114948},{"type":"v1","addr":"172.21.15.173:6817","nonce":3261114948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":3261114948},{"type":"v1","addr":"172.21.15.173:6819","nonce":3261114948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":3261114948},{"type":"v1","addr":"172.21.15.173:6823","nonce":3261114948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":3261114948},{"type":"v1","addr":"172.21.15.173:6821","nonce":3261114948}]},"public_addr":"172.21.15.173:6817/3261114948","cluster_addr":"172.21.15.173:6819/3261114948","heartbeat_back_addr":"172.21.15.173:6823/3261114948","heartbeat_front_addr":"172.21.15.173:6821/3261114948","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.188390+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.159932+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.191101+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.181776+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.224365+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.177515+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.153631+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.139972+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T14:57:54.430 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.425+0000 7fceff604640 1 -- 172.21.15.164:0/3293813959 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fcedc0672a0 msgr2=0x7fcedc069760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:54.430 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.425+0000 7fceff604640 1 --2- 172.21.15.164:0/3293813959 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fcedc0672a0 0x7fcedc069760 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fcef80b1770 tx=0x7fcef4008040 comp rx=0 tx=0).stop 2024-03-23T14:57:54.430 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.425+0000 7fceff604640 1 -- 172.21.15.164:0/3293813959 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcef80b37c0 msgr2=0x7fcef81d23a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:54.430 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.425+0000 7fceff604640 1 --2- 172.21.15.164:0/3293813959 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcef80b37c0 0x7fcef81d23a0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fcef001cdd0 tx=0x7fcee4005800 comp rx=0 tx=0).stop 2024-03-23T14:57:54.430 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.425+0000 7fceff604640 1 -- 172.21.15.164:0/3293813959 shutdown_connections 2024-03-23T14:57:54.430 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.425+0000 7fceff604640 1 --2- 172.21.15.164:0/3293813959 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcef80b37c0 0x7fcef81d23a0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:54.430 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.425+0000 7fceff604640 1 --2- 172.21.15.164:0/3293813959 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fcedc0672a0 0x7fcedc069760 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:54.431 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.425+0000 7fceff604640 1 --2- 172.21.15.164:0/3293813959 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcef80b2370 0x7fcef81d1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:54.431 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.425+0000 7fceff604640 1 --2- 172.21.15.164:0/3293813959 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcef80b33e0 0x7fcef80b1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:54.431 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.425+0000 7fceff604640 1 -- 172.21.15.164:0/3293813959 >> 172.21.15.164:0/3293813959 conn(0x7fcef80b7970 msgr2=0x7fcef806cf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:54.431 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.425+0000 7fceff604640 1 -- 172.21.15.164:0/3293813959 shutdown_connections 2024-03-23T14:57:54.431 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.425+0000 7fceff604640 1 -- 172.21.15.164:0/3293813959 wait complete. 2024-03-23T14:57:54.451 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-23T14:57:54.451 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T14:57:54.559 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.557+0000 7f3d85043640 1 Processor -- start 2024-03-23T14:57:54.560 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.557+0000 7f3d85043640 1 -- start start 2024-03-23T14:57:54.561 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.557+0000 7f3d85043640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3d8006f8e0 0x7f3d801459b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:54.561 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.557+0000 7f3d85043640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f3d80145ef0 0x7f3d8014fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:54.561 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.557+0000 7f3d85043640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d80145240 con 0x7f3d800b9ac0 2024-03-23T14:57:54.561 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.557+0000 7f3d85043640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f3d800b3b50 con 0x7f3d80145ef0 2024-03-23T14:57:54.561 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.557+0000 7f3d85043640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f3d80070050 con 0x7f3d8006f8e0 2024-03-23T14:57:54.561 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.557+0000 7f3d7f59e640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f3d80145ef0 0x7f3d8014fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:54.561 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.557+0000 7f3d7f59e640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f3d80145ef0 0x7f3d8014fba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:48684/0 (socket says 172.21.15.164:48684) 2024-03-23T14:57:54.562 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.557+0000 7f3d7ed9d640 1 --1- 172.21.15.164:0/1817812598 >> v1:172.21.15.164:6789/0 conn(0x7f3d800b9ac0 0x7f3d8013f240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:43090/0 (socket says 172.21.15.164:43090) 2024-03-23T14:57:54.562 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.557+0000 7f3d7f59e640 1 -- 172.21.15.164:0/1817812598 learned_addr learned my addr 172.21.15.164:0/1817812598 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:54.562 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.557+0000 7f3d7e59c640 1 --2- 172.21.15.164:0/1817812598 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3d8006f8e0 0x7f3d801459b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:54.562 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.557+0000 7f3d7f59e640 1 -- 172.21.15.164:0/1817812598 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3d8006f8e0 msgr2=0x7f3d801459b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:54.562 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.557+0000 7f3d7f59e640 1 --2- 172.21.15.164:0/1817812598 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3d8006f8e0 0x7f3d801459b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:54.562 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.557+0000 7f3d7dd9b640 1 -- 172.21.15.164:0/1817812598 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 441766937 0 0) 0x7f3d80145240 con 0x7f3d800b9ac0 2024-03-23T14:57:54.562 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.557+0000 7f3d7f59e640 1 -- 172.21.15.164:0/1817812598 >> v1:172.21.15.164:6789/0 conn(0x7f3d800b9ac0 legacy=0x7f3d8013f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:54.562 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.557+0000 7f3d7f59e640 1 -- 172.21.15.164:0/1817812598 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d8013f950 con 0x7f3d80145ef0 2024-03-23T14:57:54.565 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.561+0000 7f3d7f59e640 1 --2- 172.21.15.164:0/1817812598 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3d80145ef0 0x7f3d8014fba0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f3d800701f0 tx=0x7f3d7801c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7999fd5ef32c9f70 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:54.565 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.561+0000 7f3d7dd9b640 1 -- 172.21.15.164:0/1817812598 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d780a5840 con 0x7f3d80145ef0 2024-03-23T14:57:54.565 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.561+0000 7f3d7dd9b640 1 -- 172.21.15.164:0/1817812598 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d780a59a0 con 0x7f3d80145ef0 2024-03-23T14:57:54.565 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.561+0000 7f3d7dd9b640 1 -- 172.21.15.164:0/1817812598 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d780a5d30 con 0x7f3d80145ef0 2024-03-23T14:57:54.565 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.561+0000 7f3d85043640 1 -- 172.21.15.164:0/1817812598 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3d80145ef0 msgr2=0x7f3d8014fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:54.565 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.561+0000 7f3d85043640 1 --2- 172.21.15.164:0/1817812598 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3d80145ef0 0x7f3d8014fba0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f3d800701f0 tx=0x7f3d7801c980 comp rx=0 tx=0).stop 2024-03-23T14:57:54.566 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.561+0000 7f3d85043640 1 -- 172.21.15.164:0/1817812598 shutdown_connections 2024-03-23T14:57:54.566 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.561+0000 7f3d85043640 1 --2- 172.21.15.164:0/1817812598 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3d80145ef0 0x7f3d8014fba0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:54.566 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.561+0000 7f3d85043640 1 --2- 172.21.15.164:0/1817812598 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3d8006f8e0 0x7f3d801459b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:54.566 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.561+0000 7f3d85043640 1 -- 172.21.15.164:0/1817812598 >> 172.21.15.164:0/1817812598 conn(0x7f3d8006b930 msgr2=0x7f3d800b9150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:54.566 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.561+0000 7f3d85043640 1 -- 172.21.15.164:0/1817812598 shutdown_connections 2024-03-23T14:57:54.566 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.561+0000 7f3d85043640 1 -- 172.21.15.164:0/1817812598 wait complete. 2024-03-23T14:57:54.568 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.561+0000 7f3d85043640 1 Processor -- start 2024-03-23T14:57:54.569 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.561+0000 7f3d85043640 1 -- start start 2024-03-23T14:57:54.569 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.561+0000 7f3d85043640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3d800b9ac0 0x7f3d8012fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:54.569 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.561+0000 7f3d85043640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3d80145ef0 0x7f3d801dd890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:54.569 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.561+0000 7f3d85043640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3d80132470 0x7f3d801dddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:54.569 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.561+0000 7f3d85043640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f3d8012dc80 con 0x7f3d80132470 2024-03-23T14:57:54.569 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.561+0000 7f3d85043640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f3d8012d650 con 0x7f3d80145ef0 2024-03-23T14:57:54.569 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.561+0000 7f3d85043640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f3d8012eb00 con 0x7f3d800b9ac0 2024-03-23T14:57:54.569 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.561+0000 7f3d7f59e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3d80132470 0x7f3d801dddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:54.569 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.561+0000 7f3d7f59e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3d80132470 0x7f3d801dddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:48688/0 (socket says 172.21.15.164:48688) 2024-03-23T14:57:54.569 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.561+0000 7f3d7f59e640 1 -- 172.21.15.164:0/2675608276 learned_addr learned my addr 172.21.15.164:0/2675608276 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:54.569 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.561+0000 7f3d7e59c640 1 --2- 172.21.15.164:0/2675608276 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3d80145ef0 0x7f3d801dd890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:54.570 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.561+0000 7f3d7ed9d640 1 --2- 172.21.15.164:0/2675608276 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3d800b9ac0 0x7f3d8012fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:54.570 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.561+0000 7f3d7f59e640 1 -- 172.21.15.164:0/2675608276 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3d800b9ac0 msgr2=0x7f3d8012fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:54.570 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.561+0000 7f3d7f59e640 1 --2- 172.21.15.164:0/2675608276 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3d800b9ac0 0x7f3d8012fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:54.570 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.561+0000 7f3d7f59e640 1 -- 172.21.15.164:0/2675608276 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3d80145ef0 msgr2=0x7f3d801dd890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:54.570 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.561+0000 7f3d7f59e640 1 --2- 172.21.15.164:0/2675608276 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3d80145ef0 0x7f3d801dd890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:54.570 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.561+0000 7f3d7f59e640 1 -- 172.21.15.164:0/2675608276 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d78009b30 con 0x7f3d80132470 2024-03-23T14:57:54.570 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.561+0000 7f3d7ed9d640 1 --2- 172.21.15.164:0/2675608276 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3d800b9ac0 0x7f3d8012fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T14:57:54.570 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.561+0000 7f3d7e59c640 1 --2- 172.21.15.164:0/2675608276 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3d80145ef0 0x7f3d801dd890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T14:57:54.570 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.561+0000 7f3d7f59e640 1 --2- 172.21.15.164:0/2675608276 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3d80132470 0x7f3d801dddd0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f3d7801a5b0 tx=0x7f3d78019720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:54.570 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.561+0000 7f3d677fe640 1 -- 172.21.15.164:0/2675608276 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d780b8880 con 0x7f3d80132470 2024-03-23T14:57:54.570 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.565+0000 7f3d677fe640 1 -- 172.21.15.164:0/2675608276 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d780b8a00 con 0x7f3d80132470 2024-03-23T14:57:54.570 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.565+0000 7f3d85043640 1 -- 172.21.15.164:0/2675608276 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d80070af0 con 0x7f3d80132470 2024-03-23T14:57:54.570 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.565+0000 7f3d677fe640 1 -- 172.21.15.164:0/2675608276 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d780b8db0 con 0x7f3d80132470 2024-03-23T14:57:54.570 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.565+0000 7f3d85043640 1 -- 172.21.15.164:0/2675608276 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d8012d8a0 con 0x7f3d80132470 2024-03-23T14:57:54.570 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.565+0000 7f3d85043640 1 -- 172.21.15.164:0/2675608276 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d8012ddc0 con 0x7f3d80132470 2024-03-23T14:57:54.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.565+0000 7f3d677fe640 1 -- 172.21.15.164:0/2675608276 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7f3d780b6440 con 0x7f3d80132470 2024-03-23T14:57:54.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.565+0000 7f3d677fe640 1 --2- 172.21.15.164:0/2675608276 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f3d6c0672a0 0x7f3d6c069760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:54.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.565+0000 7f3d677fe640 1 -- 172.21.15.164:0/2675608276 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5709+0+0 (secure 0 0 0) 0x7f3d78120860 con 0x7f3d80132470 2024-03-23T14:57:54.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.565+0000 7f3d7ed9d640 1 --2- 172.21.15.164:0/2675608276 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f3d6c0672a0 0x7f3d6c069760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:54.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.565+0000 7f3d7ed9d640 1 --2- 172.21.15.164:0/2675608276 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f3d6c0672a0 0x7f3d6c069760 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f3d70000f70 tx=0x7f3d70009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:54.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.569+0000 7f3d677fe640 1 -- 172.21.15.164:0/2675608276 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7f3d780ed420 con 0x7f3d80132470 2024-03-23T14:57:54.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.749+0000 7f3d85043640 1 -- 172.21.15.164:0/2675608276 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3d8006fe00 con 0x7f3d80132470 2024-03-23T14:57:54.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.749+0000 7f3d677fe640 1 -- 172.21.15.164:0/2675608276 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+3537 (secure 0 0 0) 0x7f3d780bcdf0 con 0x7f3d80132470 2024-03-23T14:57:54.756 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-23T14:57:54.756 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":13,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4619,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/1401585102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":1401585102},{"type":"v1","addr":"172.21.15.173:6833","nonce":1401585102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-23T14:57:40.961416+0000","modified":"2024-03-23T14:57:43.045871+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4935},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4935":{"gid":4935,"name":"a","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.164:6835/3261092607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":3261092607},{"type":"v1","addr":"172.21.15.164:6835","nonce":3261092607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4935,"qdb_cluster":[4935]},"id":2}]} 2024-03-23T14:57:54.756 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 13 2024-03-23T14:57:54.759 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.753+0000 7f3d85043640 1 -- 172.21.15.164:0/2675608276 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f3d6c0672a0 msgr2=0x7f3d6c069760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:54.759 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.753+0000 7f3d85043640 1 --2- 172.21.15.164:0/2675608276 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f3d6c0672a0 0x7f3d6c069760 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f3d70000f70 tx=0x7f3d70009040 comp rx=0 tx=0).stop 2024-03-23T14:57:54.759 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.753+0000 7f3d85043640 1 -- 172.21.15.164:0/2675608276 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3d80132470 msgr2=0x7f3d801dddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:54.760 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.753+0000 7f3d85043640 1 --2- 172.21.15.164:0/2675608276 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3d80132470 0x7f3d801dddd0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f3d7801a5b0 tx=0x7f3d78019720 comp rx=0 tx=0).stop 2024-03-23T14:57:54.760 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.753+0000 7f3d85043640 1 -- 172.21.15.164:0/2675608276 shutdown_connections 2024-03-23T14:57:54.760 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.753+0000 7f3d85043640 1 --2- 172.21.15.164:0/2675608276 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3d80132470 0x7f3d801dddd0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:54.760 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.753+0000 7f3d85043640 1 --2- 172.21.15.164:0/2675608276 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f3d6c0672a0 0x7f3d6c069760 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:54.760 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.753+0000 7f3d85043640 1 --2- 172.21.15.164:0/2675608276 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3d80145ef0 0x7f3d801dd890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:54.760 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.753+0000 7f3d85043640 1 --2- 172.21.15.164:0/2675608276 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3d800b9ac0 0x7f3d8012fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:54.760 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.753+0000 7f3d85043640 1 -- 172.21.15.164:0/2675608276 >> 172.21.15.164:0/2675608276 conn(0x7f3d8006b930 msgr2=0x7f3d800b3ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:54.760 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.753+0000 7f3d85043640 1 -- 172.21.15.164:0/2675608276 shutdown_connections 2024-03-23T14:57:54.760 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.753+0000 7f3d85043640 1 -- 172.21.15.164:0/2675608276 wait complete. 2024-03-23T14:57:54.781 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T14:57:54.903 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.897+0000 7f22aeacf640 1 Processor -- start 2024-03-23T14:57:54.903 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.897+0000 7f22aeacf640 1 -- start start 2024-03-23T14:57:54.904 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.901+0000 7f22aeacf640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f22a80725b0 0x7f22a8148890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:54.904 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.901+0000 7f22aeacf640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f22a806e730 0x7f22a80b3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:54.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.901+0000 7f22aeacf640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22a80b0ca0 con 0x7f22a806d770 2024-03-23T14:57:54.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.901+0000 7f22aeacf640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f22a8072990 con 0x7f22a80725b0 2024-03-23T14:57:54.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.901+0000 7f22aeacf640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 -- 0x7f22a80b95f0 con 0x7f22a806e730 2024-03-23T14:57:54.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.901+0000 7f22ad06c640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f22a806d770 0x7f22a802bfc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:43102/0 (socket says 172.21.15.164:43102) 2024-03-23T14:57:54.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.901+0000 7f22ad06c640 1 -- 172.21.15.164:0/2300656057 learned_addr learned my addr 172.21.15.164:0/2300656057 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-03-23T14:57:54.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.901+0000 7f22ac86b640 1 --2- 172.21.15.164:0/2300656057 >> v2:172.21.15.164:3300/0 conn(0x7f22a80725b0 0x7f22a8148890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:54.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.901+0000 7f22a7fff640 1 --2- 172.21.15.164:0/2300656057 >> v2:172.21.15.173:3300/0 conn(0x7f22a806e730 0x7f22a80b3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:54.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.901+0000 7f22a77fe640 1 -- 172.21.15.164:0/2300656057 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 67122096 0 0) 0x7f2298001120 con 0x7f22a806d770 2024-03-23T14:57:54.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.901+0000 7f22a77fe640 1 -- 172.21.15.164:0/2300656057 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2294003530 con 0x7f22a806d770 2024-03-23T14:57:54.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.901+0000 7f22ac86b640 1 -- 172.21.15.164:0/2300656057 >> v2:172.21.15.173:3300/0 conn(0x7f22a806e730 msgr2=0x7f22a80b3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:54.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.901+0000 7f22ac86b640 1 --2- 172.21.15.164:0/2300656057 >> v2:172.21.15.173:3300/0 conn(0x7f22a806e730 0x7f22a80b3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:54.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.901+0000 7f22ac86b640 1 -- 172.21.15.164:0/2300656057 >> v1:172.21.15.164:6789/0 conn(0x7f22a806d770 legacy=0x7f22a802bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:54.905 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.901+0000 7f22ac86b640 1 -- 172.21.15.164:0/2300656057 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22a8148dd0 con 0x7f22a80725b0 2024-03-23T14:57:54.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.901+0000 7f22ac86b640 1 --2- 172.21.15.164:0/2300656057 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f22a80725b0 0x7f22a8148890 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f22a80b1560 tx=0x7f22a001c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5673e58ecbf9e79d server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:54.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.901+0000 7f22a77fe640 1 -- 172.21.15.164:0/2300656057 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22a00ab600 con 0x7f22a80725b0 2024-03-23T14:57:54.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.901+0000 7f22a77fe640 1 -- 172.21.15.164:0/2300656057 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22a00a7050 con 0x7f22a80725b0 2024-03-23T14:57:54.907 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.901+0000 7f22a77fe640 1 -- 172.21.15.164:0/2300656057 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22a00ab980 con 0x7f22a80725b0 2024-03-23T14:57:54.907 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.901+0000 7f22aeacf640 1 -- 172.21.15.164:0/2300656057 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f22a80725b0 msgr2=0x7f22a8148890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:54.907 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.901+0000 7f22aeacf640 1 --2- 172.21.15.164:0/2300656057 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f22a80725b0 0x7f22a8148890 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f22a80b1560 tx=0x7f22a001c3f0 comp rx=0 tx=0).stop 2024-03-23T14:57:54.907 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.901+0000 7f22aeacf640 1 -- 172.21.15.164:0/2300656057 shutdown_connections 2024-03-23T14:57:54.907 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.901+0000 7f22aeacf640 1 --2- 172.21.15.164:0/2300656057 >> v2:172.21.15.173:3300/0 conn(0x7f22a806e730 0x7f22a80b3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:54.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.901+0000 7f22aeacf640 1 --2- 172.21.15.164:0/2300656057 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f22a80725b0 0x7f22a8148890 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:54.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.901+0000 7f22aeacf640 1 -- 172.21.15.164:0/2300656057 >> 172.21.15.164:0/2300656057 conn(0x7f22a80b7970 msgr2=0x7f22a80b7d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:54.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.901+0000 7f22aeacf640 1 -- 172.21.15.164:0/2300656057 shutdown_connections 2024-03-23T14:57:54.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.901+0000 7f22aeacf640 1 -- 172.21.15.164:0/2300656057 wait complete. 2024-03-23T14:57:54.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.905+0000 7f22aeacf640 1 Processor -- start 2024-03-23T14:57:54.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.905+0000 7f22aeacf640 1 -- start start 2024-03-23T14:57:54.911 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.905+0000 7f22aeacf640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f22a806d770 0x7f22a813e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:54.911 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.905+0000 7f22aeacf640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f22a806e730 0x7f22a8141720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:54.911 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.905+0000 7f22aeacf640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f22a80725b0 0x7f22a81429c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:54.912 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.905+0000 7f22aeacf640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f22a8145a60 con 0x7f22a80725b0 2024-03-23T14:57:54.912 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.905+0000 7f22aeacf640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f22a8141c60 con 0x7f22a806e730 2024-03-23T14:57:54.912 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.905+0000 7f22aeacf640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f22a8142f00 con 0x7f22a806d770 2024-03-23T14:57:54.912 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.905+0000 7f22ad06c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f22a80725b0 0x7f22a81429c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:54.912 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.905+0000 7f22ad06c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f22a80725b0 0x7f22a81429c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:48700/0 (socket says 172.21.15.164:48700) 2024-03-23T14:57:54.912 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.905+0000 7f22ad06c640 1 -- 172.21.15.164:0/1659257010 learned_addr learned my addr 172.21.15.164:0/1659257010 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:54.912 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.905+0000 7f22a7fff640 1 --2- 172.21.15.164:0/1659257010 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f22a806e730 0x7f22a8141720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:54.912 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.905+0000 7f22ac86b640 1 --2- 172.21.15.164:0/1659257010 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f22a806d770 0x7f22a813e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:54.912 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.905+0000 7f22ad06c640 1 -- 172.21.15.164:0/1659257010 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f22a806d770 msgr2=0x7f22a813e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:54.912 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.905+0000 7f22ad06c640 1 --2- 172.21.15.164:0/1659257010 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f22a806d770 0x7f22a813e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:54.912 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.905+0000 7f22ad06c640 1 -- 172.21.15.164:0/1659257010 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f22a806e730 msgr2=0x7f22a8141720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:54.912 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.905+0000 7f22ad06c640 1 --2- 172.21.15.164:0/1659257010 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f22a806e730 0x7f22a8141720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:54.912 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.905+0000 7f22ad06c640 1 -- 172.21.15.164:0/1659257010 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22a0009420 con 0x7f22a80725b0 2024-03-23T14:57:54.912 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.905+0000 7f22ad06c640 1 --2- 172.21.15.164:0/1659257010 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f22a80725b0 0x7f22a81429c0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f229800c430 tx=0x7f229800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:54.912 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.905+0000 7f22a57fa640 1 -- 172.21.15.164:0/1659257010 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2298005cf0 con 0x7f22a80725b0 2024-03-23T14:57:54.913 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.905+0000 7f22a57fa640 1 -- 172.21.15.164:0/1659257010 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2298005e50 con 0x7f22a80725b0 2024-03-23T14:57:54.913 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.905+0000 7f22aeacf640 1 -- 172.21.15.164:0/1659257010 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22a8141ee0 con 0x7f22a80725b0 2024-03-23T14:57:54.913 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.905+0000 7f22a57fa640 1 -- 172.21.15.164:0/1659257010 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22980072f0 con 0x7f22a80725b0 2024-03-23T14:57:54.913 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.905+0000 7f22aeacf640 1 -- 172.21.15.164:0/1659257010 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22a8145cb0 con 0x7f22a80725b0 2024-03-23T14:57:54.914 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.909+0000 7f22aeacf640 1 -- 172.21.15.164:0/1659257010 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22a81426c0 con 0x7f22a80725b0 2024-03-23T14:57:54.914 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.909+0000 7f22a57fa640 1 -- 172.21.15.164:0/1659257010 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7f2298007450 con 0x7f22a80725b0 2024-03-23T14:57:54.914 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.909+0000 7f22a57fa640 1 --2- 172.21.15.164:0/1659257010 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f22880672a0 0x7f2288069760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:54.914 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.909+0000 7f22a57fa640 1 -- 172.21.15.164:0/1659257010 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5709+0+0 (secure 0 0 0) 0x7f22980843f0 con 0x7f22a80725b0 2024-03-23T14:57:54.914 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.909+0000 7f22ac86b640 1 --2- 172.21.15.164:0/1659257010 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f22880672a0 0x7f2288069760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:54.914 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.909+0000 7f22ac86b640 1 --2- 172.21.15.164:0/1659257010 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f22880672a0 0x7f2288069760 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f22a00ad000 tx=0x7f22a0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:54.917 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:54.913+0000 7f22a57fa640 1 -- 172.21.15.164:0/1659257010 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7f2298050fa0 con 0x7f22a80725b0 2024-03-23T14:57:55.108 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.105+0000 7f22aeacf640 1 -- 172.21.15.164:0/1659257010 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f22a80b0ca0 con 0x7f22a80725b0 2024-03-23T14:57:55.108 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.105+0000 7f22a57fa640 1 -- 172.21.15.164:0/1659257010 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+3537 (secure 0 0 0) 0x7f2298017df0 con 0x7f22a80725b0 2024-03-23T14:57:55.108 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-23T14:57:55.109 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":13,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4619,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/1401585102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":1401585102},{"type":"v1","addr":"172.21.15.173:6833","nonce":1401585102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-23T14:57:40.961416+0000","modified":"2024-03-23T14:57:43.045871+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4935},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4935":{"gid":4935,"name":"a","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.164:6835/3261092607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":3261092607},{"type":"v1","addr":"172.21.15.164:6835","nonce":3261092607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4935,"qdb_cluster":[4935]},"id":2}]} 2024-03-23T14:57:55.109 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 13 2024-03-23T14:57:55.111 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.105+0000 7f22aeacf640 1 -- 172.21.15.164:0/1659257010 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f22880672a0 msgr2=0x7f2288069760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:55.111 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.105+0000 7f22aeacf640 1 --2- 172.21.15.164:0/1659257010 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f22880672a0 0x7f2288069760 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f22a00ad000 tx=0x7f22a0008040 comp rx=0 tx=0).stop 2024-03-23T14:57:55.111 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.105+0000 7f22aeacf640 1 -- 172.21.15.164:0/1659257010 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f22a80725b0 msgr2=0x7f22a81429c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:55.111 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.105+0000 7f22aeacf640 1 --2- 172.21.15.164:0/1659257010 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f22a80725b0 0x7f22a81429c0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f229800c430 tx=0x7f229800c8f0 comp rx=0 tx=0).stop 2024-03-23T14:57:55.111 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.105+0000 7f22aeacf640 1 -- 172.21.15.164:0/1659257010 shutdown_connections 2024-03-23T14:57:55.111 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.105+0000 7f22aeacf640 1 --2- 172.21.15.164:0/1659257010 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f22a80725b0 0x7f22a81429c0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:55.111 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.105+0000 7f22aeacf640 1 --2- 172.21.15.164:0/1659257010 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f22880672a0 0x7f2288069760 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:55.111 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.105+0000 7f22aeacf640 1 --2- 172.21.15.164:0/1659257010 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f22a806e730 0x7f22a8141720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:55.111 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.105+0000 7f22aeacf640 1 --2- 172.21.15.164:0/1659257010 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f22a806d770 0x7f22a813e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:55.111 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.105+0000 7f22aeacf640 1 -- 172.21.15.164:0/1659257010 >> 172.21.15.164:0/1659257010 conn(0x7f22a80b7970 msgr2=0x7f22a80b56d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:55.111 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.105+0000 7f22aeacf640 1 -- 172.21.15.164:0/1659257010 shutdown_connections 2024-03-23T14:57:55.111 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.105+0000 7f22aeacf640 1 -- 172.21.15.164:0/1659257010 wait complete. 2024-03-23T14:57:55.126 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T14:57:55.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.229+0000 7f4eb793c640 1 Processor -- start 2024-03-23T14:57:55.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.229+0000 7f4eb793c640 1 -- start start 2024-03-23T14:57:55.236 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.233+0000 7f4eb793c640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f4eb002c2a0 0x7f4eb002c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:55.236 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.233+0000 7f4eb793c640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f4eb014fc00 0x7f4eb0151ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:55.237 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.233+0000 7f4eb793c640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4eb0141a90 con 0x7f4eb006e8e0 2024-03-23T14:57:55.237 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.233+0000 7f4eb793c640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f4eb00b9470 con 0x7f4eb002c2a0 2024-03-23T14:57:55.237 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.233+0000 7f4eb793c640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 -- 0x7f4eb0072af0 con 0x7f4eb014fc00 2024-03-23T14:57:55.237 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.233+0000 7f4eb4ed7640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f4eb002c2a0 0x7f4eb002c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:55.237 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.233+0000 7f4eb56d8640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f4eb006e8e0 0x7f4eb0141380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:43106/0 (socket says 172.21.15.164:43106) 2024-03-23T14:57:55.237 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.233+0000 7f4eb56d8640 1 -- 172.21.15.164:0/433534684 learned_addr learned my addr 172.21.15.164:0/433534684 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-03-23T14:57:55.237 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.233+0000 7f4eb4ed7640 1 -- 172.21.15.164:0/433534684 >> v2:172.21.15.173:3300/0 conn(0x7f4eb014fc00 msgr2=0x7f4eb0151ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:55.237 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.233+0000 7f4eb4ed7640 1 --2- 172.21.15.164:0/433534684 >> v2:172.21.15.173:3300/0 conn(0x7f4eb014fc00 0x7f4eb0151ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:55.237 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.233+0000 7f4eb4ed7640 1 -- 172.21.15.164:0/433534684 >> v1:172.21.15.164:6789/0 conn(0x7f4eb006e8e0 legacy=0x7f4eb0141380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:55.237 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.233+0000 7f4eb4ed7640 1 -- 172.21.15.164:0/433534684 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4eb006ecc0 con 0x7f4eb002c2a0 2024-03-23T14:57:55.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.233+0000 7f4eb4ed7640 1 --2- 172.21.15.164:0/433534684 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4eb002c2a0 0x7f4eb002c6e0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f4eb002d560 tx=0x7f4ea001ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8267238a7939daf server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:55.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.233+0000 7f4ea7fff640 1 -- 172.21.15.164:0/433534684 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea00a8970 con 0x7f4eb002c2a0 2024-03-23T14:57:55.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.233+0000 7f4ea7fff640 1 -- 172.21.15.164:0/433534684 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ea00a8ad0 con 0x7f4eb002c2a0 2024-03-23T14:57:55.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.233+0000 7f4ea7fff640 1 -- 172.21.15.164:0/433534684 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea00a8e60 con 0x7f4eb002c2a0 2024-03-23T14:57:55.240 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.237+0000 7f4eb793c640 1 -- 172.21.15.164:0/433534684 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4eb002c2a0 msgr2=0x7f4eb002c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:55.240 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.237+0000 7f4eb793c640 1 --2- 172.21.15.164:0/433534684 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4eb002c2a0 0x7f4eb002c6e0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f4eb002d560 tx=0x7f4ea001ec70 comp rx=0 tx=0).stop 2024-03-23T14:57:55.240 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.237+0000 7f4eb793c640 1 -- 172.21.15.164:0/433534684 shutdown_connections 2024-03-23T14:57:55.241 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.237+0000 7f4eb793c640 1 --2- 172.21.15.164:0/433534684 >> v2:172.21.15.173:3300/0 conn(0x7f4eb014fc00 0x7f4eb0151ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:55.241 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.237+0000 7f4eb793c640 1 --2- 172.21.15.164:0/433534684 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4eb002c2a0 0x7f4eb002c6e0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:55.241 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.237+0000 7f4eb793c640 1 -- 172.21.15.164:0/433534684 >> 172.21.15.164:0/433534684 conn(0x7f4eb00b7970 msgr2=0x7f4eb00b7d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:55.241 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.237+0000 7f4eb793c640 1 -- 172.21.15.164:0/433534684 shutdown_connections 2024-03-23T14:57:55.241 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.237+0000 7f4eb793c640 1 -- 172.21.15.164:0/433534684 wait complete. 2024-03-23T14:57:55.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.237+0000 7f4eb793c640 1 Processor -- start 2024-03-23T14:57:55.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.237+0000 7f4eb793c640 1 -- start start 2024-03-23T14:57:55.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.237+0000 7f4eb793c640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4eb002c2a0 0x7f4eb0144b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:55.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.237+0000 7f4eb793c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4eb006e8e0 0x7f4eb00b32b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:55.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.237+0000 7f4eb793c640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4eb014fc00 0x7f4eb0141fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:55.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.237+0000 7f4eb793c640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb013d930 con 0x7f4eb006e8e0 2024-03-23T14:57:55.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.237+0000 7f4eb793c640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb00b0a90 con 0x7f4eb002c2a0 2024-03-23T14:57:55.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.237+0000 7f4eb793c640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f4eb0143dc0 con 0x7f4eb014fc00 2024-03-23T14:57:55.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.237+0000 7f4eb4ed7640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4eb006e8e0 0x7f4eb00b32b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:55.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.237+0000 7f4eb4ed7640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4eb006e8e0 0x7f4eb00b32b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:48730/0 (socket says 172.21.15.164:48730) 2024-03-23T14:57:55.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.237+0000 7f4eb4ed7640 1 -- 172.21.15.164:0/2261106985 learned_addr learned my addr 172.21.15.164:0/2261106985 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:55.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.237+0000 7f4eb56d8640 1 --2- 172.21.15.164:0/2261106985 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4eb002c2a0 0x7f4eb0144b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:55.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.237+0000 7f4eb4ed7640 1 -- 172.21.15.164:0/2261106985 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4eb014fc00 msgr2=0x7f4eb0141fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:55.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.237+0000 7f4eb4ed7640 1 --2- 172.21.15.164:0/2261106985 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4eb014fc00 0x7f4eb0141fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:55.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.237+0000 7f4eb4ed7640 1 -- 172.21.15.164:0/2261106985 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4eb002c2a0 msgr2=0x7f4eb0144b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:55.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.237+0000 7f4eb4ed7640 1 --2- 172.21.15.164:0/2261106985 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4eb002c2a0 0x7f4eb0144b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:55.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.237+0000 7f4eb4ed7640 1 -- 172.21.15.164:0/2261106985 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ea0009420 con 0x7f4eb006e8e0 2024-03-23T14:57:55.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.237+0000 7f4eb4ed7640 1 --2- 172.21.15.164:0/2261106985 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4eb006e8e0 0x7f4eb00b32b0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f4ea00b3040 tx=0x7f4ea001ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:55.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.237+0000 7f4ea5ffb640 1 -- 172.21.15.164:0/2261106985 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea00a8970 con 0x7f4eb006e8e0 2024-03-23T14:57:55.244 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.237+0000 7f4ea5ffb640 1 -- 172.21.15.164:0/2261106985 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ea00b9410 con 0x7f4eb006e8e0 2024-03-23T14:57:55.245 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.237+0000 7f4eb793c640 1 -- 172.21.15.164:0/2261106985 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4eb01440a0 con 0x7f4eb006e8e0 2024-03-23T14:57:55.245 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.237+0000 7f4ea5ffb640 1 -- 172.21.15.164:0/2261106985 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea00b97c0 con 0x7f4eb006e8e0 2024-03-23T14:57:55.245 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.237+0000 7f4eb793c640 1 -- 172.21.15.164:0/2261106985 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4eb0144660 con 0x7f4eb006e8e0 2024-03-23T14:57:55.245 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.241+0000 7f4eb793c640 1 -- 172.21.15.164:0/2261106985 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4eb00b1e60 con 0x7f4eb006e8e0 2024-03-23T14:57:55.245 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.241+0000 7f4ea5ffb640 1 -- 172.21.15.164:0/2261106985 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7f4ea00b7040 con 0x7f4eb006e8e0 2024-03-23T14:57:55.246 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.241+0000 7f4ea5ffb640 1 --2- 172.21.15.164:0/2261106985 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f4e940672a0 0x7f4e94069760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:55.246 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.241+0000 7f4ea5ffb640 1 -- 172.21.15.164:0/2261106985 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5709+0+0 (secure 0 0 0) 0x7f4ea0122cf0 con 0x7f4eb006e8e0 2024-03-23T14:57:55.246 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.241+0000 7f4eb56d8640 1 --2- 172.21.15.164:0/2261106985 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f4e940672a0 0x7f4e94069760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:55.246 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.241+0000 7f4eb56d8640 1 --2- 172.21.15.164:0/2261106985 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f4e940672a0 0x7f4e94069760 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f4eac000f70 tx=0x7f4eac009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:55.248 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.245+0000 7f4ea5ffb640 1 -- 172.21.15.164:0/2261106985 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7f4ea0127070 con 0x7f4eb006e8e0 2024-03-23T14:57:55.443 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.437+0000 7f4eb793c640 1 -- 172.21.15.164:0/2261106985 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4eb0144850 con 0x7f4eb006e8e0 2024-03-23T14:57:55.445 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.441+0000 7f4ea5ffb640 1 -- 172.21.15.164:0/2261106985 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) ==== 74+0+16693 (secure 0 0 0) 0x7f4ea00b5070 con 0x7f4eb006e8e0 2024-03-23T14:57:55.446 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-23T14:57:55.446 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":29,"fsid":"45c7911b-b69d-4e68-907b-fbfeb780aa3b","created":"2024-03-23T14:56:34.367633+0000","modified":"2024-03-23T14:57:42.610096+0000","last_up_change":"2024-03-23T14:56:46.297403+0000","last_in_change":"2024-03-23T14:56:36.629884+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T14:56:47.186157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-03-23T14:57:39.290666+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-03-23T14:57:39.934337+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"29","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"80abdc62-bd0d-4e60-a554-7ed6fd67cf54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":984716690},{"type":"v1","addr":"172.21.15.164:6801","nonce":984716690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":984716690},{"type":"v1","addr":"172.21.15.164:6803","nonce":984716690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":984716690},{"type":"v1","addr":"172.21.15.164:6807","nonce":984716690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":984716690},{"type":"v1","addr":"172.21.15.164:6805","nonce":984716690}]},"public_addr":"172.21.15.164:6801/984716690","cluster_addr":"172.21.15.164:6803/984716690","heartbeat_back_addr":"172.21.15.164:6807/984716690","heartbeat_front_addr":"172.21.15.164:6805/984716690","state":["exists","up"]},{"osd":1,"uuid":"f9001b06-be4d-4f42-a0e6-bcef2afa0d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":1349657239},{"type":"v1","addr":"172.21.15.164:6817","nonce":1349657239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":1349657239},{"type":"v1","addr":"172.21.15.164:6819","nonce":1349657239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":1349657239},{"type":"v1","addr":"172.21.15.164:6823","nonce":1349657239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":1349657239},{"type":"v1","addr":"172.21.15.164:6821","nonce":1349657239}]},"public_addr":"172.21.15.164:6817/1349657239","cluster_addr":"172.21.15.164:6819/1349657239","heartbeat_back_addr":"172.21.15.164:6823/1349657239","heartbeat_front_addr":"172.21.15.164:6821/1349657239","state":["exists","up"]},{"osd":2,"uuid":"fa0aeb9f-8b54-4831-a188-3e361c7980d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":887861252},{"type":"v1","addr":"172.21.15.164:6809","nonce":887861252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":887861252},{"type":"v1","addr":"172.21.15.164:6811","nonce":887861252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":887861252},{"type":"v1","addr":"172.21.15.164:6815","nonce":887861252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":887861252},{"type":"v1","addr":"172.21.15.164:6813","nonce":887861252}]},"public_addr":"172.21.15.164:6809/887861252","cluster_addr":"172.21.15.164:6811/887861252","heartbeat_back_addr":"172.21.15.164:6815/887861252","heartbeat_front_addr":"172.21.15.164:6813/887861252","state":["exists","up"]},{"osd":3,"uuid":"7e18c957-3091-46d5-8e17-8f3354c1726c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":812540271},{"type":"v1","addr":"172.21.15.164:6825","nonce":812540271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":812540271},{"type":"v1","addr":"172.21.15.164:6827","nonce":812540271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":812540271},{"type":"v1","addr":"172.21.15.164:6831","nonce":812540271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":812540271},{"type":"v1","addr":"172.21.15.164:6829","nonce":812540271}]},"public_addr":"172.21.15.164:6825/812540271","cluster_addr":"172.21.15.164:6827/812540271","heartbeat_back_addr":"172.21.15.164:6831/812540271","heartbeat_front_addr":"172.21.15.164:6829/812540271","state":["exists","up"]},{"osd":4,"uuid":"8e83f315-42bf-4471-9ce8-d81948da0f9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":2656857421},{"type":"v1","addr":"172.21.15.173:6825","nonce":2656857421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2656857421},{"type":"v1","addr":"172.21.15.173:6827","nonce":2656857421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2656857421},{"type":"v1","addr":"172.21.15.173:6831","nonce":2656857421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":2656857421},{"type":"v1","addr":"172.21.15.173:6829","nonce":2656857421}]},"public_addr":"172.21.15.173:6825/2656857421","cluster_addr":"172.21.15.173:6827/2656857421","heartbeat_back_addr":"172.21.15.173:6831/2656857421","heartbeat_front_addr":"172.21.15.173:6829/2656857421","state":["exists","up"]},{"osd":5,"uuid":"a230c872-71f9-40ab-8999-defd0162d4ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":3676483185},{"type":"v1","addr":"172.21.15.173:6801","nonce":3676483185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":3676483185},{"type":"v1","addr":"172.21.15.173:6803","nonce":3676483185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":3676483185},{"type":"v1","addr":"172.21.15.173:6807","nonce":3676483185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":3676483185},{"type":"v1","addr":"172.21.15.173:6805","nonce":3676483185}]},"public_addr":"172.21.15.173:6801/3676483185","cluster_addr":"172.21.15.173:6803/3676483185","heartbeat_back_addr":"172.21.15.173:6807/3676483185","heartbeat_front_addr":"172.21.15.173:6805/3676483185","state":["exists","up"]},{"osd":6,"uuid":"6193bfd7-8432-4f7f-a117-a3fcee21af99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":3631684087},{"type":"v1","addr":"172.21.15.173:6809","nonce":3631684087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":3631684087},{"type":"v1","addr":"172.21.15.173:6811","nonce":3631684087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":3631684087},{"type":"v1","addr":"172.21.15.173:6815","nonce":3631684087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":3631684087},{"type":"v1","addr":"172.21.15.173:6813","nonce":3631684087}]},"public_addr":"172.21.15.173:6809/3631684087","cluster_addr":"172.21.15.173:6811/3631684087","heartbeat_back_addr":"172.21.15.173:6815/3631684087","heartbeat_front_addr":"172.21.15.173:6813/3631684087","state":["exists","up"]},{"osd":7,"uuid":"b0f1b80a-afc4-4538-83b9-f081d2dcb8cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":3261114948},{"type":"v1","addr":"172.21.15.173:6817","nonce":3261114948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":3261114948},{"type":"v1","addr":"172.21.15.173:6819","nonce":3261114948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":3261114948},{"type":"v1","addr":"172.21.15.173:6823","nonce":3261114948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":3261114948},{"type":"v1","addr":"172.21.15.173:6821","nonce":3261114948}]},"public_addr":"172.21.15.173:6817/3261114948","cluster_addr":"172.21.15.173:6819/3261114948","heartbeat_back_addr":"172.21.15.173:6823/3261114948","heartbeat_front_addr":"172.21.15.173:6821/3261114948","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.188390+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.159932+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.191101+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.181776+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.224365+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.177515+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.153631+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.139972+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T14:57:55.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.445+0000 7f4eb793c640 1 -- 172.21.15.164:0/2261106985 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f4e940672a0 msgr2=0x7f4e94069760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:55.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.445+0000 7f4eb793c640 1 --2- 172.21.15.164:0/2261106985 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f4e940672a0 0x7f4e94069760 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f4eac000f70 tx=0x7f4eac009040 comp rx=0 tx=0).stop 2024-03-23T14:57:55.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.445+0000 7f4eb793c640 1 -- 172.21.15.164:0/2261106985 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4eb006e8e0 msgr2=0x7f4eb00b32b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:55.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.445+0000 7f4eb793c640 1 --2- 172.21.15.164:0/2261106985 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4eb006e8e0 0x7f4eb00b32b0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f4ea00b3040 tx=0x7f4ea001ba10 comp rx=0 tx=0).stop 2024-03-23T14:57:55.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.445+0000 7f4eb793c640 1 -- 172.21.15.164:0/2261106985 shutdown_connections 2024-03-23T14:57:55.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.445+0000 7f4eb793c640 1 --2- 172.21.15.164:0/2261106985 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4eb014fc00 0x7f4eb0141fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:55.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.445+0000 7f4eb793c640 1 --2- 172.21.15.164:0/2261106985 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4eb006e8e0 0x7f4eb00b32b0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:55.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.445+0000 7f4eb793c640 1 --2- 172.21.15.164:0/2261106985 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f4e940672a0 0x7f4e94069760 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:55.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.445+0000 7f4eb793c640 1 --2- 172.21.15.164:0/2261106985 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4eb002c2a0 0x7f4eb0144b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:55.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.445+0000 7f4eb793c640 1 -- 172.21.15.164:0/2261106985 >> 172.21.15.164:0/2261106985 conn(0x7f4eb00b7970 msgr2=0x7f4eb013f740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:55.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.445+0000 7f4eb793c640 1 -- 172.21.15.164:0/2261106985 shutdown_connections 2024-03-23T14:57:55.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.445+0000 7f4eb793c640 1 -- 172.21.15.164:0/2261106985 wait complete. 2024-03-23T14:57:55.464 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-23T14:57:55.588 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.585+0000 7fbea4fc5640 1 Processor -- start 2024-03-23T14:57:55.588 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.585+0000 7fbea4fc5640 1 -- start start 2024-03-23T14:57:55.592 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.585+0000 7fbea4fc5640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fbea00b48e0 0x7fbea0150ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:55.592 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.585+0000 7fbea4fc5640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbea00b5530 con 0x7fbea00b6740 2024-03-23T14:57:55.592 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.585+0000 7fbea4fc5640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbea00b2a70 con 0x7fbea00b6360 2024-03-23T14:57:55.592 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.585+0000 7fbea4fc5640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 -- 0x7fbea00b5a60 con 0x7fbea00b48e0 2024-03-23T14:57:55.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.585+0000 7fbe9dd9b640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fbea00b6740 0x7fbea00b41d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:43112/0 (socket says 172.21.15.164:43112) 2024-03-23T14:57:55.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.585+0000 7fbe9dd9b640 1 -- 172.21.15.164:0/576688716 learned_addr learned my addr 172.21.15.164:0/576688716 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-03-23T14:57:55.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.585+0000 7fbe9d59a640 1 -- 172.21.15.164:0/576688716 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1926104685 0 0) 0x7fbea00b5530 con 0x7fbea00b6740 2024-03-23T14:57:55.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.585+0000 7fbe9d59a640 1 -- 172.21.15.164:0/576688716 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbe88003530 con 0x7fbea00b6740 2024-03-23T14:57:55.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.585+0000 7fbe9ed9d640 1 --2- 172.21.15.164:0/576688716 >> v2:172.21.15.173:3300/0 conn(0x7fbea00b48e0 0x7fbea0150ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:55.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.585+0000 7fbe9d59a640 1 -- 172.21.15.164:0/576688716 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 629529155 0 0) 0x7fbe88003530 con 0x7fbea00b6740 2024-03-23T14:57:55.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.585+0000 7fbe9d59a640 1 -- 172.21.15.164:0/576688716 >> v2:172.21.15.173:3300/0 conn(0x7fbea00b48e0 msgr2=0x7fbea0150ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:55.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.585+0000 7fbe9d59a640 1 --2- 172.21.15.164:0/576688716 >> v2:172.21.15.173:3300/0 conn(0x7fbea00b48e0 0x7fbea0150ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:55.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.585+0000 7fbe9d59a640 1 -- 172.21.15.164:0/576688716 >> v1:172.21.15.173:6789/0 conn(0x7fbea00b6360 legacy=0x7fbea0145a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:55.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.585+0000 7fbe9d59a640 1 -- 172.21.15.164:0/576688716 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbea00b2800 con 0x7fbea00b6740 2024-03-23T14:57:55.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.585+0000 7fbe9d59a640 1 -- 172.21.15.164:0/576688716 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2526390548 0 0) 0x7fbe94001680 con 0x7fbea00b6740 2024-03-23T14:57:55.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.585+0000 7fbe9d59a640 1 -- 172.21.15.164:0/576688716 >> v1:172.21.15.164:6789/0 conn(0x7fbea00b6740 legacy=0x7fbea00b41d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:55.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.585+0000 7fbe9d59a640 1 --2- 172.21.15.164:0/576688716 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbe88003980 0x7fbe88003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:55.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.585+0000 7fbe9d59a640 1 --2- 172.21.15.164:0/576688716 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbe880047d0 0x7fbe88004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:55.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.585+0000 7fbe9d59a640 1 --2- 172.21.15.164:0/576688716 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbe880052b0 0x7fbe8800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:55.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.585+0000 7fbe9d59a640 1 -- 172.21.15.164:0/576688716 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fbea00b5a60 con 0x7fbe88003980 2024-03-23T14:57:55.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.585+0000 7fbe9d59a640 1 -- 172.21.15.164:0/576688716 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fbe8800c180 con 0x7fbe880047d0 2024-03-23T14:57:55.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.585+0000 7fbe9d59a640 1 -- 172.21.15.164:0/576688716 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fbe8800c340 con 0x7fbe880052b0 2024-03-23T14:57:55.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.585+0000 7fbe9e59c640 1 --2- 172.21.15.164:0/576688716 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbe88003980 0x7fbe88003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:55.594 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.585+0000 7fbe9dd9b640 1 --2- 172.21.15.164:0/576688716 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbe880047d0 0x7fbe88004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:55.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.585+0000 7fbe9e59c640 1 -- 172.21.15.164:0/576688716 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbe880052b0 msgr2=0x7fbe8800b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:55.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.585+0000 7fbe9ed9d640 1 --2- 172.21.15.164:0/576688716 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbe880052b0 0x7fbe8800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:55.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.585+0000 7fbe9e59c640 1 --2- 172.21.15.164:0/576688716 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbe880052b0 0x7fbe8800b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:55.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.585+0000 7fbe9e59c640 1 -- 172.21.15.164:0/576688716 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbe880047d0 msgr2=0x7fbe88004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:55.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.585+0000 7fbe9e59c640 1 --2- 172.21.15.164:0/576688716 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbe880047d0 0x7fbe88004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:55.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.585+0000 7fbe9e59c640 1 -- 172.21.15.164:0/576688716 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe88003530 con 0x7fbe88003980 2024-03-23T14:57:55.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.585+0000 7fbe9ed9d640 1 -- 172.21.15.164:0/576688716 reap_dead start 2024-03-23T14:57:55.595 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.585+0000 7fbe9dd9b640 1 --2- 172.21.15.164:0/576688716 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbe880047d0 0x7fbe88004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T14:57:55.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.589+0000 7fbe9e59c640 1 --2- 172.21.15.164:0/576688716 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbe88003980 0x7fbe88003da0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fbe98001e40 tx=0x7fbe9801c720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b33636650b842d36 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:55.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.589+0000 7fbe9d59a640 1 -- 172.21.15.164:0/576688716 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe98003b80 con 0x7fbe88003980 2024-03-23T14:57:55.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.589+0000 7fbe9d59a640 1 -- 172.21.15.164:0/576688716 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe980a7070 con 0x7fbe88003980 2024-03-23T14:57:55.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.589+0000 7fbe9d59a640 1 -- 172.21.15.164:0/576688716 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe980050d0 con 0x7fbe88003980 2024-03-23T14:57:55.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.589+0000 7fbea4fc5640 1 -- 172.21.15.164:0/576688716 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbe88003980 msgr2=0x7fbe88003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:55.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.589+0000 7fbea4fc5640 1 --2- 172.21.15.164:0/576688716 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbe88003980 0x7fbe88003da0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fbe98001e40 tx=0x7fbe9801c720 comp rx=0 tx=0).stop 2024-03-23T14:57:55.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.589+0000 7fbea4fc5640 1 -- 172.21.15.164:0/576688716 shutdown_connections 2024-03-23T14:57:55.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.589+0000 7fbea4fc5640 1 --2- 172.21.15.164:0/576688716 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbe88003980 0x7fbe88003da0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:55.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.589+0000 7fbea4fc5640 1 -- 172.21.15.164:0/576688716 >> 172.21.15.164:0/576688716 conn(0x7fbea00b7630 msgr2=0x7fbea00b7a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:55.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.589+0000 7fbea4fc5640 1 -- 172.21.15.164:0/576688716 shutdown_connections 2024-03-23T14:57:55.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.589+0000 7fbea4fc5640 1 -- 172.21.15.164:0/576688716 wait complete. 2024-03-23T14:57:55.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.593+0000 7fbea4fc5640 1 Processor -- start 2024-03-23T14:57:55.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.593+0000 7fbea4fc5640 1 -- start start 2024-03-23T14:57:55.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.593+0000 7fbea4fc5640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbe88003980 0x7fbea0146450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:55.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.593+0000 7fbea4fc5640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbea0138390 0x7fbea0138bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:55.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.593+0000 7fbea4fc5640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbea01e0e80 0x7fbea01e1300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:55.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.593+0000 7fbea4fc5640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fbea013aae0 con 0x7fbea01e0e80 2024-03-23T14:57:55.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.593+0000 7fbea4fc5640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fbea01387f0 con 0x7fbe88003980 2024-03-23T14:57:55.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.593+0000 7fbea4fc5640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fbea0139800 con 0x7fbea0138390 2024-03-23T14:57:55.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.593+0000 7fbe9ed9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbea01e0e80 0x7fbea01e1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:55.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.593+0000 7fbe9ed9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbea01e0e80 0x7fbea01e1300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:48742/0 (socket says 172.21.15.164:48742) 2024-03-23T14:57:55.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.593+0000 7fbe9ed9d640 1 -- 172.21.15.164:0/2703622038 learned_addr learned my addr 172.21.15.164:0/2703622038 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:55.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.593+0000 7fbe9dd9b640 1 --2- 172.21.15.164:0/2703622038 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbea0138390 0x7fbea0138bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:55.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.593+0000 7fbe9e59c640 1 --2- 172.21.15.164:0/2703622038 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbe88003980 0x7fbea0146450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:55.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.593+0000 7fbe9ed9d640 1 -- 172.21.15.164:0/2703622038 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbea0138390 msgr2=0x7fbea0138bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:55.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.593+0000 7fbe9ed9d640 1 --2- 172.21.15.164:0/2703622038 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbea0138390 0x7fbea0138bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:55.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.593+0000 7fbe9ed9d640 1 -- 172.21.15.164:0/2703622038 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbe88003980 msgr2=0x7fbea0146450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:55.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.593+0000 7fbe9ed9d640 1 --2- 172.21.15.164:0/2703622038 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbe88003980 0x7fbea0146450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:55.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.593+0000 7fbe9ed9d640 1 -- 172.21.15.164:0/2703622038 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbea0139e70 con 0x7fbea01e0e80 2024-03-23T14:57:55.603 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.593+0000 7fbe9dd9b640 1 --2- 172.21.15.164:0/2703622038 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbea0138390 0x7fbea0138bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T14:57:55.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.593+0000 7fbe9ed9d640 1 --2- 172.21.15.164:0/2703622038 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbea01e0e80 0x7fbea01e1300 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fbea0072dc0 tx=0x7fbe9000e8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:55.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.593+0000 7fbe8effd640 1 -- 172.21.15.164:0/2703622038 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe9000c070 con 0x7fbea01e0e80 2024-03-23T14:57:55.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.593+0000 7fbe8effd640 1 -- 172.21.15.164:0/2703622038 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe90013610 con 0x7fbea01e0e80 2024-03-23T14:57:55.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.593+0000 7fbea4fc5640 1 -- 172.21.15.164:0/2703622038 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbea0139b00 con 0x7fbea01e0e80 2024-03-23T14:57:55.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.593+0000 7fbe8effd640 1 -- 172.21.15.164:0/2703622038 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe900139c0 con 0x7fbea01e0e80 2024-03-23T14:57:55.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.593+0000 7fbea4fc5640 1 -- 172.21.15.164:0/2703622038 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbea0148220 con 0x7fbea01e0e80 2024-03-23T14:57:55.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.593+0000 7fbea4fc5640 1 -- 172.21.15.164:0/2703622038 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbea00690a0 con 0x7fbea01e0e80 2024-03-23T14:57:55.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.593+0000 7fbe8effd640 1 -- 172.21.15.164:0/2703622038 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7fbe90013b20 con 0x7fbea01e0e80 2024-03-23T14:57:55.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.593+0000 7fbe8effd640 1 --2- 172.21.15.164:0/2703622038 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fbe800672a0 0x7fbe80069760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:55.604 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.593+0000 7fbe8effd640 1 -- 172.21.15.164:0/2703622038 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5709+0+0 (secure 0 0 0) 0x7fbe90084aa0 con 0x7fbea01e0e80 2024-03-23T14:57:55.605 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.593+0000 7fbe9e59c640 1 --2- 172.21.15.164:0/2703622038 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fbe800672a0 0x7fbe80069760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:55.605 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.597+0000 7fbe9e59c640 1 --2- 172.21.15.164:0/2703622038 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fbe800672a0 0x7fbe80069760 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fbe980ad000 tx=0x7fbe98001520 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:55.605 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.601+0000 7fbe8effd640 1 -- 172.21.15.164:0/2703622038 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7fbe900516e0 con 0x7fbea01e0e80 2024-03-23T14:57:55.874 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.869+0000 7fbea4fc5640 1 -- 172.21.15.164:0/2703622038 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fbea00b24b0 con 0x7fbea01e0e80 2024-03-23T14:57:55.875 INFO:tasks.ceph.mon.a.smithi164.stderr:2024-03-23T14:57:55.869+0000 7fd8f965e640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-23T14:57:55.877 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.873+0000 7fbe8effd640 1 -- 172.21.15.164:0/2703622038 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v14) ==== 161+0+0 (secure 0 0 0) 0x7fbe90054da0 con 0x7fbea01e0e80 2024-03-23T14:57:55.877 INFO:teuthology.orchestra.run.smithi164.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-23T14:57:55.880 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.877+0000 7fbea4fc5640 1 -- 172.21.15.164:0/2703622038 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fbe800672a0 msgr2=0x7fbe80069760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:55.880 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.877+0000 7fbea4fc5640 1 --2- 172.21.15.164:0/2703622038 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fbe800672a0 0x7fbe80069760 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fbe980ad000 tx=0x7fbe98001520 comp rx=0 tx=0).stop 2024-03-23T14:57:55.880 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.877+0000 7fbea4fc5640 1 -- 172.21.15.164:0/2703622038 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbea01e0e80 msgr2=0x7fbea01e1300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:55.880 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.877+0000 7fbea4fc5640 1 --2- 172.21.15.164:0/2703622038 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbea01e0e80 0x7fbea01e1300 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fbea0072dc0 tx=0x7fbe9000e8b0 comp rx=0 tx=0).stop 2024-03-23T14:57:55.881 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.877+0000 7fbea4fc5640 1 -- 172.21.15.164:0/2703622038 shutdown_connections 2024-03-23T14:57:55.882 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.877+0000 7fbea4fc5640 1 --2- 172.21.15.164:0/2703622038 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbea01e0e80 0x7fbea01e1300 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:55.882 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.877+0000 7fbea4fc5640 1 --2- 172.21.15.164:0/2703622038 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbea0138390 0x7fbea0138bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:55.882 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.877+0000 7fbea4fc5640 1 --2- 172.21.15.164:0/2703622038 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fbe800672a0 0x7fbe80069760 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:55.882 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.877+0000 7fbea4fc5640 1 --2- 172.21.15.164:0/2703622038 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbe88003980 0x7fbea0146450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:55.882 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.877+0000 7fbea4fc5640 1 -- 172.21.15.164:0/2703622038 >> 172.21.15.164:0/2703622038 conn(0x7fbea00b7630 msgr2=0x7fbea0071370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:55.882 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.877+0000 7fbea4fc5640 1 -- 172.21.15.164:0/2703622038 shutdown_connections 2024-03-23T14:57:55.882 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:55.877+0000 7fbea4fc5640 1 -- 172.21.15.164:0/2703622038 wait complete. 2024-03-23T14:57:55.882 INFO:tasks.ceph.mds.a.smithi164.stderr: -2665> 2024-03-23T14:57:45.561+0000 7f5013669640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-23T14:57:55.900 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-23T14:57:55.911 INFO:tasks.ceph.mds.a.smithi164.stdout:starting mds.a at 2024-03-23T14:57:56.010 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:56.005+0000 7f3327a5e640 1 Processor -- start 2024-03-23T14:57:56.010 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:56.005+0000 7f3327a5e640 1 -- start start 2024-03-23T14:57:56.012 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:56.005+0000 7f3327a5e640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f33200b4e90 0x7f332014f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:56.012 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:56.005+0000 7f3327a5e640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f332009e2d0 con 0x7f332009def0 2024-03-23T14:57:56.012 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:56.005+0000 7f3327a5e640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f332006cb70 con 0x7f33200b4a00 2024-03-23T14:57:56.012 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:56.005+0000 7f3327a5e640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f3320146de0 con 0x7f33200b4e90 2024-03-23T14:57:56.012 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:56.005+0000 7f33257fa640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f332009def0 0x7f33200ae620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:43144/0 (socket says 172.21.15.164:43144) 2024-03-23T14:57:56.012 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:56.005+0000 7f33257fa640 1 -- 172.21.15.164:0/3885698473 learned_addr learned my addr 172.21.15.164:0/3885698473 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-03-23T14:57:56.012 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:56.005+0000 7f3325ffb640 1 --2- 172.21.15.164:0/3885698473 >> v2:172.21.15.164:3300/0 conn(0x7f33200b4e90 0x7f332014f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:56.012 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:56.009+0000 7f3317fff640 1 -- 172.21.15.164:0/3885698473 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1808510601 0 0) 0x7f332009e2d0 con 0x7f332009def0 2024-03-23T14:57:56.012 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:56.009+0000 7f3317fff640 1 -- 172.21.15.164:0/3885698473 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f330c003530 con 0x7f332009def0 2024-03-23T14:57:56.013 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:56.009+0000 7f3325ffb640 1 -- 172.21.15.164:0/3885698473 >> v1:172.21.15.173:6789/0 conn(0x7f33200b4a00 legacy=0x7f3320147420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:56.013 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:56.009+0000 7f3325ffb640 1 -- 172.21.15.164:0/3885698473 >> v1:172.21.15.164:6789/0 conn(0x7f332009def0 legacy=0x7f33200ae620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:56.013 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:56.009+0000 7f3325ffb640 1 -- 172.21.15.164:0/3885698473 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f332006ce50 con 0x7f33200b4e90 2024-03-23T14:57:56.014 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:56.009+0000 7f3325ffb640 1 --2- 172.21.15.164:0/3885698473 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f33200b4e90 0x7f332014f120 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f3318009d90 tx=0x7f331801c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c583efa59becfd3 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:56.014 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:56.009+0000 7f3317fff640 1 -- 172.21.15.164:0/3885698473 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33180a8630 con 0x7f33200b4e90 2024-03-23T14:57:56.014 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:56.009+0000 7f3317fff640 1 -- 172.21.15.164:0/3885698473 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33180a8790 con 0x7f33200b4e90 2024-03-23T14:57:56.014 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:56.009+0000 7f3317fff640 1 -- 172.21.15.164:0/3885698473 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33180a8b00 con 0x7f33200b4e90 2024-03-23T14:57:56.015 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:56.009+0000 7f3327a5e640 1 -- 172.21.15.164:0/3885698473 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f33200b4e90 msgr2=0x7f332014f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:56.015 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:56.009+0000 7f3327a5e640 1 --2- 172.21.15.164:0/3885698473 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f33200b4e90 0x7f332014f120 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f3318009d90 tx=0x7f331801c810 comp rx=0 tx=0).stop 2024-03-23T14:57:56.015 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:56.009+0000 7f3327a5e640 1 -- 172.21.15.164:0/3885698473 shutdown_connections 2024-03-23T14:57:56.015 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:56.009+0000 7f3327a5e640 1 --2- 172.21.15.164:0/3885698473 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f33200b4e90 0x7f332014f120 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:56.015 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:56.009+0000 7f3327a5e640 1 -- 172.21.15.164:0/3885698473 >> 172.21.15.164:0/3885698473 conn(0x7f33200b8ca0 msgr2=0x7f33200b90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:56.016 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:56.009+0000 7f3327a5e640 1 -- 172.21.15.164:0/3885698473 shutdown_connections 2024-03-23T14:57:56.016 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:56.009+0000 7f3327a5e640 1 -- 172.21.15.164:0/3885698473 wait complete. 2024-03-23T14:57:56.016 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:56.009+0000 7f3327a5e640 1 Processor -- start 2024-03-23T14:57:56.016 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:56.013+0000 7f3327a5e640 1 -- start start 2024-03-23T14:57:56.018 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:56.013+0000 7f3327a5e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f332009def0 0x7f33200b3920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:56.018 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:56.013+0000 7f3327a5e640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f33200b4a00 0x7f33201df900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:56.018 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:56.013+0000 7f3327a5e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f33200b4e90 0x7f332014dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:56.018 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:56.013+0000 7f3327a5e640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f332014efc0 con 0x7f332009def0 2024-03-23T14:57:56.018 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:56.013+0000 7f3327a5e640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f33201e35a0 con 0x7f33200b4e90 2024-03-23T14:57:56.018 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:56.013+0000 7f3327a5e640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f33201e3880 con 0x7f33200b4a00 2024-03-23T14:57:56.018 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:56.013+0000 7f33257fa640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f332009def0 0x7f33200b3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:56.018 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:56.013+0000 7f33257fa640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f332009def0 0x7f33200b3920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:48756/0 (socket says 172.21.15.164:48756) 2024-03-23T14:57:56.018 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:56.013+0000 7f33257fa640 1 -- 172.21.15.164:0/2860995990 learned_addr learned my addr 172.21.15.164:0/2860995990 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:56.018 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:56.013+0000 7f33257fa640 1 -- 172.21.15.164:0/2860995990 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f33200b4a00 msgr2=0x7f33201df900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:56.018 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:56.013+0000 7f33257fa640 1 --2- 172.21.15.164:0/2860995990 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f33200b4a00 0x7f33201df900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:56.018 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:56.013+0000 7f33257fa640 1 -- 172.21.15.164:0/2860995990 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f33200b4e90 msgr2=0x7f332014dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:56.019 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:56.013+0000 7f33257fa640 1 --2- 172.21.15.164:0/2860995990 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f33200b4e90 0x7f332014dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:56.019 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:56.013+0000 7f33257fa640 1 -- 172.21.15.164:0/2860995990 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3318009a70 con 0x7f332009def0 2024-03-23T14:57:56.019 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:56.013+0000 7f33257fa640 1 --2- 172.21.15.164:0/2860995990 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f332009def0 0x7f33200b3920 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f331c00c450 tx=0x7f331c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:56.019 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:56.013+0000 7f3315ffb640 1 -- 172.21.15.164:0/2860995990 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f331c005cf0 con 0x7f332009def0 2024-03-23T14:57:56.019 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:56.013+0000 7f3315ffb640 1 -- 172.21.15.164:0/2860995990 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f331c005e50 con 0x7f332009def0 2024-03-23T14:57:56.019 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:56.013+0000 7f3327a5e640 1 -- 172.21.15.164:0/2860995990 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33201e3da0 con 0x7f332009def0 2024-03-23T14:57:56.019 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:56.013+0000 7f3315ffb640 1 -- 172.21.15.164:0/2860995990 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f331c007290 con 0x7f332009def0 2024-03-23T14:57:56.019 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:56.013+0000 7f3327a5e640 1 -- 172.21.15.164:0/2860995990 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33200b3e60 con 0x7f332009def0 2024-03-23T14:57:56.019 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:56.013+0000 7f3327a5e640 1 -- 172.21.15.164:0/2860995990 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33200b4670 con 0x7f332009def0 2024-03-23T14:57:56.020 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:56.013+0000 7f3315ffb640 1 -- 172.21.15.164:0/2860995990 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7f331c0073f0 con 0x7f332009def0 2024-03-23T14:57:56.020 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:56.013+0000 7f3315ffb640 1 --2- 172.21.15.164:0/2860995990 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f33040672a0 0x7f3304069760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:56.020 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:56.013+0000 7f3315ffb640 1 -- 172.21.15.164:0/2860995990 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5795+0+0 (secure 0 0 0) 0x7f331c084580 con 0x7f332009def0 2024-03-23T14:57:56.020 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:56.013+0000 7f3324ff9640 1 --2- 172.21.15.164:0/2860995990 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f33040672a0 0x7f3304069760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:56.020 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:56.013+0000 7f3324ff9640 1 --2- 172.21.15.164:0/2860995990 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f33040672a0 0x7f3304069760 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f33200b43a0 tx=0x7f33100079c0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:56.023 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:56.017+0000 7f3315ffb640 1 -- 172.21.15.164:0/2860995990 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7f331c051160 con 0x7f332009def0 2024-03-23T14:57:56.205 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:56.201+0000 7f3327a5e640 1 -- 172.21.15.164:0/2860995990 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f332014ee20 con 0x7f332009def0 2024-03-23T14:57:56.879 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:56.873+0000 7f3315ffb640 1 -- 172.21.15.164:0/2860995990 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v15) ==== 104+0+0 (secure 0 0 0) 0x7f332014ee20 con 0x7f332009def0 2024-03-23T14:57:56.883 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:56.877+0000 7f3327a5e640 1 -- 172.21.15.164:0/2860995990 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f33040672a0 msgr2=0x7f3304069760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:56.883 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:56.877+0000 7f3327a5e640 1 --2- 172.21.15.164:0/2860995990 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f33040672a0 0x7f3304069760 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f33200b43a0 tx=0x7f33100079c0 comp rx=0 tx=0).stop 2024-03-23T14:57:56.884 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:56.877+0000 7f3327a5e640 1 -- 172.21.15.164:0/2860995990 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f332009def0 msgr2=0x7f33200b3920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:56.884 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:56.877+0000 7f3327a5e640 1 --2- 172.21.15.164:0/2860995990 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f332009def0 0x7f33200b3920 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f331c00c450 tx=0x7f331c00c910 comp rx=0 tx=0).stop 2024-03-23T14:57:56.884 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:56.877+0000 7f3327a5e640 1 -- 172.21.15.164:0/2860995990 shutdown_connections 2024-03-23T14:57:56.884 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:56.877+0000 7f3327a5e640 1 --2- 172.21.15.164:0/2860995990 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f33040672a0 0x7f3304069760 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:56.884 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:56.877+0000 7f3327a5e640 1 --2- 172.21.15.164:0/2860995990 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f33200b4e90 0x7f332014dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:56.884 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:56.877+0000 7f3327a5e640 1 --2- 172.21.15.164:0/2860995990 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f33200b4a00 0x7f33201df900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:56.884 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:56.877+0000 7f3327a5e640 1 --2- 172.21.15.164:0/2860995990 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f332009def0 0x7f33200b3920 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:56.884 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:56.877+0000 7f3327a5e640 1 -- 172.21.15.164:0/2860995990 >> 172.21.15.164:0/2860995990 conn(0x7f33200b8ca0 msgr2=0x7f332006bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:56.884 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:56.881+0000 7f3327a5e640 1 -- 172.21.15.164:0/2860995990 shutdown_connections 2024-03-23T14:57:56.884 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:56.881+0000 7f3327a5e640 1 -- 172.21.15.164:0/2860995990 wait complete. 2024-03-23T14:57:56.906 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-23T14:57:57.081 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:57.077+0000 7f22112ce640 1 Processor -- start 2024-03-23T14:57:57.081 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:57.077+0000 7f22112ce640 1 -- start start 2024-03-23T14:57:57.082 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:57.077+0000 7f22112ce640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f220c09def0 0x7f220c0b45e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:57.082 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:57.077+0000 7f22112ce640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f220c0add50 0x7f220c06bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:57.082 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:57.077+0000 7f22112ce640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f220c06d650 con 0x7f220c0b4b20 2024-03-23T14:57:57.082 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:57.077+0000 7f22112ce640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f220c0af4a0 con 0x7f220c0add50 2024-03-23T14:57:57.082 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:57.077+0000 7f22112ce640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 -- 0x7f220c09e2d0 con 0x7f220c09def0 2024-03-23T14:57:57.082 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:57.077+0000 7f220a7fc640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f220c0add50 0x7f220c06bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:57.082 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:57.077+0000 7f220a7fc640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f220c0add50 0x7f220c06bf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:48760/0 (socket says 172.21.15.164:48760) 2024-03-23T14:57:57.082 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:57.077+0000 7f220a7fc640 1 -- 172.21.15.164:0/3694213315 learned_addr learned my addr 172.21.15.164:0/3694213315 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:57.082 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:57.077+0000 7f220affd640 1 --2- 172.21.15.164:0/3694213315 >> v2:172.21.15.173:3300/0 conn(0x7f220c09def0 0x7f220c0b45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:57.083 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:57.077+0000 7f220a7fc640 1 -- 172.21.15.164:0/3694213315 >> v2:172.21.15.173:3300/0 conn(0x7f220c09def0 msgr2=0x7f220c0b45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:57.083 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:57.077+0000 7f220a7fc640 1 --2- 172.21.15.164:0/3694213315 >> v2:172.21.15.173:3300/0 conn(0x7f220c09def0 0x7f220c0b45e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:57.083 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:57.077+0000 7f220a7fc640 1 -- 172.21.15.164:0/3694213315 >> v1:172.21.15.173:6789/0 conn(0x7f220c0b4b20 legacy=0x7f220c06c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:57.083 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:57.077+0000 7f220a7fc640 1 -- 172.21.15.164:0/3694213315 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f220c0aeca0 con 0x7f220c0add50 2024-03-23T14:57:57.084 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:57.081+0000 7f220a7fc640 1 --2- 172.21.15.164:0/3694213315 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f220c0add50 0x7f220c06bf70 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f220c06da30 tx=0x7f220001ea90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d01bc42fdd6eabd5 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:57.084 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:57.081+0000 7f2209ffb640 1 -- 172.21.15.164:0/3694213315 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22000a79c0 con 0x7f220c0add50 2024-03-23T14:57:57.084 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:57.081+0000 7f2209ffb640 1 -- 172.21.15.164:0/3694213315 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22000a7b20 con 0x7f220c0add50 2024-03-23T14:57:57.084 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:57.081+0000 7f2209ffb640 1 -- 172.21.15.164:0/3694213315 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22000af440 con 0x7f220c0add50 2024-03-23T14:57:57.085 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:57.081+0000 7f22112ce640 1 -- 172.21.15.164:0/3694213315 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f220c0add50 msgr2=0x7f220c06bf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:57.085 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:57.081+0000 7f22112ce640 1 --2- 172.21.15.164:0/3694213315 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f220c0add50 0x7f220c06bf70 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f220c06da30 tx=0x7f220001ea90 comp rx=0 tx=0).stop 2024-03-23T14:57:57.085 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:57.081+0000 7f22112ce640 1 -- 172.21.15.164:0/3694213315 shutdown_connections 2024-03-23T14:57:57.085 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:57.081+0000 7f22112ce640 1 --2- 172.21.15.164:0/3694213315 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f220c0add50 0x7f220c06bf70 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:57.086 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:57.081+0000 7f22112ce640 1 --2- 172.21.15.164:0/3694213315 >> v2:172.21.15.173:3300/0 conn(0x7f220c09def0 0x7f220c0b45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:57.086 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:57.081+0000 7f22112ce640 1 -- 172.21.15.164:0/3694213315 >> 172.21.15.164:0/3694213315 conn(0x7f220c0b8ca0 msgr2=0x7f220c0b90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:57.086 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:57.081+0000 7f22112ce640 1 -- 172.21.15.164:0/3694213315 shutdown_connections 2024-03-23T14:57:57.086 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:57.081+0000 7f22112ce640 1 -- 172.21.15.164:0/3694213315 wait complete. 2024-03-23T14:57:57.088 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:57.081+0000 7f22112ce640 1 Processor -- start 2024-03-23T14:57:57.088 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:57.081+0000 7f22112ce640 1 -- start start 2024-03-23T14:57:57.088 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:57.081+0000 7f22112ce640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f220c09def0 0x7f220c1def70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:57.088 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:57.081+0000 7f22112ce640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f220c0add50 0x7f220c070580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:57.088 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:57.081+0000 7f22112ce640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f220c0b4b20 0x7f220c070ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:57.088 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:57.081+0000 7f22112ce640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f220c06fc80 con 0x7f220c09def0 2024-03-23T14:57:57.088 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:57.081+0000 7f22112ce640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f220c06ff60 con 0x7f220c0add50 2024-03-23T14:57:57.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:57.081+0000 7f22112ce640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f220c13e490 con 0x7f220c0b4b20 2024-03-23T14:57:57.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:57.081+0000 7f220affd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f220c09def0 0x7f220c1def70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:57.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:57.081+0000 7f220affd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f220c09def0 0x7f220c1def70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:48774/0 (socket says 172.21.15.164:48774) 2024-03-23T14:57:57.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:57.081+0000 7f220affd640 1 -- 172.21.15.164:0/1774032451 learned_addr learned my addr 172.21.15.164:0/1774032451 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:57.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:57.081+0000 7f220affd640 1 -- 172.21.15.164:0/1774032451 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f220c0b4b20 msgr2=0x7f220c070ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:57.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:57.081+0000 7f220b7fe640 1 --2- 172.21.15.164:0/1774032451 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f220c0b4b20 0x7f220c070ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:57.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:57.081+0000 7f220a7fc640 1 --2- 172.21.15.164:0/1774032451 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f220c0add50 0x7f220c070580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:57.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:57.081+0000 7f220affd640 1 --2- 172.21.15.164:0/1774032451 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f220c0b4b20 0x7f220c070ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:57.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:57.081+0000 7f220affd640 1 -- 172.21.15.164:0/1774032451 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f220c0add50 msgr2=0x7f220c070580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:57.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:57.081+0000 7f220affd640 1 --2- 172.21.15.164:0/1774032451 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f220c0add50 0x7f220c070580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:57.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:57.081+0000 7f220affd640 1 -- 172.21.15.164:0/1774032451 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f220000b770 con 0x7f220c09def0 2024-03-23T14:57:57.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:57.081+0000 7f220affd640 1 --2- 172.21.15.164:0/1774032451 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f220c09def0 0x7f220c1def70 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f21fc00c7f0 tx=0x7f21fc00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:57.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:57.081+0000 7f21f37fe640 1 -- 172.21.15.164:0/1774032451 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21fc004220 con 0x7f220c09def0 2024-03-23T14:57:57.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:57.085+0000 7f21f37fe640 1 -- 172.21.15.164:0/1774032451 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21fc0043a0 con 0x7f220c09def0 2024-03-23T14:57:57.089 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:57.085+0000 7f22112ce640 1 -- 172.21.15.164:0/1774032451 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f220c13c4a0 con 0x7f220c09def0 2024-03-23T14:57:57.090 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:57.085+0000 7f21f37fe640 1 -- 172.21.15.164:0/1774032451 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21fc004510 con 0x7f220c09def0 2024-03-23T14:57:57.090 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:57.085+0000 7f22112ce640 1 -- 172.21.15.164:0/1774032451 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f220c1dfa50 con 0x7f220c09def0 2024-03-23T14:57:57.090 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:57.085+0000 7f22112ce640 1 -- 172.21.15.164:0/1774032451 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f220c0690a0 con 0x7f220c09def0 2024-03-23T14:57:57.091 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:57.085+0000 7f21f37fe640 1 -- 172.21.15.164:0/1774032451 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7f21fc003a70 con 0x7f220c09def0 2024-03-23T14:57:57.091 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:57.085+0000 7f21f37fe640 1 --2- 172.21.15.164:0/1774032451 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f21f80672d0 0x7f21f8069790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:57.091 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:57.085+0000 7f21f37fe640 1 -- 172.21.15.164:0/1774032451 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5795+0+0 (secure 0 0 0) 0x7f21fc084fa0 con 0x7f220c09def0 2024-03-23T14:57:57.091 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:57.085+0000 7f220a7fc640 1 --2- 172.21.15.164:0/1774032451 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f21f80672d0 0x7f21f8069790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:57.091 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:57.085+0000 7f220a7fc640 1 --2- 172.21.15.164:0/1774032451 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f21f80672d0 0x7f21f8069790 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f22000b1000 tx=0x7f22000a9070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:57.093 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:57.089+0000 7f21f37fe640 1 -- 172.21.15.164:0/1774032451 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7f21fc0b7210 con 0x7f220c09def0 2024-03-23T14:57:57.257 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:57.253+0000 7f22112ce640 1 -- 172.21.15.164:0/1774032451 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f220c0722e0 con 0x7f220c09def0 2024-03-23T14:57:58.220 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:58.217+0000 7f21f37fe640 1 -- 172.21.15.164:0/1774032451 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v31) ==== 181+0+0 (secure 0 0 0) 0x7f21fc01e020 con 0x7f220c09def0 2024-03-23T14:57:58.220 INFO:teuthology.orchestra.run.smithi164.stderr:pool 'cephfs_metadata' removed 2024-03-23T14:57:58.227 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:58.221+0000 7f22112ce640 1 -- 172.21.15.164:0/1774032451 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f21f80672d0 msgr2=0x7f21f8069790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:58.227 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:58.221+0000 7f22112ce640 1 --2- 172.21.15.164:0/1774032451 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f21f80672d0 0x7f21f8069790 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f22000b1000 tx=0x7f22000a9070 comp rx=0 tx=0).stop 2024-03-23T14:57:58.227 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:58.221+0000 7f22112ce640 1 -- 172.21.15.164:0/1774032451 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f220c09def0 msgr2=0x7f220c1def70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:58.227 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:58.221+0000 7f22112ce640 1 --2- 172.21.15.164:0/1774032451 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f220c09def0 0x7f220c1def70 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f21fc00c7f0 tx=0x7f21fc00ccb0 comp rx=0 tx=0).stop 2024-03-23T14:57:58.227 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:58.221+0000 7f22112ce640 1 -- 172.21.15.164:0/1774032451 shutdown_connections 2024-03-23T14:57:58.227 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:58.221+0000 7f22112ce640 1 --2- 172.21.15.164:0/1774032451 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f220c0b4b20 0x7f220c070ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:58.227 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:58.221+0000 7f22112ce640 1 --2- 172.21.15.164:0/1774032451 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f21f80672d0 0x7f21f8069790 secure :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f22000b1000 tx=0x7f22000a9070 comp rx=0 tx=0).stop 2024-03-23T14:57:58.227 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:58.221+0000 7f22112ce640 1 --2- 172.21.15.164:0/1774032451 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f220c0add50 0x7f220c070580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:58.227 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:58.221+0000 7f22112ce640 1 --2- 172.21.15.164:0/1774032451 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f220c09def0 0x7f220c1def70 secure :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f21fc00c7f0 tx=0x7f21fc00ccb0 comp rx=0 tx=0).stop 2024-03-23T14:57:58.228 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:58.221+0000 7f22112ce640 1 -- 172.21.15.164:0/1774032451 >> 172.21.15.164:0/1774032451 conn(0x7f220c0b8ca0 msgr2=0x7f220c141540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:58.228 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:58.225+0000 7f22112ce640 1 -- 172.21.15.164:0/1774032451 shutdown_connections 2024-03-23T14:57:58.228 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:58.225+0000 7f22112ce640 1 -- 172.21.15.164:0/1774032451 wait complete. 2024-03-23T14:57:58.251 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-23T14:57:58.380 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:58.377+0000 7fedc5781640 1 Processor -- start 2024-03-23T14:57:58.380 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:58.377+0000 7fedc5781640 1 -- start start 2024-03-23T14:57:58.381 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:58.377+0000 7fedc5781640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fedc00b6740 0x7fedc00b41d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:58.381 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:58.377+0000 7fedc5781640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fedc00b4710 0x7fedc014fb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:58.381 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:58.377+0000 7fedc5781640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fedc00b5530 con 0x7fedc00b6360 2024-03-23T14:57:58.381 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:58.377+0000 7fedc5781640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fedc00b5a60 con 0x7fedc00b6740 2024-03-23T14:57:58.382 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:58.377+0000 7fedc5781640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fedc0071260 con 0x7fedc00b4710 2024-03-23T14:57:58.382 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:58.377+0000 7fedbe7fc640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fedc00b6740 0x7fedc00b41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:58.382 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:58.377+0000 7fedbe7fc640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fedc00b6740 0x7fedc00b41d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:48790/0 (socket says 172.21.15.164:48790) 2024-03-23T14:57:58.382 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:58.377+0000 7fedbe7fc640 1 -- 172.21.15.164:0/2081378071 learned_addr learned my addr 172.21.15.164:0/2081378071 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:58.382 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:58.377+0000 7fedbf7fe640 1 --2- 172.21.15.164:0/2081378071 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fedc00b4710 0x7fedc014fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:58.382 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:58.377+0000 7fedbe7fc640 1 -- 172.21.15.164:0/2081378071 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fedc00b4710 msgr2=0x7fedc014fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:58.382 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:58.377+0000 7fedbe7fc640 1 --2- 172.21.15.164:0/2081378071 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fedc00b4710 0x7fedc014fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:58.382 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:58.377+0000 7fedbe7fc640 1 -- 172.21.15.164:0/2081378071 >> v1:172.21.15.173:6789/0 conn(0x7fedc00b6360 legacy=0x7fedc0145a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:58.382 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:58.377+0000 7fedbe7fc640 1 -- 172.21.15.164:0/2081378071 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedc00b2a70 con 0x7fedc00b6740 2024-03-23T14:57:58.382 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:58.377+0000 7fedbf7fe640 1 --2- 172.21.15.164:0/2081378071 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fedc00b4710 0x7fedc014fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T14:57:58.383 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:58.377+0000 7fedbe7fc640 1 --2- 172.21.15.164:0/2081378071 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fedc00b6740 0x7fedc00b41d0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fedb4009870 tx=0x7fedb401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5b5c7404b8505d9 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:58.383 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:58.377+0000 7fedbdffb640 1 -- 172.21.15.164:0/2081378071 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedb40ab600 con 0x7fedc00b6740 2024-03-23T14:57:58.383 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:58.377+0000 7fedbdffb640 1 -- 172.21.15.164:0/2081378071 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fedb40a7050 con 0x7fedc00b6740 2024-03-23T14:57:58.383 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:58.377+0000 7fedbdffb640 1 -- 172.21.15.164:0/2081378071 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedb40ab980 con 0x7fedc00b6740 2024-03-23T14:57:58.384 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:58.381+0000 7fedc5781640 1 -- 172.21.15.164:0/2081378071 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fedc00b6740 msgr2=0x7fedc00b41d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:58.384 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:58.381+0000 7fedc5781640 1 --2- 172.21.15.164:0/2081378071 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fedc00b6740 0x7fedc00b41d0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fedb4009870 tx=0x7fedb401c490 comp rx=0 tx=0).stop 2024-03-23T14:57:58.384 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:58.381+0000 7fedc5781640 1 -- 172.21.15.164:0/2081378071 shutdown_connections 2024-03-23T14:57:58.384 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:58.381+0000 7fedc5781640 1 --2- 172.21.15.164:0/2081378071 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fedc00b4710 0x7fedc014fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:58.385 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:58.381+0000 7fedc5781640 1 --2- 172.21.15.164:0/2081378071 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fedc00b6740 0x7fedc00b41d0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:58.385 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:58.381+0000 7fedc5781640 1 -- 172.21.15.164:0/2081378071 >> 172.21.15.164:0/2081378071 conn(0x7fedc00b7630 msgr2=0x7fedc00b7a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:58.385 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:58.381+0000 7fedc5781640 1 -- 172.21.15.164:0/2081378071 shutdown_connections 2024-03-23T14:57:58.385 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:58.381+0000 7fedc5781640 1 -- 172.21.15.164:0/2081378071 wait complete. 2024-03-23T14:57:58.387 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:58.381+0000 7fedc5781640 1 Processor -- start 2024-03-23T14:57:58.387 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:58.381+0000 7fedc5781640 1 -- start start 2024-03-23T14:57:58.387 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:58.381+0000 7fedc5781640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fedc00b4710 0x7fedc006b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:58.387 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:58.381+0000 7fedc5781640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fedc00b6360 0x7fedc0138860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:58.387 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:58.381+0000 7fedc5781640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fedc00b6740 0x7fedc0138da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:58.388 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:58.381+0000 7fedc5781640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fedc013f320 con 0x7fedc00b4710 2024-03-23T14:57:58.388 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:58.381+0000 7fedc5781640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fedc013f110 con 0x7fedc00b6360 2024-03-23T14:57:58.388 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:58.381+0000 7fedc5781640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fedc006cf90 con 0x7fedc00b6740 2024-03-23T14:57:58.388 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:58.381+0000 7fedbeffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fedc00b4710 0x7fedc006b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:58.388 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:58.381+0000 7fedbeffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fedc00b4710 0x7fedc006b520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:48794/0 (socket says 172.21.15.164:48794) 2024-03-23T14:57:58.388 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:58.381+0000 7fedbeffd640 1 -- 172.21.15.164:0/3036541473 learned_addr learned my addr 172.21.15.164:0/3036541473 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:58.388 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:58.381+0000 7fedbe7fc640 1 --2- 172.21.15.164:0/3036541473 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fedc00b6360 0x7fedc0138860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:58.388 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:58.381+0000 7fedbf7fe640 1 --2- 172.21.15.164:0/3036541473 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fedc00b6740 0x7fedc0138da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:58.388 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:58.381+0000 7fedbeffd640 1 -- 172.21.15.164:0/3036541473 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fedc00b6740 msgr2=0x7fedc0138da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:58.388 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:58.381+0000 7fedbeffd640 1 --2- 172.21.15.164:0/3036541473 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fedc00b6740 0x7fedc0138da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:58.388 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:58.381+0000 7fedbeffd640 1 -- 172.21.15.164:0/3036541473 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fedc00b6360 msgr2=0x7fedc0138860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:58.388 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:58.381+0000 7fedbeffd640 1 --2- 172.21.15.164:0/3036541473 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fedc00b6360 0x7fedc0138860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:58.388 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:58.381+0000 7fedbeffd640 1 -- 172.21.15.164:0/3036541473 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedb40094d0 con 0x7fedc00b4710 2024-03-23T14:57:58.388 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:58.381+0000 7fedbeffd640 1 --2- 172.21.15.164:0/3036541473 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fedc00b4710 0x7fedc006b520 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fedb000c450 tx=0x7fedb000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:58.389 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:58.381+0000 7feda77fe640 1 -- 172.21.15.164:0/3036541473 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedb0012830 con 0x7fedc00b4710 2024-03-23T14:57:58.389 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:58.381+0000 7feda77fe640 1 -- 172.21.15.164:0/3036541473 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fedb00129b0 con 0x7fedc00b4710 2024-03-23T14:57:58.389 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:58.381+0000 7feda77fe640 1 -- 172.21.15.164:0/3036541473 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedb0012d60 con 0x7fedc00b4710 2024-03-23T14:57:58.389 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:58.381+0000 7fedc5781640 1 -- 172.21.15.164:0/3036541473 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedc006c820 con 0x7fedc00b4710 2024-03-23T14:57:58.389 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:58.381+0000 7fedc5781640 1 -- 172.21.15.164:0/3036541473 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedc006bdd0 con 0x7fedc00b4710 2024-03-23T14:57:58.389 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:58.385+0000 7fedc5781640 1 -- 172.21.15.164:0/3036541473 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fedc006d340 con 0x7fedc00b4710 2024-03-23T14:57:58.390 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:58.385+0000 7feda77fe640 1 -- 172.21.15.164:0/3036541473 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7fedb001c460 con 0x7fedc00b4710 2024-03-23T14:57:58.390 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:58.385+0000 7feda77fe640 1 --2- 172.21.15.164:0/3036541473 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fedac0672a0 0x7fedac069760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:58.390 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:58.385+0000 7fedbe7fc640 1 --2- 172.21.15.164:0/3036541473 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fedac0672a0 0x7fedac069760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:58.390 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:58.385+0000 7feda77fe640 1 -- 172.21.15.164:0/3036541473 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5357+0+0 (secure 0 0 0) 0x7fedb0084d00 con 0x7fedc00b4710 2024-03-23T14:57:58.390 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:58.385+0000 7fedbe7fc640 1 --2- 172.21.15.164:0/3036541473 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fedac0672a0 0x7fedac069760 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fedb40ad000 tx=0x7fedb40a6000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:58.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:58.389+0000 7feda77fe640 1 -- 172.21.15.164:0/3036541473 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7fedb0051aa0 con 0x7fedc00b4710 2024-03-23T14:57:58.554 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:58.549+0000 7fedc5781640 1 -- 172.21.15.164:0/3036541473 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fedc006cb00 con 0x7fedc00b4710 2024-03-23T14:57:58.856 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-23T14:57:58.859 DEBUG:teuthology.orchestra.run.smithi173:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-23T14:57:59.230 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.225+0000 7feda77fe640 1 -- 172.21.15.164:0/3036541473 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v32) ==== 169+0+0 (secure 0 0 0) 0x7fedb0055160 con 0x7fedc00b4710 2024-03-23T14:57:59.231 INFO:teuthology.orchestra.run.smithi164.stderr:pool 'cephfs_data' removed 2024-03-23T14:57:59.234 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.229+0000 7fedc5781640 1 -- 172.21.15.164:0/3036541473 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fedac0672a0 msgr2=0x7fedac069760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:59.234 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.229+0000 7fedc5781640 1 --2- 172.21.15.164:0/3036541473 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fedac0672a0 0x7fedac069760 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fedb40ad000 tx=0x7fedb40a6000 comp rx=0 tx=0).stop 2024-03-23T14:57:59.234 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.229+0000 7fedc5781640 1 -- 172.21.15.164:0/3036541473 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fedc00b4710 msgr2=0x7fedc006b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:59.234 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.229+0000 7fedc5781640 1 --2- 172.21.15.164:0/3036541473 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fedc00b4710 0x7fedc006b520 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fedb000c450 tx=0x7fedb000c910 comp rx=0 tx=0).stop 2024-03-23T14:57:59.234 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.229+0000 7fedc5781640 1 -- 172.21.15.164:0/3036541473 shutdown_connections 2024-03-23T14:57:59.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.229+0000 7fedc5781640 1 --2- 172.21.15.164:0/3036541473 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fedc00b6740 0x7fedc0138da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:59.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.229+0000 7fedc5781640 1 --2- 172.21.15.164:0/3036541473 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fedac0672a0 0x7fedac069760 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:59.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.229+0000 7fedc5781640 1 --2- 172.21.15.164:0/3036541473 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fedc00b6360 0x7fedc0138860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:59.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.229+0000 7fedc5781640 1 --2- 172.21.15.164:0/3036541473 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fedc00b4710 0x7fedc006b520 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:59.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.229+0000 7fedc5781640 1 -- 172.21.15.164:0/3036541473 >> 172.21.15.164:0/3036541473 conn(0x7fedc00b7630 msgr2=0x7fedc0143260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:59.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.229+0000 7fedc5781640 1 -- 172.21.15.164:0/3036541473 shutdown_connections 2024-03-23T14:57:59.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.229+0000 7fedc5781640 1 -- 172.21.15.164:0/3036541473 wait complete. 2024-03-23T14:57:59.257 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-03-23T14:57:59.382 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.377+0000 7f49489e9640 1 Processor -- start 2024-03-23T14:57:59.382 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.377+0000 7f49489e9640 1 -- start start 2024-03-23T14:57:59.385 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.377+0000 7f49489e9640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f49440add50 0x7f4944147770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:59.385 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.377+0000 7f49489e9640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4944147cb0 0x7f49441480e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:59.385 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.377+0000 7f49489e9640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f494406d650 con 0x7f494409def0 2024-03-23T14:57:59.385 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.377+0000 7f49489e9640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 -- 0x7f4944148620 con 0x7f49440add50 2024-03-23T14:57:59.385 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.377+0000 7f49489e9640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f49440af4a0 con 0x7f4944147cb0 2024-03-23T14:57:59.386 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.377+0000 7f494259c640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f494409def0 0x7f49440b45e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:43154/0 (socket says 172.21.15.164:43154) 2024-03-23T14:57:59.386 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.377+0000 7f494259c640 1 -- 172.21.15.164:0/2765871859 learned_addr learned my addr 172.21.15.164:0/2765871859 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-03-23T14:57:59.386 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.377+0000 7f494159a640 1 -- 172.21.15.164:0/2765871859 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4224200630 0 0) 0x7f494406d650 con 0x7f494409def0 2024-03-23T14:57:59.386 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.377+0000 7f4942d9d640 1 --2- 172.21.15.164:0/2765871859 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4944147cb0 0x7f49441480e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:59.386 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.377+0000 7f494159a640 1 -- 172.21.15.164:0/2765871859 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4930003530 con 0x7f494409def0 2024-03-23T14:57:59.386 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.377+0000 7f4941d9b640 1 --2- 172.21.15.164:0/2765871859 >> v2:172.21.15.173:3300/0 conn(0x7f49440add50 0x7f4944147770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:59.386 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.377+0000 7f494159a640 1 -- 172.21.15.164:0/2765871859 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2792928630 0 0) 0x7f4930003530 con 0x7f494409def0 2024-03-23T14:57:59.386 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.377+0000 7f494159a640 1 -- 172.21.15.164:0/2765871859 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4944147cb0 msgr2=0x7f49441480e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:59.386 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.377+0000 7f494159a640 1 --2- 172.21.15.164:0/2765871859 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4944147cb0 0x7f49441480e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:59.386 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.377+0000 7f494159a640 1 -- 172.21.15.164:0/2765871859 >> v2:172.21.15.173:3300/0 conn(0x7f49440add50 msgr2=0x7f4944147770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:59.387 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.377+0000 7f494159a640 1 --2- 172.21.15.164:0/2765871859 >> v2:172.21.15.173:3300/0 conn(0x7f49440add50 0x7f4944147770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:59.387 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.381+0000 7f494159a640 1 -- 172.21.15.164:0/2765871859 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f494409e2d0 con 0x7f494409def0 2024-03-23T14:57:59.387 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.381+0000 7f494159a640 1 -- 172.21.15.164:0/2765871859 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2526390548 0 0) 0x7f492c001680 con 0x7f494409def0 2024-03-23T14:57:59.387 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.381+0000 7f4941d9b640 1 --2- 172.21.15.164:0/2765871859 >> v2:172.21.15.173:3300/0 conn(0x7f49440add50 0x7f4944147770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T14:57:59.387 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.381+0000 7f494159a640 1 -- 172.21.15.164:0/2765871859 >> v1:172.21.15.164:6789/0 conn(0x7f494409def0 legacy=0x7f49440b45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:59.387 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.381+0000 7f494159a640 1 --2- 172.21.15.164:0/2765871859 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4930003980 0x7f4930003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:59.387 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.381+0000 7f494159a640 1 --2- 172.21.15.164:0/2765871859 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f49300047d0 0x7f4930004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:59.387 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.381+0000 7f494159a640 1 --2- 172.21.15.164:0/2765871859 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4944147cb0 0x7f4930005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:59.387 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.381+0000 7f494159a640 1 -- 172.21.15.164:0/2765871859 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f4944148620 con 0x7f4930003980 2024-03-23T14:57:59.387 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.381+0000 7f494159a640 1 -- 172.21.15.164:0/2765871859 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f49440af4a0 con 0x7f49300047d0 2024-03-23T14:57:59.387 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.381+0000 7f494159a640 1 -- 172.21.15.164:0/2765871859 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f493000bac0 con 0x7f4944147cb0 2024-03-23T14:57:59.388 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.381+0000 7f494259c640 1 --2- 172.21.15.164:0/2765871859 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4930003980 0x7f4930003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:59.388 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.381+0000 7f4941d9b640 1 --2- 172.21.15.164:0/2765871859 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f49300047d0 0x7f4930004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:59.388 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.381+0000 7f4942d9d640 1 --2- 172.21.15.164:0/2765871859 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4944147cb0 0x7f4930005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:59.388 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.381+0000 7f494259c640 1 -- 172.21.15.164:0/2765871859 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4944147cb0 msgr2=0x7f4930005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:59.388 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.381+0000 7f494259c640 1 --2- 172.21.15.164:0/2765871859 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4944147cb0 0x7f4930005190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:59.388 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.381+0000 7f494259c640 1 -- 172.21.15.164:0/2765871859 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f49300047d0 msgr2=0x7f4930004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:59.388 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.381+0000 7f494259c640 1 --2- 172.21.15.164:0/2765871859 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f49300047d0 0x7f4930004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:59.388 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.381+0000 7f494259c640 1 -- 172.21.15.164:0/2765871859 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4930003530 con 0x7f4930003980 2024-03-23T14:57:59.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.381+0000 7f494259c640 1 --2- 172.21.15.164:0/2765871859 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4930003980 0x7f4930003da0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f492c00c210 tx=0x7f492c01cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea9a162429e2952f server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:59.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.381+0000 7f494159a640 1 -- 172.21.15.164:0/2765871859 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f492c0058e0 con 0x7f4930003980 2024-03-23T14:57:59.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.381+0000 7f494159a640 1 -- 172.21.15.164:0/2765871859 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f492c005a40 con 0x7f4930003980 2024-03-23T14:57:59.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.381+0000 7f494159a640 1 -- 172.21.15.164:0/2765871859 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f492c005e10 con 0x7f4930003980 2024-03-23T14:57:59.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.381+0000 7f49489e9640 1 -- 172.21.15.164:0/2765871859 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4930003980 msgr2=0x7f4930003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:59.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.381+0000 7f49489e9640 1 --2- 172.21.15.164:0/2765871859 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4930003980 0x7f4930003da0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f492c00c210 tx=0x7f492c01cc80 comp rx=0 tx=0).stop 2024-03-23T14:57:59.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.381+0000 7f4942d9d640 1 -- 172.21.15.164:0/2765871859 reap_dead start 2024-03-23T14:57:59.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.385+0000 7f49489e9640 1 -- 172.21.15.164:0/2765871859 shutdown_connections 2024-03-23T14:57:59.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.385+0000 7f49489e9640 1 -- 172.21.15.164:0/2765871859 >> 172.21.15.164:0/2765871859 conn(0x7f49440b8ca0 msgr2=0x7f49440b90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:59.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.385+0000 7f49489e9640 1 -- 172.21.15.164:0/2765871859 shutdown_connections 2024-03-23T14:57:59.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.385+0000 7f49489e9640 1 -- 172.21.15.164:0/2765871859 wait complete. 2024-03-23T14:57:59.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.385+0000 7f49489e9640 1 Processor -- start 2024-03-23T14:57:59.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.385+0000 7f49489e9640 1 -- start start 2024-03-23T14:57:59.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.385+0000 7f49489e9640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4930003980 0x7f49440b80a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:59.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.385+0000 7f49489e9640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f494406f490 0x7f494413b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:59.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.385+0000 7f49489e9640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f49441e0cc0 0x7f4944134bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:59.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.385+0000 7f49489e9640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f49441432f0 con 0x7f4930003980 2024-03-23T14:57:59.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.385+0000 7f49489e9640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f49441e1140 con 0x7f49441e0cc0 2024-03-23T14:57:59.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.385+0000 7f49489e9640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f494406f9c0 con 0x7f494406f490 2024-03-23T14:57:59.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.385+0000 7f494259c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4930003980 0x7f49440b80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:59.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.385+0000 7f494259c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4930003980 0x7f49440b80a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:48816/0 (socket says 172.21.15.164:48816) 2024-03-23T14:57:59.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.385+0000 7f494259c640 1 -- 172.21.15.164:0/1794846828 learned_addr learned my addr 172.21.15.164:0/1794846828 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:59.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.385+0000 7f4941d9b640 1 --2- 172.21.15.164:0/1794846828 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f494406f490 0x7f494413b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:59.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.385+0000 7f4942d9d640 1 --2- 172.21.15.164:0/1794846828 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f49441e0cc0 0x7f4944134bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:59.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.385+0000 7f494259c640 1 -- 172.21.15.164:0/1794846828 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f494406f490 msgr2=0x7f494413b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:59.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.385+0000 7f494259c640 1 --2- 172.21.15.164:0/1794846828 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f494406f490 0x7f494413b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:59.396 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.385+0000 7f494259c640 1 -- 172.21.15.164:0/1794846828 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f49441e0cc0 msgr2=0x7f4944134bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:59.396 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.385+0000 7f494259c640 1 --2- 172.21.15.164:0/1794846828 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f49441e0cc0 0x7f4944134bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:59.396 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.385+0000 7f494259c640 1 -- 172.21.15.164:0/1794846828 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f494406fca0 con 0x7f4930003980 2024-03-23T14:57:59.396 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.385+0000 7f494259c640 1 --2- 172.21.15.164:0/1794846828 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4930003980 0x7f49440b80a0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f492c0ab040 tx=0x7f4930005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:59.396 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.385+0000 7f493affd640 1 -- 172.21.15.164:0/1794846828 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f492c005e70 con 0x7f4930003980 2024-03-23T14:57:59.396 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.385+0000 7f493affd640 1 -- 172.21.15.164:0/1794846828 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f492c0a5870 con 0x7f4930003980 2024-03-23T14:57:59.396 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.385+0000 7f49489e9640 1 -- 172.21.15.164:0/1794846828 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49440b75c0 con 0x7f4930003980 2024-03-23T14:57:59.396 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.385+0000 7f493affd640 1 -- 172.21.15.164:0/1794846828 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f492c0a5c00 con 0x7f4930003980 2024-03-23T14:57:59.396 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.385+0000 7f49489e9640 1 -- 172.21.15.164:0/1794846828 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49440b7b80 con 0x7f4930003980 2024-03-23T14:57:59.396 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.385+0000 7f49489e9640 1 -- 172.21.15.164:0/1794846828 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49440690a0 con 0x7f4930003980 2024-03-23T14:57:59.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.389+0000 7f493affd640 1 -- 172.21.15.164:0/1794846828 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7f492c0047f0 con 0x7f4930003980 2024-03-23T14:57:59.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.389+0000 7f493affd640 1 --2- 172.21.15.164:0/1794846828 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f49240672d0 0x7f4924069790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:59.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.389+0000 7f4941d9b640 1 --2- 172.21.15.164:0/1794846828 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f49240672d0 0x7f4924069790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:59.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.389+0000 7f493affd640 1 -- 172.21.15.164:0/1794846828 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4943+0+0 (secure 0 0 0) 0x7f492c11e6d0 con 0x7f4930003980 2024-03-23T14:57:59.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.389+0000 7f4941d9b640 1 --2- 172.21.15.164:0/1794846828 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f49240672d0 0x7f4924069790 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f4944143160 tx=0x7f4934008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:59.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.393+0000 7f493affd640 1 -- 172.21.15.164:0/1794846828 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7f492c0eb610 con 0x7f4930003980 2024-03-23T14:57:59.559 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.553+0000 7f49489e9640 1 -- 172.21.15.164:0/1794846828 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f49440b2d60 con 0x7f4930003980 2024-03-23T14:57:59.564 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.561+0000 7f493affd640 1 -- 172.21.15.164:0/1794846828 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) ==== 104+0+0 (secure 0 0 0) 0x7f492c0b7e90 con 0x7f4930003980 2024-03-23T14:57:59.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.561+0000 7f49489e9640 1 -- 172.21.15.164:0/1794846828 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f49240672d0 msgr2=0x7f4924069790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:59.567 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.561+0000 7f49489e9640 1 --2- 172.21.15.164:0/1794846828 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f49240672d0 0x7f4924069790 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f4944143160 tx=0x7f4934008040 comp rx=0 tx=0).stop 2024-03-23T14:57:59.568 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.561+0000 7f49489e9640 1 -- 172.21.15.164:0/1794846828 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4930003980 msgr2=0x7f49440b80a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:59.568 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.561+0000 7f49489e9640 1 --2- 172.21.15.164:0/1794846828 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4930003980 0x7f49440b80a0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f492c0ab040 tx=0x7f4930005800 comp rx=0 tx=0).stop 2024-03-23T14:57:59.568 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.561+0000 7f49489e9640 1 -- 172.21.15.164:0/1794846828 shutdown_connections 2024-03-23T14:57:59.568 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.561+0000 7f49489e9640 1 --2- 172.21.15.164:0/1794846828 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f49240672d0 0x7f4924069790 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:59.568 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.561+0000 7f49489e9640 1 --2- 172.21.15.164:0/1794846828 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f49441e0cc0 0x7f4944134bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:59.568 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.561+0000 7f49489e9640 1 --2- 172.21.15.164:0/1794846828 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f494406f490 0x7f494413b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:59.568 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.561+0000 7f49489e9640 1 --2- 172.21.15.164:0/1794846828 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4930003980 0x7f49440b80a0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:59.568 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.561+0000 7f49489e9640 1 -- 172.21.15.164:0/1794846828 >> 172.21.15.164:0/1794846828 conn(0x7f49440b8ca0 msgr2=0x7f49440b5cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:59.568 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.561+0000 7f49489e9640 1 -- 172.21.15.164:0/1794846828 shutdown_connections 2024-03-23T14:57:59.568 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.561+0000 7f49489e9640 1 -- 172.21.15.164:0/1794846828 wait complete. 2024-03-23T14:57:59.584 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_cephfs_shell.TestCD.test_cd_with_args' 2024-03-23T14:57:59.700 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.697+0000 7fb7656b9640 1 Processor -- start 2024-03-23T14:57:59.701 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.697+0000 7fb7656b9640 1 -- start start 2024-03-23T14:57:59.703 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.697+0000 7fb7656b9640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb7600725b0 0x7fb760148890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:59.703 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.697+0000 7fb7656b9640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fb7600b3990 0x7fb76014fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:59.703 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.697+0000 7fb7656b9640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7600b0ca0 con 0x7fb76006e730 2024-03-23T14:57:59.703 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.697+0000 7fb7656b9640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 -- 0x7fb760072990 con 0x7fb7600b3990 2024-03-23T14:57:59.703 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.697+0000 7fb7656b9640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fb7600b95f0 con 0x7fb7600725b0 2024-03-23T14:57:59.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.697+0000 7fb7567fc640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fb76006e730 0x7fb7600b3280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:43164/0 (socket says 172.21.15.164:43164) 2024-03-23T14:57:59.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.697+0000 7fb7567fc640 1 -- 172.21.15.164:0/2505325958 learned_addr learned my addr 172.21.15.164:0/2505325958 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-03-23T14:57:59.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.697+0000 7fb75f7fe640 1 --2- 172.21.15.164:0/2505325958 >> v2:172.21.15.173:3300/0 conn(0x7fb7600b3990 0x7fb76014fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:59.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.697+0000 7fb75effd640 1 --2- 172.21.15.164:0/2505325958 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb7600725b0 0x7fb760148890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:59.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.697+0000 7fb75e7fc640 1 -- 172.21.15.164:0/2505325958 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2993998382 0 0) 0x7fb748001120 con 0x7fb76006e730 2024-03-23T14:57:59.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.697+0000 7fb75e7fc640 1 -- 172.21.15.164:0/2505325958 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb74c003530 con 0x7fb76006e730 2024-03-23T14:57:59.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.697+0000 7fb75e7fc640 1 -- 172.21.15.164:0/2505325958 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1810861586 0 0) 0x7fb74c003530 con 0x7fb76006e730 2024-03-23T14:57:59.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.697+0000 7fb75e7fc640 1 -- 172.21.15.164:0/2505325958 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb7600725b0 msgr2=0x7fb760148890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:59.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.697+0000 7fb75e7fc640 1 --2- 172.21.15.164:0/2505325958 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb7600725b0 0x7fb760148890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:59.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.697+0000 7fb75e7fc640 1 -- 172.21.15.164:0/2505325958 >> v2:172.21.15.173:3300/0 conn(0x7fb7600b3990 msgr2=0x7fb76014fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:59.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.697+0000 7fb75e7fc640 1 --2- 172.21.15.164:0/2505325958 >> v2:172.21.15.173:3300/0 conn(0x7fb7600b3990 0x7fb76014fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:59.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.697+0000 7fb75e7fc640 1 -- 172.21.15.164:0/2505325958 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb760148dd0 con 0x7fb76006e730 2024-03-23T14:57:59.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.697+0000 7fb75e7fc640 1 -- 172.21.15.164:0/2505325958 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2526390548 0 0) 0x7fb748001810 con 0x7fb76006e730 2024-03-23T14:57:59.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.697+0000 7fb75e7fc640 1 -- 172.21.15.164:0/2505325958 >> v1:172.21.15.164:6789/0 conn(0x7fb76006e730 legacy=0x7fb7600b3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:59.704 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.697+0000 7fb75e7fc640 1 --2- 172.21.15.164:0/2505325958 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb74c003980 0x7fb74c003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:59.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.697+0000 7fb75e7fc640 1 --2- 172.21.15.164:0/2505325958 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb7600725b0 0x7fb74c0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:59.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.697+0000 7fb75e7fc640 1 --2- 172.21.15.164:0/2505325958 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb74c004d10 0x7fb74c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:59.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.697+0000 7fb75e7fc640 1 -- 172.21.15.164:0/2505325958 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb760072990 con 0x7fb74c004d10 2024-03-23T14:57:59.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.697+0000 7fb75e7fc640 1 -- 172.21.15.164:0/2505325958 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fb7600b95f0 con 0x7fb74c003980 2024-03-23T14:57:59.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.697+0000 7fb75e7fc640 1 -- 172.21.15.164:0/2505325958 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fb74c00baf0 con 0x7fb7600725b0 2024-03-23T14:57:59.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.697+0000 7fb75f7fe640 1 --2- 172.21.15.164:0/2505325958 >> v2:172.21.15.173:3300/0 conn(0x7fb7600b3990 0x7fb76014fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T14:57:59.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.697+0000 7fb75f7fe640 1 --2- 172.21.15.164:0/2505325958 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb74c004d10 0x7fb74c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:59.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.697+0000 7fb75effd640 1 --2- 172.21.15.164:0/2505325958 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb74c003980 0x7fb74c003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:59.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.697+0000 7fb7567fc640 1 --2- 172.21.15.164:0/2505325958 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb7600725b0 0x7fb74c0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:59.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.697+0000 7fb75f7fe640 1 -- 172.21.15.164:0/2505325958 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb7600725b0 msgr2=0x7fb74c0047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:59.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.697+0000 7fb75f7fe640 1 --2- 172.21.15.164:0/2505325958 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb7600725b0 0x7fb74c0047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:59.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.697+0000 7fb75f7fe640 1 -- 172.21.15.164:0/2505325958 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb74c003980 msgr2=0x7fb74c003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:57:59.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.697+0000 7fb75f7fe640 1 --2- 172.21.15.164:0/2505325958 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb74c003980 0x7fb74c003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:59.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.697+0000 7fb75f7fe640 1 -- 172.21.15.164:0/2505325958 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb74c003530 con 0x7fb74c004d10 2024-03-23T14:57:59.705 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.697+0000 7fb7567fc640 1 --2- 172.21.15.164:0/2505325958 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb7600725b0 0x7fb74c0047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T14:57:59.708 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.701+0000 7fb75f7fe640 1 --2- 172.21.15.164:0/2505325958 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb74c004d10 0x7fb74c005190 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fb7580073e0 tx=0x7fb75801cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49ea8468df8d53c5 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:59.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.701+0000 7fb75e7fc640 1 -- 172.21.15.164:0/2505325958 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7580a5d30 con 0x7fb74c004d10 2024-03-23T14:57:59.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.701+0000 7fb75e7fc640 1 -- 172.21.15.164:0/2505325958 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7580a5e90 con 0x7fb74c004d10 2024-03-23T14:57:59.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.701+0000 7fb75e7fc640 1 -- 172.21.15.164:0/2505325958 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7580ad640 con 0x7fb74c004d10 2024-03-23T14:57:59.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.701+0000 7fb7656b9640 1 -- 172.21.15.164:0/2505325958 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb74c004d10 msgr2=0x7fb74c005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:59.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.701+0000 7fb7656b9640 1 --2- 172.21.15.164:0/2505325958 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb74c004d10 0x7fb74c005190 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fb7580073e0 tx=0x7fb75801cf40 comp rx=0 tx=0).stop 2024-03-23T14:57:59.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.701+0000 7fb75f7fe640 1 -- 172.21.15.164:0/2505325958 reap_dead start 2024-03-23T14:57:59.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.701+0000 7fb7656b9640 1 -- 172.21.15.164:0/2505325958 shutdown_connections 2024-03-23T14:57:59.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.701+0000 7fb7656b9640 1 -- 172.21.15.164:0/2505325958 >> 172.21.15.164:0/2505325958 conn(0x7fb7600b7970 msgr2=0x7fb7600b7d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:57:59.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.701+0000 7fb7656b9640 1 -- 172.21.15.164:0/2505325958 shutdown_connections 2024-03-23T14:57:59.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.701+0000 7fb7656b9640 1 -- 172.21.15.164:0/2505325958 wait complete. 2024-03-23T14:57:59.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.701+0000 7fb7656b9640 1 Processor -- start 2024-03-23T14:57:59.709 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.701+0000 7fb7656b9640 1 -- start start 2024-03-23T14:57:59.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.701+0000 7fb7656b9640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb7600adb30 0x7fb7601dcc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:59.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.701+0000 7fb7656b9640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb7601dd170 0x7fb7601e01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:59.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.701+0000 7fb7656b9640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb7601e06e0 0x7fb7601e0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:59.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.701+0000 7fb7656b9640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb760071330 con 0x7fb7601dd170 2024-03-23T14:57:59.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.701+0000 7fb7656b9640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fb760142f80 con 0x7fb7601e06e0 2024-03-23T14:57:59.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.701+0000 7fb7656b9640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fb7600aeee0 con 0x7fb7600adb30 2024-03-23T14:57:59.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.701+0000 7fb7567fc640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb7601dd170 0x7fb7601e01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:59.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.701+0000 7fb7567fc640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb7601dd170 0x7fb7601e01a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:48832/0 (socket says 172.21.15.164:48832) 2024-03-23T14:57:59.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.701+0000 7fb7567fc640 1 -- 172.21.15.164:0/2292330232 learned_addr learned my addr 172.21.15.164:0/2292330232 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:57:59.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.701+0000 7fb75f7fe640 1 --2- 172.21.15.164:0/2292330232 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb7601e06e0 0x7fb7601e0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:59.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.701+0000 7fb75effd640 1 --2- 172.21.15.164:0/2292330232 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb7600adb30 0x7fb7601dcc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:59.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.701+0000 7fb7567fc640 1 -- 172.21.15.164:0/2292330232 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb7600adb30 msgr2=0x7fb7601dcc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:59.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.701+0000 7fb7567fc640 1 --2- 172.21.15.164:0/2292330232 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb7600adb30 0x7fb7601dcc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:59.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.701+0000 7fb7567fc640 1 -- 172.21.15.164:0/2292330232 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb7601e06e0 msgr2=0x7fb7601e0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:57:59.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.701+0000 7fb7567fc640 1 --2- 172.21.15.164:0/2292330232 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb7601e06e0 0x7fb7601e0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:57:59.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.701+0000 7fb7567fc640 1 -- 172.21.15.164:0/2292330232 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb760142d00 con 0x7fb7601dd170 2024-03-23T14:57:59.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.701+0000 7fb75effd640 1 --2- 172.21.15.164:0/2292330232 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb7600adb30 0x7fb7601dcc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T14:57:59.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.705+0000 7fb7567fc640 1 --2- 172.21.15.164:0/2292330232 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb7601dd170 0x7fb7601e01a0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fb748001ec0 tx=0x7fb748010ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:59.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.705+0000 7fb757fff640 1 -- 172.21.15.164:0/2292330232 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb74800ec70 con 0x7fb7601dd170 2024-03-23T14:57:59.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.705+0000 7fb757fff640 1 -- 172.21.15.164:0/2292330232 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb74800edd0 con 0x7fb7601dd170 2024-03-23T14:57:59.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.705+0000 7fb7656b9640 1 -- 172.21.15.164:0/2292330232 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7600adf10 con 0x7fb7601dd170 2024-03-23T14:57:59.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.705+0000 7fb757fff640 1 -- 172.21.15.164:0/2292330232 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7480145a0 con 0x7fb7601dd170 2024-03-23T14:57:59.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.705+0000 7fb7656b9640 1 -- 172.21.15.164:0/2292330232 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7600af150 con 0x7fb7601dd170 2024-03-23T14:57:59.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.705+0000 7fb7656b9640 1 -- 172.21.15.164:0/2292330232 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7600690a0 con 0x7fb7601dd170 2024-03-23T14:57:59.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.705+0000 7fb757fff640 1 -- 172.21.15.164:0/2292330232 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7fb748014700 con 0x7fb7601dd170 2024-03-23T14:57:59.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.705+0000 7fb757fff640 1 --2- 172.21.15.164:0/2292330232 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fb7400672a0 0x7fb740069760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:57:59.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.705+0000 7fb757fff640 1 -- 172.21.15.164:0/2292330232 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4943+0+0 (secure 0 0 0) 0x7fb748087980 con 0x7fb7601dd170 2024-03-23T14:57:59.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.705+0000 7fb75effd640 1 --2- 172.21.15.164:0/2292330232 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fb7400672a0 0x7fb740069760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:57:59.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.705+0000 7fb75effd640 1 --2- 172.21.15.164:0/2292330232 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fb7400672a0 0x7fb740069760 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb75000e870 tx=0x7fb750008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:57:59.714 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.709+0000 7fb757fff640 1 -- 172.21.15.164:0/2292330232 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7fb7480548c0 con 0x7fb7601dd170 2024-03-23T14:57:59.926 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:57:59.921+0000 7fb7656b9640 1 -- 172.21.15.164:0/2292330232 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_cephfs_shell.TestCD.test_cd_with_args"]} v 0) -- 0x7fb7600b0ca0 con 0x7fb7601dd170 2024-03-23T14:58:00.232 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:00.229+0000 7fb757fff640 1 -- 172.21.15.164:0/2292330232 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_cephfs_shell.TestCD.test_cd_with_args"]}]=0 v82) ==== 134+0+0 (secure 0 0 0) 0x7fb7600b0ca0 con 0x7fb7601dd170 2024-03-23T14:58:00.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:00.229+0000 7fb7656b9640 1 -- 172.21.15.164:0/2292330232 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fb7400672a0 msgr2=0x7fb740069760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:00.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:00.229+0000 7fb7656b9640 1 --2- 172.21.15.164:0/2292330232 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fb7400672a0 0x7fb740069760 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb75000e870 tx=0x7fb750008040 comp rx=0 tx=0).stop 2024-03-23T14:58:00.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:00.229+0000 7fb7656b9640 1 -- 172.21.15.164:0/2292330232 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb7601dd170 msgr2=0x7fb7601e01a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:00.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:00.229+0000 7fb7656b9640 1 --2- 172.21.15.164:0/2292330232 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb7601dd170 0x7fb7601e01a0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fb748001ec0 tx=0x7fb748010ef0 comp rx=0 tx=0).stop 2024-03-23T14:58:00.236 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:00.229+0000 7fb7656b9640 1 -- 172.21.15.164:0/2292330232 shutdown_connections 2024-03-23T14:58:00.236 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:00.229+0000 7fb7656b9640 1 --2- 172.21.15.164:0/2292330232 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fb7400672a0 0x7fb740069760 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:00.236 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:00.229+0000 7fb7656b9640 1 --2- 172.21.15.164:0/2292330232 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb7601e06e0 0x7fb7601e0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:00.236 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:00.229+0000 7fb7656b9640 1 --2- 172.21.15.164:0/2292330232 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb7601dd170 0x7fb7601e01a0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:00.236 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:00.229+0000 7fb7656b9640 1 --2- 172.21.15.164:0/2292330232 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb7600adb30 0x7fb7601dcc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:00.236 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:00.229+0000 7fb7656b9640 1 -- 172.21.15.164:0/2292330232 >> 172.21.15.164:0/2292330232 conn(0x7fb7600b7970 msgr2=0x7fb76002bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:58:00.236 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:00.229+0000 7fb7656b9640 1 -- 172.21.15.164:0/2292330232 shutdown_connections 2024-03-23T14:58:00.236 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:00.229+0000 7fb7656b9640 1 -- 172.21.15.164:0/2292330232 wait complete. 2024-03-23T14:58:00.255 INFO:tasks.cephfs_test_runner:Test that when cd is issued with an argument, CWD is changed ... ERROR 2024-03-23T14:58:00.256 INFO:tasks.cephfs_test_runner: 2024-03-23T14:58:00.256 INFO:tasks.cephfs_test_runner:====================================================================== 2024-03-23T14:58:00.256 INFO:tasks.cephfs_test_runner:ERROR: test_cd_with_args (tasks.cephfs.test_cephfs_shell.TestCD) 2024-03-23T14:58:00.256 INFO:tasks.cephfs_test_runner:Test that when cd is issued with an argument, CWD is changed 2024-03-23T14:58:00.256 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-03-23T14:58:00.256 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2024-03-23T14:58:00.256 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_batrick_ceph_d39ab7bb1f1162ec7b83b4f13a6bb876d4baf21b/qa/tasks/cephfs/test_cephfs_shell.py", line 602, in test_cd_with_args 2024-03-23T14:58:00.256 INFO:tasks.cephfs_test_runner: output = self.get_cephfs_shell_script_output(script) 2024-03-23T14:58:00.256 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_batrick_ceph_d39ab7bb1f1162ec7b83b4f13a6bb876d4baf21b/qa/tasks/cephfs/test_cephfs_shell.py", line 161, in get_cephfs_shell_script_output 2024-03-23T14:58:00.256 INFO:tasks.cephfs_test_runner: return ensure_str(self.run_cephfs_shell_script( 2024-03-23T14:58:00.256 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_batrick_ceph_d39ab7bb1f1162ec7b83b4f13a6bb876d4baf21b/qa/tasks/cephfs/test_cephfs_shell.py", line 153, in run_cephfs_shell_script 2024-03-23T14:58:00.256 INFO:tasks.cephfs_test_runner: return mount_x.client_remote.run(args=args, stdout=stdout, 2024-03-23T14:58:00.256 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/remote.py", line 523, in run 2024-03-23T14:58:00.257 INFO:tasks.cephfs_test_runner: r = self._runner(client=self.ssh, name=self.shortname, **kwargs) 2024-03-23T14:58:00.257 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 455, in run 2024-03-23T14:58:00.257 INFO:tasks.cephfs_test_runner: r.wait() 2024-03-23T14:58:00.257 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 161, in wait 2024-03-23T14:58:00.257 INFO:tasks.cephfs_test_runner: self._raise_for_status() 2024-03-23T14:58:00.257 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 181, in _raise_for_status 2024-03-23T14:58:00.257 INFO:tasks.cephfs_test_runner: raise CommandFailedError( 2024-03-23T14:58:00.257 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi164 with status 127: 'cephfs-shell -b /tmp/test-cephfsai49c3po' 2024-03-23T14:58:00.257 INFO:tasks.cephfs_test_runner: 2024-03-23T14:58:00.257 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-03-23T14:58:00.257 INFO:tasks.cephfs_test_runner:Ran 1 test in 37.648s 2024-03-23T14:58:00.257 INFO:tasks.cephfs_test_runner: 2024-03-23T14:58:00.257 INFO:tasks.cephfs_test_runner:FAILED (errors=1) 2024-03-23T14:58:00.257 INFO:tasks.cephfs_test_runner: 2024-03-23T14:58:00.258 INFO:tasks.cephfs_test_runner:====================================================================== 2024-03-23T14:58:00.258 INFO:tasks.cephfs_test_runner:ERROR: test_cd_with_args (tasks.cephfs.test_cephfs_shell.TestCD) 2024-03-23T14:58:00.258 INFO:tasks.cephfs_test_runner:Test that when cd is issued with an argument, CWD is changed 2024-03-23T14:58:00.258 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-03-23T14:58:00.258 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2024-03-23T14:58:00.258 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_batrick_ceph_d39ab7bb1f1162ec7b83b4f13a6bb876d4baf21b/qa/tasks/cephfs/test_cephfs_shell.py", line 602, in test_cd_with_args 2024-03-23T14:58:00.258 INFO:tasks.cephfs_test_runner: output = self.get_cephfs_shell_script_output(script) 2024-03-23T14:58:00.258 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_batrick_ceph_d39ab7bb1f1162ec7b83b4f13a6bb876d4baf21b/qa/tasks/cephfs/test_cephfs_shell.py", line 161, in get_cephfs_shell_script_output 2024-03-23T14:58:00.258 INFO:tasks.cephfs_test_runner: return ensure_str(self.run_cephfs_shell_script( 2024-03-23T14:58:00.258 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_batrick_ceph_d39ab7bb1f1162ec7b83b4f13a6bb876d4baf21b/qa/tasks/cephfs/test_cephfs_shell.py", line 153, in run_cephfs_shell_script 2024-03-23T14:58:00.258 INFO:tasks.cephfs_test_runner: return mount_x.client_remote.run(args=args, stdout=stdout, 2024-03-23T14:58:00.258 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/remote.py", line 523, in run 2024-03-23T14:58:00.258 INFO:tasks.cephfs_test_runner: r = self._runner(client=self.ssh, name=self.shortname, **kwargs) 2024-03-23T14:58:00.258 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 455, in run 2024-03-23T14:58:00.258 INFO:tasks.cephfs_test_runner: r.wait() 2024-03-23T14:58:00.258 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 161, in wait 2024-03-23T14:58:00.258 INFO:tasks.cephfs_test_runner: self._raise_for_status() 2024-03-23T14:58:00.259 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 181, in _raise_for_status 2024-03-23T14:58:00.259 INFO:tasks.cephfs_test_runner: raise CommandFailedError( 2024-03-23T14:58:00.259 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi164 with status 127: 'cephfs-shell -b /tmp/test-cephfsai49c3po' 2024-03-23T14:58:00.259 INFO:tasks.cephfs_test_runner: 2024-03-23T14:58:00.259 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_batrick_ceph_d39ab7bb1f1162ec7b83b4f13a6bb876d4baf21b/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_cd_with_args (tasks.cephfs.test_cephfs_shell.TestCD) 2024-03-23T14:58:00.484 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=a5867e8f3df54845912472e1ea6e5857 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_batrick_ceph_d39ab7bb1f1162ec7b83b4f13a6bb876d4baf21b/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_cd_with_args (tasks.cephfs.test_cephfs_shell.TestCD) 2024-03-23T14:58:00.486 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-03-23T14:58:00.494 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-03-23T14:58:00.503 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-03-23T14:58:00.504 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-03-23T14:58:00.504 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-23T14:58:00.517 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336524k,nr_inodes=4084131,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20870 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274692k,nr_inodes=818673,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 2024-03-23T14:58:00.517 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T14:58:00.517 DEBUG:teuthology.orchestra.run.smithi164:> ip netns list 2024-03-23T14:58:00.565 INFO:teuthology.orchestra.run.smithi164.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-23T14:58:00.565 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T14:58:00.565 DEBUG:teuthology.orchestra.run.smithi164:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-23T14:58:00.621 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-23T14:58:00.622 DEBUG:teuthology.orchestra.run.smithi164:> sudo ip link delete ceph-brx 2024-03-23T14:58:00.834 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-03-23T14:58:00.843 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-03-23T14:58:00.843 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-23T14:58:00.974 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:00.969+0000 7fbb11906640 1 Processor -- start 2024-03-23T14:58:00.974 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:00.969+0000 7fbb11906640 1 -- start start 2024-03-23T14:58:00.975 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:00.969+0000 7fbb11906640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fbb0c06e8e0 0x7fbb0c141380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:00.976 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:00.969+0000 7fbb11906640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fbb0c02c2a0 0x7fbb0c02c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:00.976 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:00.969+0000 7fbb11906640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb0c0b9470 con 0x7fbb0c14a880 2024-03-23T14:58:00.976 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:00.969+0000 7fbb11906640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fbb0c072af0 con 0x7fbb0c06e8e0 2024-03-23T14:58:00.976 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:00.969+0000 7fbb11906640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 -- 0x7fbb0c06ecc0 con 0x7fbb0c02c2a0 2024-03-23T14:58:00.976 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:00.969+0000 7fbb0affd640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fbb0c06e8e0 0x7fbb0c141380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:00.976 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:00.969+0000 7fbb0affd640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fbb0c06e8e0 0x7fbb0c141380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:48836/0 (socket says 172.21.15.164:48836) 2024-03-23T14:58:00.976 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:00.969+0000 7fbb0affd640 1 -- 172.21.15.164:0/967937088 learned_addr learned my addr 172.21.15.164:0/967937088 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:58:00.976 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:00.969+0000 7fbb0affd640 1 -- 172.21.15.164:0/967937088 >> v2:172.21.15.173:3300/0 conn(0x7fbb0c02c2a0 msgr2=0x7fbb0c02c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:58:00.976 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:00.969+0000 7fbb0affd640 1 --2- 172.21.15.164:0/967937088 >> v2:172.21.15.173:3300/0 conn(0x7fbb0c02c2a0 0x7fbb0c02c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:00.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:00.969+0000 7fbb0affd640 1 -- 172.21.15.164:0/967937088 >> v1:172.21.15.173:6789/0 conn(0x7fbb0c14a880 legacy=0x7fbb0c14acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:58:00.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:00.969+0000 7fbb0affd640 1 -- 172.21.15.164:0/967937088 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb0c072040 con 0x7fbb0c06e8e0 2024-03-23T14:58:00.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:00.973+0000 7fbb0affd640 1 --2- 172.21.15.164:0/967937088 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbb0c06e8e0 0x7fbb0c141380 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c073190 tx=0x7fbafc01ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e74af2d63614952 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:58:00.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:00.973+0000 7fbb0a7fc640 1 -- 172.21.15.164:0/967937088 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbafc0a89d0 con 0x7fbb0c06e8e0 2024-03-23T14:58:00.978 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:00.973+0000 7fbb0a7fc640 1 -- 172.21.15.164:0/967937088 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbafc0a8b30 con 0x7fbb0c06e8e0 2024-03-23T14:58:00.979 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:00.973+0000 7fbb0a7fc640 1 -- 172.21.15.164:0/967937088 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbafc0b0440 con 0x7fbb0c06e8e0 2024-03-23T14:58:00.979 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:00.973+0000 7fbb11906640 1 -- 172.21.15.164:0/967937088 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbb0c06e8e0 msgr2=0x7fbb0c141380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:00.979 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:00.973+0000 7fbb11906640 1 --2- 172.21.15.164:0/967937088 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbb0c06e8e0 0x7fbb0c141380 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c073190 tx=0x7fbafc01ecd0 comp rx=0 tx=0).stop 2024-03-23T14:58:00.979 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:00.973+0000 7fbb11906640 1 -- 172.21.15.164:0/967937088 shutdown_connections 2024-03-23T14:58:00.979 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:00.973+0000 7fbb11906640 1 --2- 172.21.15.164:0/967937088 >> v2:172.21.15.173:3300/0 conn(0x7fbb0c02c2a0 0x7fbb0c02c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:00.979 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:00.973+0000 7fbb11906640 1 --2- 172.21.15.164:0/967937088 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbb0c06e8e0 0x7fbb0c141380 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:00.979 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:00.973+0000 7fbb11906640 1 -- 172.21.15.164:0/967937088 >> 172.21.15.164:0/967937088 conn(0x7fbb0c0b7970 msgr2=0x7fbb0c0b7d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:58:00.979 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:00.973+0000 7fbb11906640 1 -- 172.21.15.164:0/967937088 shutdown_connections 2024-03-23T14:58:00.980 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:00.973+0000 7fbb11906640 1 -- 172.21.15.164:0/967937088 wait complete. 2024-03-23T14:58:00.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:00.973+0000 7fbb11906640 1 Processor -- start 2024-03-23T14:58:00.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:00.977+0000 7fbb11906640 1 -- start start 2024-03-23T14:58:00.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:00.977+0000 7fbb11906640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbb0c02c2a0 0x7fbb0c1e06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:00.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:00.977+0000 7fbb11906640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbb0c06e8e0 0x7fbb0c1e0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:00.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:00.977+0000 7fbb11906640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbb0c14a880 0x7fbb0c13c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:00.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:00.977+0000 7fbb11906640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fbb0c0b2090 con 0x7fbb0c06e8e0 2024-03-23T14:58:00.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:00.977+0000 7fbb11906640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fbb0c0b3a90 con 0x7fbb0c14a880 2024-03-23T14:58:00.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:00.977+0000 7fbb11906640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fbb0c145120 con 0x7fbb0c02c2a0 2024-03-23T14:58:00.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:00.977+0000 7fbb03fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbb0c06e8e0 0x7fbb0c1e0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:00.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:00.977+0000 7fbb03fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbb0c06e8e0 0x7fbb0c1e0c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:48838/0 (socket says 172.21.15.164:48838) 2024-03-23T14:58:00.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:00.977+0000 7fbb03fff640 1 -- 172.21.15.164:0/4243312299 learned_addr learned my addr 172.21.15.164:0/4243312299 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:58:00.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:00.977+0000 7fbb0b7fe640 1 --2- 172.21.15.164:0/4243312299 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbb0c14a880 0x7fbb0c13c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:00.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:00.977+0000 7fbb0affd640 1 --2- 172.21.15.164:0/4243312299 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbb0c02c2a0 0x7fbb0c1e06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:00.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:00.977+0000 7fbb03fff640 1 -- 172.21.15.164:0/4243312299 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbb0c02c2a0 msgr2=0x7fbb0c1e06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:00.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:00.977+0000 7fbb03fff640 1 --2- 172.21.15.164:0/4243312299 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbb0c02c2a0 0x7fbb0c1e06c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:00.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:00.977+0000 7fbb03fff640 1 -- 172.21.15.164:0/4243312299 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbb0c14a880 msgr2=0x7fbb0c13c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:00.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:00.977+0000 7fbb03fff640 1 --2- 172.21.15.164:0/4243312299 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbb0c14a880 0x7fbb0c13c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:00.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:00.977+0000 7fbb03fff640 1 -- 172.21.15.164:0/4243312299 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbafc0094d0 con 0x7fbb0c06e8e0 2024-03-23T14:58:00.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:00.977+0000 7fbb03fff640 1 --2- 172.21.15.164:0/4243312299 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbb0c06e8e0 0x7fbb0c1e0c00 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fbaf400b550 tx=0x7fbaf400ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:58:00.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:00.977+0000 7fbb08ff9640 1 -- 172.21.15.164:0/4243312299 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaf4010870 con 0x7fbb0c06e8e0 2024-03-23T14:58:00.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:00.977+0000 7fbb08ff9640 1 -- 172.21.15.164:0/4243312299 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbaf40109f0 con 0x7fbb0c06e8e0 2024-03-23T14:58:00.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:00.977+0000 7fbb11906640 1 -- 172.21.15.164:0/4243312299 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb0c0b2580 con 0x7fbb0c06e8e0 2024-03-23T14:58:00.984 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:00.977+0000 7fbb08ff9640 1 -- 172.21.15.164:0/4243312299 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaf4010da0 con 0x7fbb0c06e8e0 2024-03-23T14:58:00.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:00.977+0000 7fbb11906640 1 -- 172.21.15.164:0/4243312299 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb0c147110 con 0x7fbb0c06e8e0 2024-03-23T14:58:00.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:00.977+0000 7fbb11906640 1 -- 172.21.15.164:0/4243312299 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb0c0690a0 con 0x7fbb0c06e8e0 2024-03-23T14:58:00.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:00.977+0000 7fbb08ff9640 1 -- 172.21.15.164:0/4243312299 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7fbaf400f3c0 con 0x7fbb0c06e8e0 2024-03-23T14:58:00.986 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:00.977+0000 7fbb08ff9640 1 --2- 172.21.15.164:0/4243312299 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fbaec0672a0 0x7fbaec069760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:00.986 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:00.977+0000 7fbb08ff9640 1 -- 172.21.15.164:0/4243312299 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4943+0+0 (secure 0 0 0) 0x7fbaf4084d30 con 0x7fbb0c06e8e0 2024-03-23T14:58:00.986 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:00.977+0000 7fbb0affd640 1 --2- 172.21.15.164:0/4243312299 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fbaec0672a0 0x7fbaec069760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:00.986 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:00.977+0000 7fbb0affd640 1 --2- 172.21.15.164:0/4243312299 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fbaec0672a0 0x7fbaec069760 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fbafc0b2000 tx=0x7fbafc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:58:00.987 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:00.981+0000 7fbb08ff9640 1 -- 172.21.15.164:0/4243312299 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7fbaf4051be0 con 0x7fbb0c06e8e0 2024-03-23T14:58:01.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.137+0000 7fbb11906640 1 -- 172.21.15.164:0/4243312299 --> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbb0c072600 con 0x7fbaec0672a0 2024-03-23T14:58:01.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.137+0000 7fbb08ff9640 1 -- 172.21.15.164:0/4243312299 <== mgr.4107 v2:172.21.15.164:6832/2139087985 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13470 (secure 0 0 0) 0x7fbafc0b05c0 con 0x7fbaec0672a0 2024-03-23T14:58:01.143 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-23T14:58:01.143 INFO:teuthology.orchestra.run.smithi164.stderr:dumped all 2024-03-23T14:58:01.145 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.141+0000 7fbb11906640 1 -- 172.21.15.164:0/4243312299 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fbaec0672a0 msgr2=0x7fbaec069760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:01.145 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.141+0000 7fbb11906640 1 --2- 172.21.15.164:0/4243312299 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fbaec0672a0 0x7fbaec069760 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fbafc0b2000 tx=0x7fbafc008040 comp rx=0 tx=0).stop 2024-03-23T14:58:01.145 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.141+0000 7fbb11906640 1 -- 172.21.15.164:0/4243312299 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbb0c06e8e0 msgr2=0x7fbb0c1e0c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:01.145 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.141+0000 7fbb11906640 1 --2- 172.21.15.164:0/4243312299 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbb0c06e8e0 0x7fbb0c1e0c00 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fbaf400b550 tx=0x7fbaf400ba10 comp rx=0 tx=0).stop 2024-03-23T14:58:01.145 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.141+0000 7fbb11906640 1 -- 172.21.15.164:0/4243312299 shutdown_connections 2024-03-23T14:58:01.146 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.141+0000 7fbb11906640 1 --2- 172.21.15.164:0/4243312299 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fbaec0672a0 0x7fbaec069760 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:01.146 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.141+0000 7fbb11906640 1 --2- 172.21.15.164:0/4243312299 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbb0c14a880 0x7fbb0c13c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:01.146 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.141+0000 7fbb11906640 1 --2- 172.21.15.164:0/4243312299 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbb0c06e8e0 0x7fbb0c1e0c00 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:01.146 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.141+0000 7fbb11906640 1 --2- 172.21.15.164:0/4243312299 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbb0c02c2a0 0x7fbb0c1e06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:01.146 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.141+0000 7fbb11906640 1 -- 172.21.15.164:0/4243312299 >> 172.21.15.164:0/4243312299 conn(0x7fbb0c0b7970 msgr2=0x7fbb0c0b4370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:58:01.146 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.141+0000 7fbb11906640 1 -- 172.21.15.164:0/4243312299 shutdown_connections 2024-03-23T14:58:01.146 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.141+0000 7fbb11906640 1 -- 172.21.15.164:0/4243312299 wait complete. 2024-03-23T14:58:01.161 INFO:teuthology.orchestra.run.smithi164.stdout:{"pg_ready":true,"pg_map":{"version":62,"stamp":"2024-03-23T14:58:01.125548+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220016,"kb_used_data":3860,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754704,"statfs":{"total":773094113280,"available":772868816896,"internally_reserved":0,"allocated":3952640,"data_stored":2245555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-10891,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-55,"num_write_kb":-28,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001354"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":189,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-23T14:57:56.262449+0000","last_change":"2024-03-23T14:56:48.722063+0000","last_active":"2024-03-23T14:57:56.262449+0000","last_peered":"2024-03-23T14:57:56.262449+0000","last_clean":"2024-03-23T14:57:56.262449+0000","last_became_active":"2024-03-23T14:56:48.720079+0000","last_became_peered":"2024-03-23T14:56:48.720079+0000","last_unstale":"2024-03-23T14:57:56.262449+0000","last_undegraded":"2024-03-23T14:57:56.262449+0000","last_fullsized":"2024-03-23T14:57:56.262449+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:47.478078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:47.478078+0000","last_clean_scrub_stamp":"2024-03-23T14:56:47.478078+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:55:51.231901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574866,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27932,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343908,"statfs":{"total":96636764160,"available":96608161792,"internally_reserved":0,"allocated":933888,"data_stored":720551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574866,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27356,"kb_used_data":336,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344484,"statfs":{"total":96636764160,"available":96608751616,"internally_reserved":0,"allocated":344064,"data_stored":130709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574866,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27356,"kb_used_data":336,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344484,"statfs":{"total":96636764160,"available":96608751616,"internally_reserved":0,"allocated":344064,"data_stored":130765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574866,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27356,"kb_used_data":336,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344484,"statfs":{"total":96636764160,"available":96608751616,"internally_reserved":0,"allocated":344064,"data_stored":130197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574867,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27376,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344464,"statfs":{"total":96636764160,"available":96608731136,"internally_reserved":0,"allocated":364544,"data_stored":144395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574866,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27352,"kb_used_data":332,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344488,"statfs":{"total":96636764160,"available":96608755712,"internally_reserved":0,"allocated":339968,"data_stored":130163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574867,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":352256,"data_stored":138314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574865,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27928,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343912,"statfs":{"total":96636764160,"available":96608165888,"internally_reserved":0,"allocated":929792,"data_stored":720461,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-23T14:58:01.162 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-23T14:58:01.279 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.273+0000 7fdf4a68d640 1 Processor -- start 2024-03-23T14:58:01.280 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.277+0000 7fdf4a68d640 1 -- start start 2024-03-23T14:58:01.280 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.277+0000 7fdf4a68d640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fdf440b0230 0x7fdf440b34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:01.280 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.277+0000 7fdf4a68d640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdf44148cf0 0x7fdf44149180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:01.280 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.277+0000 7fdf4a68d640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf440b51d0 con 0x7fdf440b1c70 2024-03-23T14:58:01.281 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.277+0000 7fdf4a68d640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 -- 0x7fdf440b0610 con 0x7fdf440b0230 2024-03-23T14:58:01.281 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.277+0000 7fdf4a68d640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fdf440b1500 con 0x7fdf44148cf0 2024-03-23T14:58:01.282 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.277+0000 7fdf43fff640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fdf440b0230 0x7fdf440b34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:01.282 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.277+0000 7fdf3b7fe640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdf44148cf0 0x7fdf44149180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:01.282 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.277+0000 7fdf43fff640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fdf440b0230 0x7fdf440b34d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.164:49686/0 (socket says 172.21.15.164:49686) 2024-03-23T14:58:01.282 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.277+0000 7fdf48c2a640 1 --1- >> v1:172.21.15.173:6789/0 conn(0x7fdf440b1c70 0x7fdf440b20a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.173:6789/0 says I am v1:172.21.15.164:38058/0 (socket says 172.21.15.164:38058) 2024-03-23T14:58:01.282 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.277+0000 7fdf3b7fe640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdf44148cf0 0x7fdf44149180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.164:48564/0 (socket says 172.21.15.164:48564) 2024-03-23T14:58:01.282 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.277+0000 7fdf43fff640 1 -- 172.21.15.164:0/4007041610 learned_addr learned my addr 172.21.15.164:0/4007041610 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:58:01.282 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.277+0000 7fdf43fff640 1 -- 172.21.15.164:0/4007041610 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdf44148cf0 msgr2=0x7fdf44149180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:58:01.283 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.277+0000 7fdf43fff640 1 --2- 172.21.15.164:0/4007041610 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdf44148cf0 0x7fdf44149180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:01.283 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.277+0000 7fdf43fff640 1 -- 172.21.15.164:0/4007041610 >> v1:172.21.15.173:6789/0 conn(0x7fdf440b1c70 legacy=0x7fdf440b20a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:01.283 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.277+0000 7fdf43fff640 1 -- 172.21.15.164:0/4007041610 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf4406d250 con 0x7fdf440b0230 2024-03-23T14:58:01.283 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.281+0000 7fdf43fff640 1 --2- 172.21.15.164:0/4007041610 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdf440b0230 0x7fdf440b34d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fdf34009870 tx=0x7fdf3401c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8742c87ac5e6f932 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:58:01.284 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.281+0000 7fdf437fe640 1 -- 172.21.15.164:0/4007041610 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf340a6040 con 0x7fdf440b0230 2024-03-23T14:58:01.284 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.281+0000 7fdf437fe640 1 -- 172.21.15.164:0/4007041610 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf340aa600 con 0x7fdf440b0230 2024-03-23T14:58:01.284 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.281+0000 7fdf437fe640 1 -- 172.21.15.164:0/4007041610 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf340aa990 con 0x7fdf440b0230 2024-03-23T14:58:01.285 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.281+0000 7fdf4a68d640 1 -- 172.21.15.164:0/4007041610 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdf440b0230 msgr2=0x7fdf440b34d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:01.285 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.281+0000 7fdf4a68d640 1 --2- 172.21.15.164:0/4007041610 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdf440b0230 0x7fdf440b34d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fdf34009870 tx=0x7fdf3401c4a0 comp rx=0 tx=0).stop 2024-03-23T14:58:01.285 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.281+0000 7fdf4a68d640 1 -- 172.21.15.164:0/4007041610 shutdown_connections 2024-03-23T14:58:01.285 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.281+0000 7fdf4a68d640 1 --2- 172.21.15.164:0/4007041610 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdf44148cf0 0x7fdf44149180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:01.286 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.281+0000 7fdf4a68d640 1 --2- 172.21.15.164:0/4007041610 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdf440b0230 0x7fdf440b34d0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:01.286 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.281+0000 7fdf4a68d640 1 -- 172.21.15.164:0/4007041610 >> 172.21.15.164:0/4007041610 conn(0x7fdf440b8ca0 msgr2=0x7fdf440b90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:58:01.286 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.281+0000 7fdf4a68d640 1 -- 172.21.15.164:0/4007041610 shutdown_connections 2024-03-23T14:58:01.286 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.281+0000 7fdf4a68d640 1 -- 172.21.15.164:0/4007041610 wait complete. 2024-03-23T14:58:01.288 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.281+0000 7fdf4a68d640 1 Processor -- start 2024-03-23T14:58:01.288 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.281+0000 7fdf4a68d640 1 -- start start 2024-03-23T14:58:01.288 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.281+0000 7fdf4a68d640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdf440b0230 0x7fdf4409d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:01.288 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.281+0000 7fdf4a68d640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdf440b1c70 0x7fdf4409c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:01.289 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.281+0000 7fdf4a68d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf44148cf0 0x7fdf441dd9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:01.289 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.281+0000 7fdf4a68d640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fdf4409bb90 con 0x7fdf44148cf0 2024-03-23T14:58:01.289 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.281+0000 7fdf4a68d640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fdf4409c280 con 0x7fdf440b1c70 2024-03-23T14:58:01.289 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.281+0000 7fdf4a68d640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fdf44143e80 con 0x7fdf440b0230 2024-03-23T14:58:01.289 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.281+0000 7fdf48c2a640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf44148cf0 0x7fdf441dd9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:01.289 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.281+0000 7fdf48c2a640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf44148cf0 0x7fdf441dd9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:48854/0 (socket says 172.21.15.164:48854) 2024-03-23T14:58:01.289 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.281+0000 7fdf48c2a640 1 -- 172.21.15.164:0/3234408257 learned_addr learned my addr 172.21.15.164:0/3234408257 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:58:01.289 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.281+0000 7fdf43fff640 1 --2- 172.21.15.164:0/3234408257 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdf440b0230 0x7fdf4409d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:01.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.281+0000 7fdf3b7fe640 1 --2- 172.21.15.164:0/3234408257 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdf440b1c70 0x7fdf4409c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:01.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.281+0000 7fdf48c2a640 1 -- 172.21.15.164:0/3234408257 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdf440b0230 msgr2=0x7fdf4409d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:01.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.281+0000 7fdf48c2a640 1 --2- 172.21.15.164:0/3234408257 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdf440b0230 0x7fdf4409d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:01.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.281+0000 7fdf48c2a640 1 -- 172.21.15.164:0/3234408257 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdf440b1c70 msgr2=0x7fdf4409c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:01.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.285+0000 7fdf48c2a640 1 --2- 172.21.15.164:0/3234408257 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdf440b1c70 0x7fdf4409c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:01.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.285+0000 7fdf48c2a640 1 -- 172.21.15.164:0/3234408257 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf340094d0 con 0x7fdf44148cf0 2024-03-23T14:58:01.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.285+0000 7fdf48c2a640 1 --2- 172.21.15.164:0/3234408257 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf44148cf0 0x7fdf441dd9d0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c00c7a0 tx=0x7fdf3c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:58:01.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.285+0000 7fdf417fa640 1 -- 172.21.15.164:0/3234408257 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf3c00a640 con 0x7fdf44148cf0 2024-03-23T14:58:01.290 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.285+0000 7fdf417fa640 1 -- 172.21.15.164:0/3234408257 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf3c00a7a0 con 0x7fdf44148cf0 2024-03-23T14:58:01.291 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.285+0000 7fdf4a68d640 1 -- 172.21.15.164:0/3234408257 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf4409bde0 con 0x7fdf44148cf0 2024-03-23T14:58:01.291 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.285+0000 7fdf417fa640 1 -- 172.21.15.164:0/3234408257 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf3c00ab10 con 0x7fdf44148cf0 2024-03-23T14:58:01.291 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.285+0000 7fdf4a68d640 1 -- 172.21.15.164:0/3234408257 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf44143950 con 0x7fdf44148cf0 2024-03-23T14:58:01.292 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.285+0000 7fdf4a68d640 1 -- 172.21.15.164:0/3234408257 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf440b3990 con 0x7fdf44148cf0 2024-03-23T14:58:01.292 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.285+0000 7fdf417fa640 1 -- 172.21.15.164:0/3234408257 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7fdf3c00ac70 con 0x7fdf44148cf0 2024-03-23T14:58:01.292 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.285+0000 7fdf417fa640 1 --2- 172.21.15.164:0/3234408257 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fdf240672a0 0x7fdf24069760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:01.292 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.285+0000 7fdf417fa640 1 -- 172.21.15.164:0/3234408257 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4943+0+0 (secure 0 0 0) 0x7fdf3c0843c0 con 0x7fdf44148cf0 2024-03-23T14:58:01.292 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.285+0000 7fdf43fff640 1 --2- 172.21.15.164:0/3234408257 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fdf240672a0 0x7fdf24069760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:01.292 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.285+0000 7fdf43fff640 1 --2- 172.21.15.164:0/3234408257 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fdf240672a0 0x7fdf24069760 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fdf340ae000 tx=0x7fdf340b5040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:58:01.294 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.289+0000 7fdf417fa640 1 -- 172.21.15.164:0/3234408257 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7fdf3c0513b0 con 0x7fdf44148cf0 2024-03-23T14:58:01.448 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.445+0000 7fdf4a68d640 1 -- 172.21.15.164:0/3234408257 --> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fdf4409c100 con 0x7fdf240672a0 2024-03-23T14:58:01.449 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.445+0000 7fdf417fa640 1 -- 172.21.15.164:0/3234408257 <== mgr.4107 v2:172.21.15.164:6832/2139087985 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13470 (secure 0 0 0) 0x7fdf340aa870 con 0x7fdf240672a0 2024-03-23T14:58:01.449 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-23T14:58:01.450 INFO:teuthology.orchestra.run.smithi164.stderr:dumped all 2024-03-23T14:58:01.452 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.445+0000 7fdf4a68d640 1 -- 172.21.15.164:0/3234408257 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fdf240672a0 msgr2=0x7fdf24069760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:01.452 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.445+0000 7fdf4a68d640 1 --2- 172.21.15.164:0/3234408257 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fdf240672a0 0x7fdf24069760 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fdf340ae000 tx=0x7fdf340b5040 comp rx=0 tx=0).stop 2024-03-23T14:58:01.452 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.445+0000 7fdf4a68d640 1 -- 172.21.15.164:0/3234408257 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf44148cf0 msgr2=0x7fdf441dd9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:01.452 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.445+0000 7fdf4a68d640 1 --2- 172.21.15.164:0/3234408257 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf44148cf0 0x7fdf441dd9d0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c00c7a0 tx=0x7fdf3c00cc60 comp rx=0 tx=0).stop 2024-03-23T14:58:01.452 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.445+0000 7fdf4a68d640 1 -- 172.21.15.164:0/3234408257 shutdown_connections 2024-03-23T14:58:01.452 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.445+0000 7fdf4a68d640 1 --2- 172.21.15.164:0/3234408257 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf44148cf0 0x7fdf441dd9d0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:01.452 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.445+0000 7fdf4a68d640 1 --2- 172.21.15.164:0/3234408257 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fdf240672a0 0x7fdf24069760 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:01.452 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.445+0000 7fdf4a68d640 1 --2- 172.21.15.164:0/3234408257 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdf440b1c70 0x7fdf4409c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:01.452 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.445+0000 7fdf4a68d640 1 --2- 172.21.15.164:0/3234408257 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdf440b0230 0x7fdf4409d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:01.453 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.445+0000 7fdf4a68d640 1 -- 172.21.15.164:0/3234408257 >> 172.21.15.164:0/3234408257 conn(0x7fdf440b8ca0 msgr2=0x7fdf440b5cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:58:01.453 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.449+0000 7fdf4a68d640 1 -- 172.21.15.164:0/3234408257 shutdown_connections 2024-03-23T14:58:01.453 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.449+0000 7fdf4a68d640 1 -- 172.21.15.164:0/3234408257 wait complete. 2024-03-23T14:58:01.468 INFO:teuthology.orchestra.run.smithi164.stdout:{"pg_ready":true,"pg_map":{"version":62,"stamp":"2024-03-23T14:58:01.125548+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220016,"kb_used_data":3860,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754704,"statfs":{"total":773094113280,"available":772868816896,"internally_reserved":0,"allocated":3952640,"data_stored":2245555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-10891,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-55,"num_write_kb":-28,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001354"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":189,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-23T14:57:56.262449+0000","last_change":"2024-03-23T14:56:48.722063+0000","last_active":"2024-03-23T14:57:56.262449+0000","last_peered":"2024-03-23T14:57:56.262449+0000","last_clean":"2024-03-23T14:57:56.262449+0000","last_became_active":"2024-03-23T14:56:48.720079+0000","last_became_peered":"2024-03-23T14:56:48.720079+0000","last_unstale":"2024-03-23T14:57:56.262449+0000","last_undegraded":"2024-03-23T14:57:56.262449+0000","last_fullsized":"2024-03-23T14:57:56.262449+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:47.478078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:47.478078+0000","last_clean_scrub_stamp":"2024-03-23T14:56:47.478078+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:55:51.231901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574866,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27932,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343908,"statfs":{"total":96636764160,"available":96608161792,"internally_reserved":0,"allocated":933888,"data_stored":720551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574866,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27356,"kb_used_data":336,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344484,"statfs":{"total":96636764160,"available":96608751616,"internally_reserved":0,"allocated":344064,"data_stored":130709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574866,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27356,"kb_used_data":336,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344484,"statfs":{"total":96636764160,"available":96608751616,"internally_reserved":0,"allocated":344064,"data_stored":130765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574866,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27356,"kb_used_data":336,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344484,"statfs":{"total":96636764160,"available":96608751616,"internally_reserved":0,"allocated":344064,"data_stored":130197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574867,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27376,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344464,"statfs":{"total":96636764160,"available":96608731136,"internally_reserved":0,"allocated":364544,"data_stored":144395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574866,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27352,"kb_used_data":332,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344488,"statfs":{"total":96636764160,"available":96608755712,"internally_reserved":0,"allocated":339968,"data_stored":130163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574867,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":352256,"data_stored":138314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574865,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27928,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343912,"statfs":{"total":96636764160,"available":96608165888,"internally_reserved":0,"allocated":929792,"data_stored":720461,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-23T14:58:01.468 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-03-23T14:58:01.469 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-23T14:58:01.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.581+0000 7f22fea1f640 1 Processor -- start 2024-03-23T14:58:01.587 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.581+0000 7f22fea1f640 1 -- start start 2024-03-23T14:58:01.589 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.585+0000 7f22fea1f640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f22f80b0230 0x7f22f80b34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:01.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.585+0000 7f22fea1f640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22f80b51d0 con 0x7f22f8148cf0 2024-03-23T14:58:01.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.585+0000 7f22fea1f640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22f80b0610 con 0x7f22f814db70 2024-03-23T14:58:01.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.585+0000 7f22fea1f640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f22f80b1500 con 0x7f22f80b0230 2024-03-23T14:58:01.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.585+0000 7f22f77fe640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f22f8148cf0 0x7f22f8149150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:43172/0 (socket says 172.21.15.164:43172) 2024-03-23T14:58:01.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.585+0000 7f22f77fe640 1 -- 172.21.15.164:0/1541664185 learned_addr learned my addr 172.21.15.164:0/1541664185 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-03-23T14:58:01.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.585+0000 7f22f7fff640 1 --2- 172.21.15.164:0/1541664185 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f22f80b0230 0x7f22f80b34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:01.590 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.585+0000 7f22f6ffd640 1 -- 172.21.15.164:0/1541664185 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3293809150 0 0) 0x7f22f80b51d0 con 0x7f22f8148cf0 2024-03-23T14:58:01.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.585+0000 7f22f6ffd640 1 -- 172.21.15.164:0/1541664185 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f22e4003530 con 0x7f22f8148cf0 2024-03-23T14:58:01.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.585+0000 7f22f6ffd640 1 -- 172.21.15.164:0/1541664185 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1677042616 0 0) 0x7f22e4003530 con 0x7f22f8148cf0 2024-03-23T14:58:01.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.585+0000 7f22f6ffd640 1 -- 172.21.15.164:0/1541664185 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f22f80b0230 msgr2=0x7f22f80b34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:58:01.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.585+0000 7f22f6ffd640 1 --2- 172.21.15.164:0/1541664185 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f22f80b0230 0x7f22f80b34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:01.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.585+0000 7f22f6ffd640 1 -- 172.21.15.164:0/1541664185 >> v1:172.21.15.173:6789/0 conn(0x7f22f814db70 legacy=0x7f22f814ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:01.591 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.585+0000 7f22f6ffd640 1 -- 172.21.15.164:0/1541664185 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22f806d250 con 0x7f22f8148cf0 2024-03-23T14:58:01.592 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.585+0000 7f22f6ffd640 1 -- 172.21.15.164:0/1541664185 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2526390548 0 0) 0x7f22e0001a00 con 0x7f22f8148cf0 2024-03-23T14:58:01.592 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.585+0000 7f22f6ffd640 1 -- 172.21.15.164:0/1541664185 >> v1:172.21.15.164:6789/0 conn(0x7f22f8148cf0 legacy=0x7f22f8149150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:01.592 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.585+0000 7f22f6ffd640 1 --2- 172.21.15.164:0/1541664185 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f22e40039b0 0x7f22e4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:01.592 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.585+0000 7f22f6ffd640 1 --2- 172.21.15.164:0/1541664185 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f22e40047d0 0x7f22e4004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:01.592 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.585+0000 7f22f6ffd640 1 --2- 172.21.15.164:0/1541664185 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f22f80b0230 0x7f22e4005240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:01.592 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.585+0000 7f22f6ffd640 1 -- 172.21.15.164:0/1541664185 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f22f80b1500 con 0x7f22e40047d0 2024-03-23T14:58:01.592 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.585+0000 7f22f6ffd640 1 -- 172.21.15.164:0/1541664185 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f22e400bb20 con 0x7f22e40039b0 2024-03-23T14:58:01.592 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.585+0000 7f22f6ffd640 1 -- 172.21.15.164:0/1541664185 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f22e400bce0 con 0x7f22f80b0230 2024-03-23T14:58:01.592 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.585+0000 7f22f77fe640 1 --2- 172.21.15.164:0/1541664185 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f22e40047d0 0x7f22e4004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:01.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.585+0000 7f22f7fff640 1 --2- 172.21.15.164:0/1541664185 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f22e40039b0 0x7f22e4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:01.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.585+0000 7f22fcfbc640 1 --2- 172.21.15.164:0/1541664185 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f22f80b0230 0x7f22e4005240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:01.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.585+0000 7f22f77fe640 1 -- 172.21.15.164:0/1541664185 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f22f80b0230 msgr2=0x7f22e4005240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:58:01.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.585+0000 7f22f77fe640 1 --2- 172.21.15.164:0/1541664185 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f22f80b0230 0x7f22e4005240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:01.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.585+0000 7f22f77fe640 1 -- 172.21.15.164:0/1541664185 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f22e40039b0 msgr2=0x7f22e4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:58:01.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.585+0000 7f22f77fe640 1 --2- 172.21.15.164:0/1541664185 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f22e40039b0 0x7f22e4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:01.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.585+0000 7f22f77fe640 1 -- 172.21.15.164:0/1541664185 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22e4003530 con 0x7f22e40047d0 2024-03-23T14:58:01.593 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.585+0000 7f22fcfbc640 1 --2- 172.21.15.164:0/1541664185 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f22f80b0230 0x7f22e4005240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T14:58:01.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.585+0000 7f22f77fe640 1 --2- 172.21.15.164:0/1541664185 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f22e40047d0 0x7f22e4004c50 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f22e000a750 tx=0x7f22e001d250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d89647ff024bbb5d server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:58:01.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.585+0000 7f22f6ffd640 1 -- 172.21.15.164:0/1541664185 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22e00ad630 con 0x7f22e40047d0 2024-03-23T14:58:01.597 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.585+0000 7f22f6ffd640 1 -- 172.21.15.164:0/1541664185 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22e00a8070 con 0x7f22e40047d0 2024-03-23T14:58:01.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.585+0000 7f22f6ffd640 1 -- 172.21.15.164:0/1541664185 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22e00ad9d0 con 0x7f22e40047d0 2024-03-23T14:58:01.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.589+0000 7f22fea1f640 1 -- 172.21.15.164:0/1541664185 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f22e40047d0 msgr2=0x7f22e4004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:01.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.589+0000 7f22fea1f640 1 --2- 172.21.15.164:0/1541664185 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f22e40047d0 0x7f22e4004c50 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f22e000a750 tx=0x7f22e001d250 comp rx=0 tx=0).stop 2024-03-23T14:58:01.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.589+0000 7f22fcfbc640 1 -- 172.21.15.164:0/1541664185 reap_dead start 2024-03-23T14:58:01.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.589+0000 7f22fea1f640 1 -- 172.21.15.164:0/1541664185 shutdown_connections 2024-03-23T14:58:01.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.589+0000 7f22fea1f640 1 -- 172.21.15.164:0/1541664185 >> 172.21.15.164:0/1541664185 conn(0x7f22f80b8ca0 msgr2=0x7f22f80b90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:58:01.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.589+0000 7f22fea1f640 1 -- 172.21.15.164:0/1541664185 shutdown_connections 2024-03-23T14:58:01.598 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.589+0000 7f22fea1f640 1 -- 172.21.15.164:0/1541664185 wait complete. 2024-03-23T14:58:01.599 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.589+0000 7f22fea1f640 1 Processor -- start 2024-03-23T14:58:01.599 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.589+0000 7f22fea1f640 1 -- start start 2024-03-23T14:58:01.599 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.589+0000 7f22fea1f640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f22e40047d0 0x7f22f806f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:01.599 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.589+0000 7f22fea1f640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f22f8070f20 0x7f22f80703e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:01.599 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.589+0000 7f22fea1f640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f22f81de280 0x7f22f81de700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:01.599 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.589+0000 7f22fea1f640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f22f8150b30 con 0x7f22f8070f20 2024-03-23T14:58:01.599 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.589+0000 7f22fea1f640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f22f81dec40 con 0x7f22e40047d0 2024-03-23T14:58:01.599 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.589+0000 7f22fea1f640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f22f809c890 con 0x7f22f81de280 2024-03-23T14:58:01.599 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.589+0000 7f22f77fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f22f8070f20 0x7f22f80703e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:01.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.589+0000 7f22f77fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f22f8070f20 0x7f22f80703e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:48872/0 (socket says 172.21.15.164:48872) 2024-03-23T14:58:01.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.589+0000 7f22f77fe640 1 -- 172.21.15.164:0/4105580223 learned_addr learned my addr 172.21.15.164:0/4105580223 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:58:01.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.589+0000 7f22f7fff640 1 --2- 172.21.15.164:0/4105580223 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f22e40047d0 0x7f22f806f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:01.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.589+0000 7f22fcfbc640 1 --2- 172.21.15.164:0/4105580223 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f22f81de280 0x7f22f81de700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:01.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.589+0000 7f22f77fe640 1 -- 172.21.15.164:0/4105580223 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f22f81de280 msgr2=0x7f22f81de700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:01.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.589+0000 7f22f77fe640 1 --2- 172.21.15.164:0/4105580223 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f22f81de280 0x7f22f81de700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:01.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.589+0000 7f22f77fe640 1 -- 172.21.15.164:0/4105580223 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f22e40047d0 msgr2=0x7f22f806f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:01.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.589+0000 7f22f77fe640 1 --2- 172.21.15.164:0/4105580223 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f22e40047d0 0x7f22f806f410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:01.600 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.589+0000 7f22f77fe640 1 -- 172.21.15.164:0/4105580223 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22f80b0f70 con 0x7f22f8070f20 2024-03-23T14:58:01.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.589+0000 7f22f77fe640 1 --2- 172.21.15.164:0/4105580223 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f22f8070f20 0x7f22f80703e0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f22e001e930 tx=0x7f22e4005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:58:01.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.589+0000 7f22f4ff9640 1 -- 172.21.15.164:0/4105580223 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22e00ad430 con 0x7f22f8070f20 2024-03-23T14:58:01.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.589+0000 7f22f4ff9640 1 -- 172.21.15.164:0/4105580223 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22e00ad590 con 0x7f22f8070f20 2024-03-23T14:58:01.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.589+0000 7f22f4ff9640 1 -- 172.21.15.164:0/4105580223 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22e00ad9b0 con 0x7f22f8070f20 2024-03-23T14:58:01.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.589+0000 7f22fea1f640 1 -- 172.21.15.164:0/4105580223 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22f8071730 con 0x7f22f8070f20 2024-03-23T14:58:01.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.589+0000 7f22fea1f640 1 -- 172.21.15.164:0/4105580223 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22f8071ea0 con 0x7f22f8070f20 2024-03-23T14:58:01.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.593+0000 7f22fea1f640 1 -- 172.21.15.164:0/4105580223 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22f80690a0 con 0x7f22f8070f20 2024-03-23T14:58:01.601 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.593+0000 7f22f4ff9640 1 -- 172.21.15.164:0/4105580223 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7f22e00aa030 con 0x7f22f8070f20 2024-03-23T14:58:01.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.593+0000 7f22f4ff9640 1 --2- 172.21.15.164:0/4105580223 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f22d80672a0 0x7f22d8069760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:01.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.593+0000 7f22f4ff9640 1 -- 172.21.15.164:0/4105580223 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4943+0+0 (secure 0 0 0) 0x7f22e011fd60 con 0x7f22f8070f20 2024-03-23T14:58:01.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.593+0000 7f22f7fff640 1 --2- 172.21.15.164:0/4105580223 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f22d80672a0 0x7f22d8069760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:01.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.593+0000 7f22f7fff640 1 --2- 172.21.15.164:0/4105580223 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f22d80672a0 0x7f22d8069760 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f22e8009970 tx=0x7f22e8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:58:01.602 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.597+0000 7f22f4ff9640 1 -- 172.21.15.164:0/4105580223 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7f22e00ecca0 con 0x7f22f8070f20 2024-03-23T14:58:01.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.749+0000 7f22fea1f640 1 -- 172.21.15.164:0/4105580223 --> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f22f80b0df0 con 0x7f22d80672a0 2024-03-23T14:58:01.756 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.753+0000 7f22f4ff9640 1 -- 172.21.15.164:0/4105580223 <== mgr.4107 v2:172.21.15.164:6832/2139087985 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13470 (secure 0 0 0) 0x7f22e80095e0 con 0x7f22d80672a0 2024-03-23T14:58:01.757 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-23T14:58:01.757 INFO:teuthology.orchestra.run.smithi164.stderr:dumped all 2024-03-23T14:58:01.759 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.753+0000 7f22fea1f640 1 -- 172.21.15.164:0/4105580223 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f22d80672a0 msgr2=0x7f22d8069760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:01.759 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.753+0000 7f22fea1f640 1 --2- 172.21.15.164:0/4105580223 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f22d80672a0 0x7f22d8069760 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f22e8009970 tx=0x7f22e8008040 comp rx=0 tx=0).stop 2024-03-23T14:58:01.759 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.753+0000 7f22fea1f640 1 -- 172.21.15.164:0/4105580223 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f22f8070f20 msgr2=0x7f22f80703e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:01.759 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.753+0000 7f22fea1f640 1 --2- 172.21.15.164:0/4105580223 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f22f8070f20 0x7f22f80703e0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f22e001e930 tx=0x7f22e4005800 comp rx=0 tx=0).stop 2024-03-23T14:58:01.759 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.753+0000 7f22fea1f640 1 -- 172.21.15.164:0/4105580223 shutdown_connections 2024-03-23T14:58:01.760 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.753+0000 7f22fea1f640 1 --2- 172.21.15.164:0/4105580223 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f22f81de280 0x7f22f81de700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:01.760 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.753+0000 7f22fea1f640 1 --2- 172.21.15.164:0/4105580223 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f22f8070f20 0x7f22f80703e0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:01.760 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.753+0000 7f22fea1f640 1 --2- 172.21.15.164:0/4105580223 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f22d80672a0 0x7f22d8069760 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:01.760 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.753+0000 7f22fea1f640 1 --2- 172.21.15.164:0/4105580223 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f22e40047d0 0x7f22f806f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:01.760 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.753+0000 7f22fea1f640 1 -- 172.21.15.164:0/4105580223 >> 172.21.15.164:0/4105580223 conn(0x7f22f80b8ca0 msgr2=0x7f22f80b2310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:58:01.760 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.753+0000 7f22fea1f640 1 -- 172.21.15.164:0/4105580223 shutdown_connections 2024-03-23T14:58:01.760 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.753+0000 7f22fea1f640 1 -- 172.21.15.164:0/4105580223 wait complete. 2024-03-23T14:58:01.775 INFO:teuthology.orchestra.run.smithi164.stdout:{"pg_ready":true,"pg_map":{"version":62,"stamp":"2024-03-23T14:58:01.125548+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220016,"kb_used_data":3860,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754704,"statfs":{"total":773094113280,"available":772868816896,"internally_reserved":0,"allocated":3952640,"data_stored":2245555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-10891,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-55,"num_write_kb":-28,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001354"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":189,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-23T14:57:56.262449+0000","last_change":"2024-03-23T14:56:48.722063+0000","last_active":"2024-03-23T14:57:56.262449+0000","last_peered":"2024-03-23T14:57:56.262449+0000","last_clean":"2024-03-23T14:57:56.262449+0000","last_became_active":"2024-03-23T14:56:48.720079+0000","last_became_peered":"2024-03-23T14:56:48.720079+0000","last_unstale":"2024-03-23T14:57:56.262449+0000","last_undegraded":"2024-03-23T14:57:56.262449+0000","last_fullsized":"2024-03-23T14:57:56.262449+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:47.478078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:47.478078+0000","last_clean_scrub_stamp":"2024-03-23T14:56:47.478078+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:55:51.231901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574866,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27932,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343908,"statfs":{"total":96636764160,"available":96608161792,"internally_reserved":0,"allocated":933888,"data_stored":720551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574866,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27356,"kb_used_data":336,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344484,"statfs":{"total":96636764160,"available":96608751616,"internally_reserved":0,"allocated":344064,"data_stored":130709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574866,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27356,"kb_used_data":336,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344484,"statfs":{"total":96636764160,"available":96608751616,"internally_reserved":0,"allocated":344064,"data_stored":130765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574866,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27356,"kb_used_data":336,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344484,"statfs":{"total":96636764160,"available":96608751616,"internally_reserved":0,"allocated":344064,"data_stored":130197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574867,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27376,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344464,"statfs":{"total":96636764160,"available":96608731136,"internally_reserved":0,"allocated":364544,"data_stored":144395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574866,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27352,"kb_used_data":332,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344488,"statfs":{"total":96636764160,"available":96608755712,"internally_reserved":0,"allocated":339968,"data_stored":130163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574867,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":352256,"data_stored":138314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574865,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27928,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343912,"statfs":{"total":96636764160,"available":96608165888,"internally_reserved":0,"allocated":929792,"data_stored":720461,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-23T14:58:01.776 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-23T14:58:01.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.893+0000 7f3ea51ab640 1 Processor -- start 2024-03-23T14:58:01.896 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.893+0000 7f3ea51ab640 1 -- start start 2024-03-23T14:58:01.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.893+0000 7f3ea51ab640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f3ea00b0230 0x7f3ea00b34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:01.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.893+0000 7f3ea51ab640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3ea014db70 0x7f3ea014ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:01.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.893+0000 7f3ea51ab640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ea00b51d0 con 0x7f3ea0148cf0 2024-03-23T14:58:01.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.893+0000 7f3ea51ab640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 -- 0x7f3ea00b0610 con 0x7f3ea00b0230 2024-03-23T14:58:01.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.893+0000 7f3ea51ab640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f3ea00b1500 con 0x7f3ea014db70 2024-03-23T14:58:01.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.893+0000 7f3e9e59c640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f3ea0148cf0 0x7f3ea0149180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:43180/0 (socket says 172.21.15.164:43180) 2024-03-23T14:58:01.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.893+0000 7f3e9e59c640 1 -- 172.21.15.164:0/2083459759 learned_addr learned my addr 172.21.15.164:0/2083459759 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-03-23T14:58:01.899 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.893+0000 7f3e9f59e640 1 --2- 172.21.15.164:0/2083459759 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3ea014db70 0x7f3ea014ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:01.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.893+0000 7f3e9ed9d640 1 --2- 172.21.15.164:0/2083459759 >> v2:172.21.15.173:3300/0 conn(0x7f3ea00b0230 0x7f3ea00b34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:01.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.893+0000 7f3e9dd9b640 1 -- 172.21.15.164:0/2083459759 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3469011252 0 0) 0x7f3ea00b51d0 con 0x7f3ea0148cf0 2024-03-23T14:58:01.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.893+0000 7f3e9dd9b640 1 -- 172.21.15.164:0/2083459759 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3e88003530 con 0x7f3ea0148cf0 2024-03-23T14:58:01.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.893+0000 7f3e9dd9b640 1 -- 172.21.15.164:0/2083459759 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4136514144 0 0) 0x7f3e88003530 con 0x7f3ea0148cf0 2024-03-23T14:58:01.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.893+0000 7f3e9dd9b640 1 -- 172.21.15.164:0/2083459759 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3ea014db70 msgr2=0x7f3ea014ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:58:01.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.893+0000 7f3e9dd9b640 1 --2- 172.21.15.164:0/2083459759 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3ea014db70 0x7f3ea014ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:01.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.893+0000 7f3e9dd9b640 1 -- 172.21.15.164:0/2083459759 >> v2:172.21.15.173:3300/0 conn(0x7f3ea00b0230 msgr2=0x7f3ea00b34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:58:01.900 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.893+0000 7f3e9dd9b640 1 --2- 172.21.15.164:0/2083459759 >> v2:172.21.15.173:3300/0 conn(0x7f3ea00b0230 0x7f3ea00b34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:01.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.893+0000 7f3e9dd9b640 1 -- 172.21.15.164:0/2083459759 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ea006d250 con 0x7f3ea0148cf0 2024-03-23T14:58:01.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.893+0000 7f3e9dd9b640 1 -- 172.21.15.164:0/2083459759 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2526390548 0 0) 0x7f3e94001680 con 0x7f3ea0148cf0 2024-03-23T14:58:01.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.893+0000 7f3e9dd9b640 1 -- 172.21.15.164:0/2083459759 >> v1:172.21.15.164:6789/0 conn(0x7f3ea0148cf0 legacy=0x7f3ea0149180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:01.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.893+0000 7f3e9dd9b640 1 --2- 172.21.15.164:0/2083459759 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3ea014db70 0x7f3e88003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:01.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.893+0000 7f3e9dd9b640 1 --2- 172.21.15.164:0/2083459759 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3e88003f00 0x7f3e880047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:01.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.893+0000 7f3e9dd9b640 1 --2- 172.21.15.164:0/2083459759 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3e88004d10 0x7f3e88005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:01.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.893+0000 7f3e9dd9b640 1 -- 172.21.15.164:0/2083459759 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea00b0610 con 0x7f3e88003f00 2024-03-23T14:58:01.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.893+0000 7f3e9dd9b640 1 -- 172.21.15.164:0/2083459759 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea00b1500 con 0x7f3e88004d10 2024-03-23T14:58:01.901 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.893+0000 7f3e9dd9b640 1 -- 172.21.15.164:0/2083459759 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f3e8800b9e0 con 0x7f3ea014db70 2024-03-23T14:58:01.902 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.893+0000 7f3e9e59c640 1 --2- 172.21.15.164:0/2083459759 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3e88003f00 0x7f3e880047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:01.902 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.893+0000 7f3e9ed9d640 1 --2- 172.21.15.164:0/2083459759 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3ea014db70 0x7f3e88003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:01.902 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.893+0000 7f3e9f59e640 1 --2- 172.21.15.164:0/2083459759 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3e88004d10 0x7f3e88005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:01.902 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.893+0000 7f3e9e59c640 1 -- 172.21.15.164:0/2083459759 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3ea014db70 msgr2=0x7f3e88003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:58:01.902 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.893+0000 7f3e9e59c640 1 --2- 172.21.15.164:0/2083459759 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3ea014db70 0x7f3e88003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:01.902 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.893+0000 7f3e9e59c640 1 -- 172.21.15.164:0/2083459759 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3e88004d10 msgr2=0x7f3e88005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:58:01.902 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.893+0000 7f3e9e59c640 1 --2- 172.21.15.164:0/2083459759 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3e88004d10 0x7f3e88005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:01.902 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.893+0000 7f3e9e59c640 1 -- 172.21.15.164:0/2083459759 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e88003530 con 0x7f3e88003f00 2024-03-23T14:58:01.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.897+0000 7f3e9e59c640 1 --2- 172.21.15.164:0/2083459759 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3e88003f00 0x7f3e880047d0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f3e9400c2c0 tx=0x7f3e9401ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfd6bbf576445b25 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:58:01.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.897+0000 7f3e9dd9b640 1 -- 172.21.15.164:0/2083459759 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e940a5900 con 0x7f3e88003f00 2024-03-23T14:58:01.906 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.897+0000 7f3e9dd9b640 1 -- 172.21.15.164:0/2083459759 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e940a5a60 con 0x7f3e88003f00 2024-03-23T14:58:01.907 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.897+0000 7f3e9dd9b640 1 -- 172.21.15.164:0/2083459759 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e940a5e10 con 0x7f3e88003f00 2024-03-23T14:58:01.907 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.897+0000 7f3ea51ab640 1 -- 172.21.15.164:0/2083459759 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3e88003f00 msgr2=0x7f3e880047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:01.907 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.897+0000 7f3ea51ab640 1 --2- 172.21.15.164:0/2083459759 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3e88003f00 0x7f3e880047d0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f3e9400c2c0 tx=0x7f3e9401ccb0 comp rx=0 tx=0).stop 2024-03-23T14:58:01.907 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.897+0000 7f3e9f59e640 1 -- 172.21.15.164:0/2083459759 reap_dead start 2024-03-23T14:58:01.907 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.897+0000 7f3ea51ab640 1 -- 172.21.15.164:0/2083459759 shutdown_connections 2024-03-23T14:58:01.907 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.897+0000 7f3ea51ab640 1 -- 172.21.15.164:0/2083459759 >> 172.21.15.164:0/2083459759 conn(0x7f3ea00b8ca0 msgr2=0x7f3ea00b90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:58:01.907 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.897+0000 7f3ea51ab640 1 -- 172.21.15.164:0/2083459759 shutdown_connections 2024-03-23T14:58:01.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.897+0000 7f3ea51ab640 1 -- 172.21.15.164:0/2083459759 wait complete. 2024-03-23T14:58:01.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.897+0000 7f3ea51ab640 1 Processor -- start 2024-03-23T14:58:01.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.897+0000 7f3ea51ab640 1 -- start start 2024-03-23T14:58:01.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.897+0000 7f3ea51ab640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3e88003f00 0x7f3ea009c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:01.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.897+0000 7f3ea51ab640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3ea00af380 0x7f3ea006f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:01.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.897+0000 7f3ea51ab640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3ea01e2a90 0x7f3ea01e2f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:01.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.897+0000 7f3ea51ab640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea00af800 con 0x7f3ea01e2a90 2024-03-23T14:58:01.908 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.897+0000 7f3ea51ab640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea006f220 con 0x7f3ea00af380 2024-03-23T14:58:01.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.897+0000 7f3ea51ab640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f3ea006f470 con 0x7f3e88003f00 2024-03-23T14:58:01.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.897+0000 7f3e9f59e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3ea01e2a90 0x7f3ea01e2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:01.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.897+0000 7f3e9f59e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3ea01e2a90 0x7f3ea01e2f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:48882/0 (socket says 172.21.15.164:48882) 2024-03-23T14:58:01.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.897+0000 7f3e9f59e640 1 -- 172.21.15.164:0/2031410761 learned_addr learned my addr 172.21.15.164:0/2031410761 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:58:01.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.897+0000 7f3e9e59c640 1 --2- 172.21.15.164:0/2031410761 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3ea00af380 0x7f3ea006f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:01.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.897+0000 7f3e9ed9d640 1 --2- 172.21.15.164:0/2031410761 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3e88003f00 0x7f3ea009c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:01.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.897+0000 7f3e9f59e640 1 -- 172.21.15.164:0/2031410761 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3e88003f00 msgr2=0x7f3ea009c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:01.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.897+0000 7f3e9f59e640 1 --2- 172.21.15.164:0/2031410761 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3e88003f00 0x7f3ea009c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:01.909 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.897+0000 7f3e9f59e640 1 -- 172.21.15.164:0/2031410761 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3ea00af380 msgr2=0x7f3ea006f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:01.910 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.897+0000 7f3e9f59e640 1 --2- 172.21.15.164:0/2031410761 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3ea00af380 0x7f3ea006f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:01.910 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.897+0000 7f3e9f59e640 1 -- 172.21.15.164:0/2031410761 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ea0071520 con 0x7f3ea01e2a90 2024-03-23T14:58:01.910 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.897+0000 7f3e9f59e640 1 --2- 172.21.15.164:0/2031410761 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3ea01e2a90 0x7f3ea01e2f10 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f3ea00afea0 tx=0x7f3e980115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:58:01.910 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.897+0000 7f3e877fe640 1 -- 172.21.15.164:0/2031410761 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e9800e070 con 0x7f3ea01e2a90 2024-03-23T14:58:01.910 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.897+0000 7f3e877fe640 1 -- 172.21.15.164:0/2031410761 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e98011e40 con 0x7f3ea01e2a90 2024-03-23T14:58:01.910 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.897+0000 7f3ea51ab640 1 -- 172.21.15.164:0/2031410761 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ea0070060 con 0x7f3ea01e2a90 2024-03-23T14:58:01.910 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.897+0000 7f3e877fe640 1 -- 172.21.15.164:0/2031410761 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e980039a0 con 0x7f3ea01e2a90 2024-03-23T14:58:01.910 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.897+0000 7f3ea51ab640 1 -- 172.21.15.164:0/2031410761 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ea0071240 con 0x7f3ea01e2a90 2024-03-23T14:58:01.911 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.901+0000 7f3ea51ab640 1 -- 172.21.15.164:0/2031410761 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ea00690a0 con 0x7f3ea01e2a90 2024-03-23T14:58:01.911 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.901+0000 7f3e877fe640 1 -- 172.21.15.164:0/2031410761 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7f3e98003b00 con 0x7f3ea01e2a90 2024-03-23T14:58:01.911 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.901+0000 7f3e877fe640 1 --2- 172.21.15.164:0/2031410761 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f3e8c0672d0 0x7f3e8c069790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:01.911 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.901+0000 7f3e877fe640 1 -- 172.21.15.164:0/2031410761 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4943+0+0 (secure 0 0 0) 0x7f3e98085220 con 0x7f3ea01e2a90 2024-03-23T14:58:01.911 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.901+0000 7f3e9ed9d640 1 --2- 172.21.15.164:0/2031410761 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f3e8c0672d0 0x7f3e8c069790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:01.911 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.901+0000 7f3e9ed9d640 1 --2- 172.21.15.164:0/2031410761 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f3e8c0672d0 0x7f3e8c069790 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f3ea009d1e0 tx=0x7f3e90008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:58:01.912 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:01.905+0000 7f3e877fe640 1 -- 172.21.15.164:0/2031410761 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7f3e98052160 con 0x7f3ea01e2a90 2024-03-23T14:58:02.069 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:02.065+0000 7f3ea51ab640 1 -- 172.21.15.164:0/2031410761 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3ea00b2fa0 con 0x7f3ea01e2a90 2024-03-23T14:58:02.070 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:02.065+0000 7f3e877fe640 1 -- 172.21.15.164:0/2031410761 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) ==== 74+0+13046 (secure 0 0 0) 0x7f3e98055820 con 0x7f3ea01e2a90 2024-03-23T14:58:02.071 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-23T14:58:02.071 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":32,"fsid":"45c7911b-b69d-4e68-907b-fbfeb780aa3b","created":"2024-03-23T14:56:34.367633+0000","modified":"2024-03-23T14:57:59.221133+0000","last_up_change":"2024-03-23T14:56:46.297403+0000","last_in_change":"2024-03-23T14:56:36.629884+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-23T14:56:47.186157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"80abdc62-bd0d-4e60-a554-7ed6fd67cf54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":984716690},{"type":"v1","addr":"172.21.15.164:6801","nonce":984716690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":984716690},{"type":"v1","addr":"172.21.15.164:6803","nonce":984716690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":984716690},{"type":"v1","addr":"172.21.15.164:6807","nonce":984716690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":984716690},{"type":"v1","addr":"172.21.15.164:6805","nonce":984716690}]},"public_addr":"172.21.15.164:6801/984716690","cluster_addr":"172.21.15.164:6803/984716690","heartbeat_back_addr":"172.21.15.164:6807/984716690","heartbeat_front_addr":"172.21.15.164:6805/984716690","state":["exists","up"]},{"osd":1,"uuid":"f9001b06-be4d-4f42-a0e6-bcef2afa0d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":1349657239},{"type":"v1","addr":"172.21.15.164:6817","nonce":1349657239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":1349657239},{"type":"v1","addr":"172.21.15.164:6819","nonce":1349657239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":1349657239},{"type":"v1","addr":"172.21.15.164:6823","nonce":1349657239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":1349657239},{"type":"v1","addr":"172.21.15.164:6821","nonce":1349657239}]},"public_addr":"172.21.15.164:6817/1349657239","cluster_addr":"172.21.15.164:6819/1349657239","heartbeat_back_addr":"172.21.15.164:6823/1349657239","heartbeat_front_addr":"172.21.15.164:6821/1349657239","state":["exists","up"]},{"osd":2,"uuid":"fa0aeb9f-8b54-4831-a188-3e361c7980d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":887861252},{"type":"v1","addr":"172.21.15.164:6809","nonce":887861252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":887861252},{"type":"v1","addr":"172.21.15.164:6811","nonce":887861252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":887861252},{"type":"v1","addr":"172.21.15.164:6815","nonce":887861252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":887861252},{"type":"v1","addr":"172.21.15.164:6813","nonce":887861252}]},"public_addr":"172.21.15.164:6809/887861252","cluster_addr":"172.21.15.164:6811/887861252","heartbeat_back_addr":"172.21.15.164:6815/887861252","heartbeat_front_addr":"172.21.15.164:6813/887861252","state":["exists","up"]},{"osd":3,"uuid":"7e18c957-3091-46d5-8e17-8f3354c1726c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":812540271},{"type":"v1","addr":"172.21.15.164:6825","nonce":812540271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":812540271},{"type":"v1","addr":"172.21.15.164:6827","nonce":812540271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":812540271},{"type":"v1","addr":"172.21.15.164:6831","nonce":812540271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":812540271},{"type":"v1","addr":"172.21.15.164:6829","nonce":812540271}]},"public_addr":"172.21.15.164:6825/812540271","cluster_addr":"172.21.15.164:6827/812540271","heartbeat_back_addr":"172.21.15.164:6831/812540271","heartbeat_front_addr":"172.21.15.164:6829/812540271","state":["exists","up"]},{"osd":4,"uuid":"8e83f315-42bf-4471-9ce8-d81948da0f9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":2656857421},{"type":"v1","addr":"172.21.15.173:6825","nonce":2656857421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2656857421},{"type":"v1","addr":"172.21.15.173:6827","nonce":2656857421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2656857421},{"type":"v1","addr":"172.21.15.173:6831","nonce":2656857421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":2656857421},{"type":"v1","addr":"172.21.15.173:6829","nonce":2656857421}]},"public_addr":"172.21.15.173:6825/2656857421","cluster_addr":"172.21.15.173:6827/2656857421","heartbeat_back_addr":"172.21.15.173:6831/2656857421","heartbeat_front_addr":"172.21.15.173:6829/2656857421","state":["exists","up"]},{"osd":5,"uuid":"a230c872-71f9-40ab-8999-defd0162d4ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":3676483185},{"type":"v1","addr":"172.21.15.173:6801","nonce":3676483185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":3676483185},{"type":"v1","addr":"172.21.15.173:6803","nonce":3676483185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":3676483185},{"type":"v1","addr":"172.21.15.173:6807","nonce":3676483185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":3676483185},{"type":"v1","addr":"172.21.15.173:6805","nonce":3676483185}]},"public_addr":"172.21.15.173:6801/3676483185","cluster_addr":"172.21.15.173:6803/3676483185","heartbeat_back_addr":"172.21.15.173:6807/3676483185","heartbeat_front_addr":"172.21.15.173:6805/3676483185","state":["exists","up"]},{"osd":6,"uuid":"6193bfd7-8432-4f7f-a117-a3fcee21af99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":3631684087},{"type":"v1","addr":"172.21.15.173:6809","nonce":3631684087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":3631684087},{"type":"v1","addr":"172.21.15.173:6811","nonce":3631684087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":3631684087},{"type":"v1","addr":"172.21.15.173:6815","nonce":3631684087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":3631684087},{"type":"v1","addr":"172.21.15.173:6813","nonce":3631684087}]},"public_addr":"172.21.15.173:6809/3631684087","cluster_addr":"172.21.15.173:6811/3631684087","heartbeat_back_addr":"172.21.15.173:6815/3631684087","heartbeat_front_addr":"172.21.15.173:6813/3631684087","state":["exists","up"]},{"osd":7,"uuid":"b0f1b80a-afc4-4538-83b9-f081d2dcb8cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":3261114948},{"type":"v1","addr":"172.21.15.173:6817","nonce":3261114948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":3261114948},{"type":"v1","addr":"172.21.15.173:6819","nonce":3261114948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":3261114948},{"type":"v1","addr":"172.21.15.173:6823","nonce":3261114948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":3261114948},{"type":"v1","addr":"172.21.15.173:6821","nonce":3261114948}]},"public_addr":"172.21.15.173:6817/3261114948","cluster_addr":"172.21.15.173:6819/3261114948","heartbeat_back_addr":"172.21.15.173:6823/3261114948","heartbeat_front_addr":"172.21.15.173:6821/3261114948","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.188390+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.159932+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.191101+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.181776+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.224365+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.177515+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.153631+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-23T14:56:42.139972+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.164:6835/3261092607":"2024-03-24T14:57:55.875543+0000","172.21.15.164:6834/3261092607":"2024-03-24T14:57:55.875543+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-23T14:58:02.073 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:02.069+0000 7f3ea51ab640 1 -- 172.21.15.164:0/2031410761 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f3e8c0672d0 msgr2=0x7f3e8c069790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:02.073 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:02.069+0000 7f3ea51ab640 1 --2- 172.21.15.164:0/2031410761 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f3e8c0672d0 0x7f3e8c069790 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f3ea009d1e0 tx=0x7f3e90008040 comp rx=0 tx=0).stop 2024-03-23T14:58:02.073 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:02.069+0000 7f3ea51ab640 1 -- 172.21.15.164:0/2031410761 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3ea01e2a90 msgr2=0x7f3ea01e2f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:02.073 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:02.069+0000 7f3ea51ab640 1 --2- 172.21.15.164:0/2031410761 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3ea01e2a90 0x7f3ea01e2f10 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f3ea00afea0 tx=0x7f3e980115a0 comp rx=0 tx=0).stop 2024-03-23T14:58:02.073 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:02.069+0000 7f3ea51ab640 1 -- 172.21.15.164:0/2031410761 shutdown_connections 2024-03-23T14:58:02.073 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:02.069+0000 7f3ea51ab640 1 --2- 172.21.15.164:0/2031410761 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3ea01e2a90 0x7f3ea01e2f10 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:02.073 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:02.069+0000 7f3ea51ab640 1 --2- 172.21.15.164:0/2031410761 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f3e8c0672d0 0x7f3e8c069790 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:02.074 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:02.069+0000 7f3ea51ab640 1 --2- 172.21.15.164:0/2031410761 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3ea00af380 0x7f3ea006f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:02.074 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:02.069+0000 7f3ea51ab640 1 --2- 172.21.15.164:0/2031410761 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3e88003f00 0x7f3ea009c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:02.074 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:02.069+0000 7f3ea51ab640 1 -- 172.21.15.164:0/2031410761 >> 172.21.15.164:0/2031410761 conn(0x7f3ea00b8ca0 msgr2=0x7f3ea00b3c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:58:02.074 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:02.069+0000 7f3ea51ab640 1 -- 172.21.15.164:0/2031410761 shutdown_connections 2024-03-23T14:58:02.074 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:02.069+0000 7f3ea51ab640 1 -- 172.21.15.164:0/2031410761 wait complete. 2024-03-23T14:58:03.089 INFO:tasks.ceph:Scrubbing osd.0 2024-03-23T14:58:03.089 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-03-23T14:58:03.228 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.225+0000 7f3dcf003640 1 Processor -- start 2024-03-23T14:58:03.229 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.225+0000 7f3dcf003640 1 -- start start 2024-03-23T14:58:03.230 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.225+0000 7f3dcf003640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f3dc80b7630 0x7f3dc80b55a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:03.230 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.225+0000 7f3dcf003640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3dc80b6260 con 0x7f3dc80b7a10 2024-03-23T14:58:03.230 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.225+0000 7f3dcf003640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3dc80b5bc0 con 0x7f3dc8145260 2024-03-23T14:58:03.230 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.225+0000 7f3dcf003640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f3dc80b64b0 con 0x7f3dc80b7630 2024-03-23T14:58:03.231 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.225+0000 7f3dccd9f640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f3dc80b7630 0x7f3dc80b55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:03.231 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.225+0000 7f3dccd9f640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f3dc80b7630 0x7f3dc80b55a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:48840/0 (socket says 172.21.15.164:48840) 2024-03-23T14:58:03.231 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.225+0000 7f3dccd9f640 1 -- 172.21.15.164:0/3054651036 learned_addr learned my addr 172.21.15.164:0/3054651036 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:58:03.231 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.225+0000 7f3dccd9f640 1 -- 172.21.15.164:0/3054651036 >> v1:172.21.15.173:6789/0 conn(0x7f3dc8145260 legacy=0x7f3dc80b41d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:58:03.231 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.225+0000 7f3dccd9f640 1 -- 172.21.15.164:0/3054651036 >> v1:172.21.15.164:6789/0 conn(0x7f3dc80b7a10 legacy=0x7f3dc80b69d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:03.231 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.225+0000 7f3dccd9f640 1 -- 172.21.15.164:0/3054651036 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dc80b4f80 con 0x7f3dc80b7630 2024-03-23T14:58:03.231 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.225+0000 7f3dc77fe640 1 -- 172.21.15.164:0/3054651036 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1409316972 0 0) 0x7f3dc80b6260 con 0x7f3dc80b7a10 2024-03-23T14:58:03.232 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.229+0000 7f3dccd9f640 1 --2- 172.21.15.164:0/3054651036 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3dc80b7630 0x7f3dc80b55a0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f3db8009870 tx=0x7f3db801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85b7504175110c23 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:58:03.232 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.229+0000 7f3dc77fe640 1 -- 172.21.15.164:0/3054651036 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3db80aa610 con 0x7f3dc80b7630 2024-03-23T14:58:03.232 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.229+0000 7f3dc77fe640 1 -- 172.21.15.164:0/3054651036 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3db80a6040 con 0x7f3dc80b7630 2024-03-23T14:58:03.233 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.229+0000 7f3dc77fe640 1 -- 172.21.15.164:0/3054651036 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3db80aa990 con 0x7f3dc80b7630 2024-03-23T14:58:03.234 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.229+0000 7f3dcf003640 1 -- 172.21.15.164:0/3054651036 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3dc80b7630 msgr2=0x7f3dc80b55a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:03.234 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.229+0000 7f3dcf003640 1 --2- 172.21.15.164:0/3054651036 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3dc80b7630 0x7f3dc80b55a0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f3db8009870 tx=0x7f3db801c490 comp rx=0 tx=0).stop 2024-03-23T14:58:03.234 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.229+0000 7f3dcf003640 1 -- 172.21.15.164:0/3054651036 shutdown_connections 2024-03-23T14:58:03.234 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.229+0000 7f3dcf003640 1 --2- 172.21.15.164:0/3054651036 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3dc80b7630 0x7f3dc80b55a0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:03.234 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.229+0000 7f3dcf003640 1 -- 172.21.15.164:0/3054651036 >> 172.21.15.164:0/3054651036 conn(0x7f3dc802bfc0 msgr2=0x7f3dc802c3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:58:03.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.229+0000 7f3dcf003640 1 -- 172.21.15.164:0/3054651036 shutdown_connections 2024-03-23T14:58:03.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.229+0000 7f3dcf003640 1 -- 172.21.15.164:0/3054651036 wait complete. 2024-03-23T14:58:03.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.229+0000 7f3dcf003640 1 Processor -- start 2024-03-23T14:58:03.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.229+0000 7f3dcf003640 1 -- start start 2024-03-23T14:58:03.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.229+0000 7f3dcf003640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3dc80b7630 0x7f3dc8142220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:03.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.229+0000 7f3dcf003640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3dc80b7a10 0x7f3dc8142760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:03.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.229+0000 7f3dcf003640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3dc8145260 0x7f3dc81e30b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:03.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.229+0000 7f3dcf003640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f3dc8134ee0 con 0x7f3dc80b7630 2024-03-23T14:58:03.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.229+0000 7f3dcf003640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f3dc8142ca0 con 0x7f3dc8145260 2024-03-23T14:58:03.239 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.229+0000 7f3dcf003640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f3dc80b1210 con 0x7f3dc80b7a10 2024-03-23T14:58:03.240 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.229+0000 7f3dccd9f640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3dc80b7630 0x7f3dc8142220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:03.240 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.229+0000 7f3dccd9f640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3dc80b7630 0x7f3dc8142220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:48848/0 (socket says 172.21.15.164:48848) 2024-03-23T14:58:03.240 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.229+0000 7f3dccd9f640 1 -- 172.21.15.164:0/4114776696 learned_addr learned my addr 172.21.15.164:0/4114776696 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:58:03.240 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.229+0000 7f3dc7fff640 1 --2- 172.21.15.164:0/4114776696 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3dc80b7a10 0x7f3dc8142760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:03.240 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.229+0000 7f3dcd5a0640 1 --2- 172.21.15.164:0/4114776696 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3dc8145260 0x7f3dc81e30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:03.240 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.229+0000 7f3dccd9f640 1 -- 172.21.15.164:0/4114776696 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3dc80b7a10 msgr2=0x7f3dc8142760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:03.240 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.229+0000 7f3dccd9f640 1 --2- 172.21.15.164:0/4114776696 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3dc80b7a10 0x7f3dc8142760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:03.240 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.229+0000 7f3dccd9f640 1 -- 172.21.15.164:0/4114776696 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3dc8145260 msgr2=0x7f3dc81e30b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:03.241 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.229+0000 7f3dccd9f640 1 --2- 172.21.15.164:0/4114776696 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3dc8145260 0x7f3dc81e30b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:03.241 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.229+0000 7f3dccd9f640 1 -- 172.21.15.164:0/4114776696 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3db80094d0 con 0x7f3dc80b7630 2024-03-23T14:58:03.241 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.229+0000 7f3dcd5a0640 1 --2- 172.21.15.164:0/4114776696 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3dc8145260 0x7f3dc81e30b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T14:58:03.241 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.229+0000 7f3dccd9f640 1 --2- 172.21.15.164:0/4114776696 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3dc80b7630 0x7f3dc8142220 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f3db80190a0 tx=0x7f3db801acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:58:03.241 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.229+0000 7f3dc57fa640 1 -- 172.21.15.164:0/4114776696 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3db8004020 con 0x7f3dc80b7630 2024-03-23T14:58:03.241 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.229+0000 7f3dc57fa640 1 -- 172.21.15.164:0/4114776696 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3db80b7070 con 0x7f3dc80b7630 2024-03-23T14:58:03.241 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.229+0000 7f3dcf003640 1 -- 172.21.15.164:0/4114776696 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3dc8140a10 con 0x7f3dc80b7630 2024-03-23T14:58:03.241 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.229+0000 7f3dc57fa640 1 -- 172.21.15.164:0/4114776696 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3db80043a0 con 0x7f3dc80b7630 2024-03-23T14:58:03.242 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.229+0000 7f3dcf003640 1 -- 172.21.15.164:0/4114776696 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3dc8137570 con 0x7f3dc80b7630 2024-03-23T14:58:03.242 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.233+0000 7f3dcf003640 1 -- 172.21.15.164:0/4114776696 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3dc80b9e40 con 0x7f3dc80b7630 2024-03-23T14:58:03.242 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.233+0000 7f3dc57fa640 1 -- 172.21.15.164:0/4114776696 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7f3db8003620 con 0x7f3dc80b7630 2024-03-23T14:58:03.242 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.233+0000 7f3dc57fa640 1 --2- 172.21.15.164:0/4114776696 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f3db40672d0 0x7f3db4069790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:03.242 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.233+0000 7f3dc57fa640 1 -- 172.21.15.164:0/4114776696 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4943+0+0 (secure 0 0 0) 0x7f3dc8072c60 con 0x7f3dc80b7630 2024-03-23T14:58:03.242 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.233+0000 7f3dc7fff640 1 --2- 172.21.15.164:0/4114776696 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f3db40672d0 0x7f3db4069790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:03.242 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.233+0000 7f3dc57fa640 1 --2- 172.21.15.164:0/4114776696 >> [v2:172.21.15.164:6800/984716690,v1:172.21.15.164:6801/984716690] conn(0x7f3db406d190 0x7f3db406f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:03.242 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.233+0000 7f3dc57fa640 1 -- 172.21.15.164:0/4114776696 --> [v2:172.21.15.164:6800/984716690,v1:172.21.15.164:6801/984716690] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3db406fc60 con 0x7f3db406d190 2024-03-23T14:58:03.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.233+0000 7f3dc57fa640 1 -- 172.21.15.164:0/4114776696 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f3db811f750 con 0x7f3dc80b7630 2024-03-23T14:58:03.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.233+0000 7f3dcd5a0640 1 --2- 172.21.15.164:0/4114776696 >> [v2:172.21.15.164:6800/984716690,v1:172.21.15.164:6801/984716690] conn(0x7f3db406d190 0x7f3db406f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:03.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.233+0000 7f3dc7fff640 1 --2- 172.21.15.164:0/4114776696 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f3db40672d0 0x7f3db4069790 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f3dc802c880 tx=0x7f3dbc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:58:03.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.233+0000 7f3dcd5a0640 1 --2- 172.21.15.164:0/4114776696 >> [v2:172.21.15.164:6800/984716690,v1:172.21.15.164:6801/984716690] conn(0x7f3db406d190 0x7f3db406f5b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:58:03.243 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.233+0000 7f3dc57fa640 1 -- 172.21.15.164:0/4114776696 <== osd.0 v2:172.21.15.164:6800/984716690 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3db406fc60 con 0x7f3db406d190 2024-03-23T14:58:03.252 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.249+0000 7f3dcf003640 1 -- 172.21.15.164:0/4114776696 --> [v2:172.21.15.164:6800/984716690,v1:172.21.15.164:6801/984716690] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f3dc806ca90 con 0x7f3db406d190 2024-03-23T14:58:03.253 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.249+0000 7f3dc57fa640 1 -- 172.21.15.164:0/4114776696 <== osd.0 v2:172.21.15.164:6800/984716690 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f3dc806ca90 con 0x7f3db406d190 2024-03-23T14:58:03.253 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-03-23T14:58:03.253 INFO:teuthology.orchestra.run.smithi164.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-23T14:58:03.253 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-03-23T14:58:03.254 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.249+0000 7f3dcf003640 1 -- 172.21.15.164:0/4114776696 >> [v2:172.21.15.164:6800/984716690,v1:172.21.15.164:6801/984716690] conn(0x7f3db406d190 msgr2=0x7f3db406f5b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:03.254 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.249+0000 7f3dcf003640 1 --2- 172.21.15.164:0/4114776696 >> [v2:172.21.15.164:6800/984716690,v1:172.21.15.164:6801/984716690] conn(0x7f3db406d190 0x7f3db406f5b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:03.255 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.249+0000 7f3dcf003640 1 -- 172.21.15.164:0/4114776696 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f3db40672d0 msgr2=0x7f3db4069790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:03.255 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.249+0000 7f3dcf003640 1 --2- 172.21.15.164:0/4114776696 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f3db40672d0 0x7f3db4069790 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f3dc802c880 tx=0x7f3dbc008040 comp rx=0 tx=0).stop 2024-03-23T14:58:03.255 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.249+0000 7f3dcf003640 1 -- 172.21.15.164:0/4114776696 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3dc80b7630 msgr2=0x7f3dc8142220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:03.255 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.249+0000 7f3dcf003640 1 --2- 172.21.15.164:0/4114776696 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3dc80b7630 0x7f3dc8142220 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f3db80190a0 tx=0x7f3db801acd0 comp rx=0 tx=0).stop 2024-03-23T14:58:03.255 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.249+0000 7f3dcd5a0640 1 -- 172.21.15.164:0/4114776696 reap_dead start 2024-03-23T14:58:03.255 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.249+0000 7f3dcf003640 1 -- 172.21.15.164:0/4114776696 shutdown_connections 2024-03-23T14:58:03.255 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.249+0000 7f3dcf003640 1 -- 172.21.15.164:0/4114776696 >> 172.21.15.164:0/4114776696 conn(0x7f3dc802bfc0 msgr2=0x7f3dc8144850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:58:03.255 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.249+0000 7f3dcf003640 1 -- 172.21.15.164:0/4114776696 shutdown_connections 2024-03-23T14:58:03.255 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.249+0000 7f3dcf003640 1 -- 172.21.15.164:0/4114776696 wait complete. 2024-03-23T14:58:03.269 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-03-23T14:58:03.381 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.377+0000 7f9c138a8640 1 Processor -- start 2024-03-23T14:58:03.382 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.377+0000 7f9c138a8640 1 -- start start 2024-03-23T14:58:03.384 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.377+0000 7f9c138a8640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f9c0c0b0230 0x7f9c0c0b34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:03.384 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.377+0000 7f9c138a8640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9c0c14db70 0x7f9c0c14ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:03.384 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.377+0000 7f9c138a8640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c0c0b51d0 con 0x7f9c0c148cf0 2024-03-23T14:58:03.385 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.377+0000 7f9c138a8640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 -- 0x7f9c0c0b0610 con 0x7f9c0c0b0230 2024-03-23T14:58:03.385 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.377+0000 7f9c138a8640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f9c0c0b1500 con 0x7f9c0c14db70 2024-03-23T14:58:03.385 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.377+0000 7f9c10e43640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f9c0c148cf0 0x7f9c0c149180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:37794/0 (socket says 172.21.15.164:37794) 2024-03-23T14:58:03.385 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.377+0000 7f9c10e43640 1 -- 172.21.15.164:0/2594730911 learned_addr learned my addr 172.21.15.164:0/2594730911 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-03-23T14:58:03.385 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.377+0000 7f9c11e45640 1 --2- 172.21.15.164:0/2594730911 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9c0c14db70 0x7f9c0c14ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:03.385 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.377+0000 7f9c11644640 1 --2- 172.21.15.164:0/2594730911 >> v2:172.21.15.173:3300/0 conn(0x7f9c0c0b0230 0x7f9c0c0b34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:03.385 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.377+0000 7f9c0bfff640 1 -- 172.21.15.164:0/2594730911 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3417073002 0 0) 0x7f9c0c0b51d0 con 0x7f9c0c148cf0 2024-03-23T14:58:03.385 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.377+0000 7f9c0bfff640 1 -- 172.21.15.164:0/2594730911 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9bf4003530 con 0x7f9c0c148cf0 2024-03-23T14:58:03.386 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.377+0000 7f9c0bfff640 1 -- 172.21.15.164:0/2594730911 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1044839569 0 0) 0x7f9bf4003530 con 0x7f9c0c148cf0 2024-03-23T14:58:03.386 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.377+0000 7f9c0bfff640 1 -- 172.21.15.164:0/2594730911 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9c0c14db70 msgr2=0x7f9c0c14ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:58:03.386 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.377+0000 7f9c0bfff640 1 --2- 172.21.15.164:0/2594730911 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9c0c14db70 0x7f9c0c14ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:03.386 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.377+0000 7f9c0bfff640 1 -- 172.21.15.164:0/2594730911 >> v2:172.21.15.173:3300/0 conn(0x7f9c0c0b0230 msgr2=0x7f9c0c0b34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:58:03.386 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.377+0000 7f9c0bfff640 1 --2- 172.21.15.164:0/2594730911 >> v2:172.21.15.173:3300/0 conn(0x7f9c0c0b0230 0x7f9c0c0b34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:03.386 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.377+0000 7f9c0bfff640 1 -- 172.21.15.164:0/2594730911 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c0c06d250 con 0x7f9c0c148cf0 2024-03-23T14:58:03.386 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.377+0000 7f9c0bfff640 1 -- 172.21.15.164:0/2594730911 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2526390548 0 0) 0x7f9c00001680 con 0x7f9c0c148cf0 2024-03-23T14:58:03.387 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.377+0000 7f9c0bfff640 1 -- 172.21.15.164:0/2594730911 >> v1:172.21.15.164:6789/0 conn(0x7f9c0c148cf0 legacy=0x7f9c0c149180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:03.387 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.377+0000 7f9c0bfff640 1 --2- 172.21.15.164:0/2594730911 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9bf4003980 0x7f9bf4003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:03.387 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.377+0000 7f9c0bfff640 1 --2- 172.21.15.164:0/2594730911 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9c0c14db70 0x7f9bf40047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:03.387 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.377+0000 7f9c0bfff640 1 --2- 172.21.15.164:0/2594730911 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9bf4004d10 0x7f9bf4005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:03.387 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.377+0000 7f9c0bfff640 1 -- 172.21.15.164:0/2594730911 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9c0c0b0610 con 0x7f9bf4004d10 2024-03-23T14:58:03.387 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.377+0000 7f9c0bfff640 1 -- 172.21.15.164:0/2594730911 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f9c0c0b1500 con 0x7f9bf4003980 2024-03-23T14:58:03.387 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.377+0000 7f9c0bfff640 1 -- 172.21.15.164:0/2594730911 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f9bf400b990 con 0x7f9c0c14db70 2024-03-23T14:58:03.387 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.377+0000 7f9c11e45640 1 --2- 172.21.15.164:0/2594730911 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9bf4004d10 0x7f9bf4005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:03.388 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.377+0000 7f9c10e43640 1 --2- 172.21.15.164:0/2594730911 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9c0c14db70 0x7f9bf40047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:03.388 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.377+0000 7f9c11644640 1 --2- 172.21.15.164:0/2594730911 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9bf4003980 0x7f9bf4003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:03.388 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.381+0000 7f9c11e45640 1 -- 172.21.15.164:0/2594730911 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9c0c14db70 msgr2=0x7f9bf40047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:58:03.388 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.381+0000 7f9c11e45640 1 --2- 172.21.15.164:0/2594730911 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9c0c14db70 0x7f9bf40047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:03.388 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.381+0000 7f9c11e45640 1 -- 172.21.15.164:0/2594730911 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9bf4003980 msgr2=0x7f9bf4003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:58:03.388 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.381+0000 7f9c11e45640 1 --2- 172.21.15.164:0/2594730911 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9bf4003980 0x7f9bf4003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:03.389 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.381+0000 7f9c11e45640 1 -- 172.21.15.164:0/2594730911 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bf4003530 con 0x7f9bf4004d10 2024-03-23T14:58:03.389 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.381+0000 7f9c10e43640 1 --2- 172.21.15.164:0/2594730911 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9c0c14db70 0x7f9bf40047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T14:58:03.392 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.381+0000 7f9c11e45640 1 --2- 172.21.15.164:0/2594730911 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9bf4004d10 0x7f9bf4005190 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f9c04009ae0 tx=0x7f9c0401ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f27f1d9e24ea348c server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:58:03.392 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.381+0000 7f9c0bfff640 1 -- 172.21.15.164:0/2594730911 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c040a5660 con 0x7f9bf4004d10 2024-03-23T14:58:03.392 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.381+0000 7f9c0bfff640 1 -- 172.21.15.164:0/2594730911 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c040a57c0 con 0x7f9bf4004d10 2024-03-23T14:58:03.392 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.381+0000 7f9c0bfff640 1 -- 172.21.15.164:0/2594730911 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c040a5b70 con 0x7f9bf4004d10 2024-03-23T14:58:03.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.381+0000 7f9c138a8640 1 -- 172.21.15.164:0/2594730911 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9bf4004d10 msgr2=0x7f9bf4005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:03.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.381+0000 7f9c138a8640 1 --2- 172.21.15.164:0/2594730911 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9bf4004d10 0x7f9bf4005190 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f9c04009ae0 tx=0x7f9c0401ca70 comp rx=0 tx=0).stop 2024-03-23T14:58:03.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.381+0000 7f9c11e45640 1 -- 172.21.15.164:0/2594730911 reap_dead start 2024-03-23T14:58:03.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.381+0000 7f9c138a8640 1 -- 172.21.15.164:0/2594730911 shutdown_connections 2024-03-23T14:58:03.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.381+0000 7f9c138a8640 1 -- 172.21.15.164:0/2594730911 >> 172.21.15.164:0/2594730911 conn(0x7f9c0c0b8ca0 msgr2=0x7f9c0c0b90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:58:03.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.381+0000 7f9c138a8640 1 -- 172.21.15.164:0/2594730911 shutdown_connections 2024-03-23T14:58:03.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.381+0000 7f9c138a8640 1 -- 172.21.15.164:0/2594730911 wait complete. 2024-03-23T14:58:03.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.381+0000 7f9c138a8640 1 Processor -- start 2024-03-23T14:58:03.393 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.385+0000 7f9c138a8640 1 -- start start 2024-03-23T14:58:03.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.385+0000 7f9c138a8640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9c0c1e0380 0x7f9c0c09ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:03.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.385+0000 7f9c138a8640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9c0c0af000 0x7f9c0c09cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:03.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.385+0000 7f9c138a8640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9c0c09d2a0 0x7f9c0c0ada30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:03.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.385+0000 7f9c138a8640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9c0c09cb80 con 0x7f9c0c09d2a0 2024-03-23T14:58:03.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.385+0000 7f9c138a8640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f9c0c09b190 con 0x7f9c0c0af000 2024-03-23T14:58:03.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.385+0000 7f9c138a8640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f9c0c072180 con 0x7f9c0c1e0380 2024-03-23T14:58:03.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.385+0000 7f9c11e45640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9c0c09d2a0 0x7f9c0c0ada30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:03.394 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.385+0000 7f9c11e45640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9c0c09d2a0 0x7f9c0c0ada30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:48858/0 (socket says 172.21.15.164:48858) 2024-03-23T14:58:03.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.385+0000 7f9c11e45640 1 -- 172.21.15.164:0/4195664978 learned_addr learned my addr 172.21.15.164:0/4195664978 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:58:03.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.385+0000 7f9c10e43640 1 --2- 172.21.15.164:0/4195664978 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9c0c0af000 0x7f9c0c09cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:03.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.385+0000 7f9c11644640 1 --2- 172.21.15.164:0/4195664978 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9c0c1e0380 0x7f9c0c09ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:03.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.385+0000 7f9c11e45640 1 -- 172.21.15.164:0/4195664978 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9c0c1e0380 msgr2=0x7f9c0c09ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:03.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.385+0000 7f9c11e45640 1 --2- 172.21.15.164:0/4195664978 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9c0c1e0380 0x7f9c0c09ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:03.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.385+0000 7f9c11e45640 1 -- 172.21.15.164:0/4195664978 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9c0c0af000 msgr2=0x7f9c0c09cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:03.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.385+0000 7f9c11e45640 1 --2- 172.21.15.164:0/4195664978 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9c0c0af000 0x7f9c0c09cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:03.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.385+0000 7f9c11e45640 1 -- 172.21.15.164:0/4195664978 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c0c1e0760 con 0x7f9c0c09d2a0 2024-03-23T14:58:03.396 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.385+0000 7f9c10e43640 1 --2- 172.21.15.164:0/4195664978 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9c0c0af000 0x7f9c0c09cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T14:58:03.396 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.385+0000 7f9c11e45640 1 --2- 172.21.15.164:0/4195664978 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9c0c09d2a0 0x7f9c0c0ada30 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f9c040b0040 tx=0x7f9c04019810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:58:03.396 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.385+0000 7f9c09ffb640 1 -- 172.21.15.164:0/4195664978 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c040a5660 con 0x7f9c0c09d2a0 2024-03-23T14:58:03.396 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.385+0000 7f9c09ffb640 1 -- 172.21.15.164:0/4195664978 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c04004220 con 0x7f9c0c09d2a0 2024-03-23T14:58:03.396 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.385+0000 7f9c138a8640 1 -- 172.21.15.164:0/4195664978 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c0c1e0fc0 con 0x7f9c0c09d2a0 2024-03-23T14:58:03.396 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.385+0000 7f9c09ffb640 1 -- 172.21.15.164:0/4195664978 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c040a5e30 con 0x7f9c0c09d2a0 2024-03-23T14:58:03.396 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.385+0000 7f9c138a8640 1 -- 172.21.15.164:0/4195664978 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c0c09d870 con 0x7f9c0c09d2a0 2024-03-23T14:58:03.396 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.385+0000 7f9c138a8640 1 -- 172.21.15.164:0/4195664978 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c0c071fe0 con 0x7f9c0c09d2a0 2024-03-23T14:58:03.396 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.385+0000 7f9c09ffb640 1 -- 172.21.15.164:0/4195664978 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7f9c040b4070 con 0x7f9c0c09d2a0 2024-03-23T14:58:03.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.389+0000 7f9c09ffb640 1 --2- 172.21.15.164:0/4195664978 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f9bf80672d0 0x7f9bf8069790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:03.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.389+0000 7f9c11644640 1 --2- 172.21.15.164:0/4195664978 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f9bf80672d0 0x7f9bf8069790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:03.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.389+0000 7f9c09ffb640 1 -- 172.21.15.164:0/4195664978 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4943+0+0 (secure 0 0 0) 0x7f9c0411eca0 con 0x7f9c0c09d2a0 2024-03-23T14:58:03.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.389+0000 7f9c11644640 1 --2- 172.21.15.164:0/4195664978 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f9bf80672d0 0x7f9bf8069790 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc009fd0 tx=0x7f9bfc00b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:58:03.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.389+0000 7f9c09ffb640 1 -- 172.21.15.164:0/4195664978 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7f9c040ebc50 con 0x7f9c0c09d2a0 2024-03-23T14:58:03.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.541+0000 7f9c138a8640 1 -- 172.21.15.164:0/4195664978 --> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7f9c0c06c0a0 con 0x7f9bf80672d0 2024-03-23T14:58:03.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.541+0000 7f9c09ffb640 1 -- 172.21.15.164:0/4195664978 <== mgr.4107 v2:172.21.15.164:6832/2139087985 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f9bfc0039a0 con 0x7f9bf80672d0 2024-03-23T14:58:03.547 INFO:teuthology.orchestra.run.smithi164.stderr:instructed osd(s) 0 to deep-scrub 2024-03-23T14:58:03.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.545+0000 7f9c138a8640 1 -- 172.21.15.164:0/4195664978 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f9bf80672d0 msgr2=0x7f9bf8069790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:03.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.545+0000 7f9c138a8640 1 --2- 172.21.15.164:0/4195664978 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f9bf80672d0 0x7f9bf8069790 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc009fd0 tx=0x7f9bfc00b040 comp rx=0 tx=0).stop 2024-03-23T14:58:03.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.545+0000 7f9c138a8640 1 -- 172.21.15.164:0/4195664978 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9c0c09d2a0 msgr2=0x7f9c0c0ada30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:03.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.545+0000 7f9c138a8640 1 --2- 172.21.15.164:0/4195664978 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9c0c09d2a0 0x7f9c0c0ada30 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f9c040b0040 tx=0x7f9c04019810 comp rx=0 tx=0).stop 2024-03-23T14:58:03.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.545+0000 7f9c138a8640 1 -- 172.21.15.164:0/4195664978 shutdown_connections 2024-03-23T14:58:03.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.545+0000 7f9c138a8640 1 --2- 172.21.15.164:0/4195664978 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9c0c09d2a0 0x7f9c0c0ada30 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:03.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.545+0000 7f9c138a8640 1 --2- 172.21.15.164:0/4195664978 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f9bf80672d0 0x7f9bf8069790 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:03.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.545+0000 7f9c138a8640 1 --2- 172.21.15.164:0/4195664978 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9c0c0af000 0x7f9c0c09cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:03.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.545+0000 7f9c138a8640 1 --2- 172.21.15.164:0/4195664978 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9c0c1e0380 0x7f9c0c09ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:03.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.545+0000 7f9c138a8640 1 -- 172.21.15.164:0/4195664978 >> 172.21.15.164:0/4195664978 conn(0x7f9c0c0b8ca0 msgr2=0x7f9c0c145640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:58:03.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.545+0000 7f9c138a8640 1 -- 172.21.15.164:0/4195664978 shutdown_connections 2024-03-23T14:58:03.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.545+0000 7f9c138a8640 1 -- 172.21.15.164:0/4195664978 wait complete. 2024-03-23T14:58:03.565 INFO:tasks.ceph:Scrubbing osd.1 2024-03-23T14:58:03.565 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-03-23T14:58:03.685 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.681+0000 7f581978f640 1 Processor -- start 2024-03-23T14:58:03.685 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.681+0000 7f581978f640 1 -- start start 2024-03-23T14:58:03.686 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.681+0000 7f581978f640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f581409def0 0x7f58140ae620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:03.686 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.681+0000 7f581978f640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f58140b4e90 0x7f581414e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:03.686 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.681+0000 7f581978f640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f581409e2d0 con 0x7f58140b4a00 2024-03-23T14:58:03.687 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.681+0000 7f581978f640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f5814146de0 con 0x7f581409def0 2024-03-23T14:58:03.687 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.681+0000 7f581978f640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 -- 0x7f58140b52e0 con 0x7f58140b4e90 2024-03-23T14:58:03.687 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.681+0000 7f5812ffd640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f581409def0 0x7f58140ae620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:03.687 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.681+0000 7f5812ffd640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f581409def0 0x7f58140ae620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:48870/0 (socket says 172.21.15.164:48870) 2024-03-23T14:58:03.687 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.681+0000 7f5812ffd640 1 -- 172.21.15.164:0/2553919793 learned_addr learned my addr 172.21.15.164:0/2553919793 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:58:03.687 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.681+0000 7f58137fe640 1 --2- 172.21.15.164:0/2553919793 >> v2:172.21.15.173:3300/0 conn(0x7f58140b4e90 0x7f581414e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:03.687 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.681+0000 7f5812ffd640 1 -- 172.21.15.164:0/2553919793 >> v2:172.21.15.173:3300/0 conn(0x7f58140b4e90 msgr2=0x7f581414e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:58:03.687 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.681+0000 7f5812ffd640 1 --2- 172.21.15.164:0/2553919793 >> v2:172.21.15.173:3300/0 conn(0x7f58140b4e90 0x7f581414e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:03.688 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.681+0000 7f5812ffd640 1 -- 172.21.15.164:0/2553919793 >> v1:172.21.15.173:6789/0 conn(0x7f58140b4a00 legacy=0x7f5814147420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:58:03.688 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.681+0000 7f5812ffd640 1 -- 172.21.15.164:0/2553919793 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f581406cb70 con 0x7f581409def0 2024-03-23T14:58:03.688 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.681+0000 7f58137fe640 1 --2- 172.21.15.164:0/2553919793 >> v2:172.21.15.173:3300/0 conn(0x7f58140b4e90 0x7f581414e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T14:58:03.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.685+0000 7f5812ffd640 1 --2- 172.21.15.164:0/2553919793 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f581409def0 0x7f58140ae620 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f58140af080 tx=0x7f580401d790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fea99c2868106d3 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:58:03.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.685+0000 7f5811ffb640 1 -- 172.21.15.164:0/2553919793 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58040a6730 con 0x7f581409def0 2024-03-23T14:58:03.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.685+0000 7f5811ffb640 1 -- 172.21.15.164:0/2553919793 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58040a6890 con 0x7f581409def0 2024-03-23T14:58:03.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.685+0000 7f5811ffb640 1 -- 172.21.15.164:0/2553919793 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58040a6c20 con 0x7f581409def0 2024-03-23T14:58:03.690 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.685+0000 7f581978f640 1 -- 172.21.15.164:0/2553919793 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f581409def0 msgr2=0x7f58140ae620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:03.690 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.685+0000 7f581978f640 1 --2- 172.21.15.164:0/2553919793 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f581409def0 0x7f58140ae620 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f58140af080 tx=0x7f580401d790 comp rx=0 tx=0).stop 2024-03-23T14:58:03.690 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.685+0000 7f581978f640 1 -- 172.21.15.164:0/2553919793 shutdown_connections 2024-03-23T14:58:03.691 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.685+0000 7f581978f640 1 --2- 172.21.15.164:0/2553919793 >> v2:172.21.15.173:3300/0 conn(0x7f58140b4e90 0x7f581414e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:03.691 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.685+0000 7f581978f640 1 --2- 172.21.15.164:0/2553919793 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f581409def0 0x7f58140ae620 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:03.691 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.685+0000 7f581978f640 1 -- 172.21.15.164:0/2553919793 >> 172.21.15.164:0/2553919793 conn(0x7f58140b8ca0 msgr2=0x7f58140b90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:58:03.691 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.685+0000 7f581978f640 1 -- 172.21.15.164:0/2553919793 shutdown_connections 2024-03-23T14:58:03.691 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.685+0000 7f581978f640 1 -- 172.21.15.164:0/2553919793 wait complete. 2024-03-23T14:58:03.691 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.685+0000 7f581978f640 1 Processor -- start 2024-03-23T14:58:03.691 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.685+0000 7f581978f640 1 -- start start 2024-03-23T14:58:03.693 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.685+0000 7f581978f640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f581409def0 0x7f58141401e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:03.694 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.685+0000 7f581978f640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f58140b4a00 0x7f58140b1a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:03.694 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.685+0000 7f581978f640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f58140b4e90 0x7f58140b08e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:03.694 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.685+0000 7f581978f640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5814071790 con 0x7f581409def0 2024-03-23T14:58:03.694 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.685+0000 7f581978f640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f5814071540 con 0x7f58140b4a00 2024-03-23T14:58:03.694 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.685+0000 7f581978f640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f58140b1fb0 con 0x7f58140b4e90 2024-03-23T14:58:03.694 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.685+0000 7f5812ffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f581409def0 0x7f58141401e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:03.694 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.685+0000 7f5812ffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f581409def0 0x7f58141401e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:48878/0 (socket says 172.21.15.164:48878) 2024-03-23T14:58:03.695 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.685+0000 7f5812ffd640 1 -- 172.21.15.164:0/1060823314 learned_addr learned my addr 172.21.15.164:0/1060823314 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:58:03.695 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.689+0000 7f58127fc640 1 --2- 172.21.15.164:0/1060823314 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f58140b4a00 0x7f58140b1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:03.695 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.689+0000 7f58137fe640 1 --2- 172.21.15.164:0/1060823314 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f58140b4e90 0x7f58140b08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:03.695 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.689+0000 7f5812ffd640 1 -- 172.21.15.164:0/1060823314 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f58140b4e90 msgr2=0x7f58140b08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:03.695 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.689+0000 7f5812ffd640 1 --2- 172.21.15.164:0/1060823314 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f58140b4e90 0x7f58140b08e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:03.695 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.689+0000 7f5812ffd640 1 -- 172.21.15.164:0/1060823314 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f58140b4a00 msgr2=0x7f58140b1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:03.695 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.689+0000 7f5812ffd640 1 --2- 172.21.15.164:0/1060823314 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f58140b4a00 0x7f58140b1a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:03.695 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.689+0000 7f5812ffd640 1 -- 172.21.15.164:0/1060823314 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f580400a670 con 0x7f581409def0 2024-03-23T14:58:03.696 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.689+0000 7f5812ffd640 1 --2- 172.21.15.164:0/1060823314 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f581409def0 0x7f58141401e0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f580400a1c0 tx=0x7f580401bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:58:03.696 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.689+0000 7f57fb7fe640 1 -- 172.21.15.164:0/1060823314 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58040ae8d0 con 0x7f581409def0 2024-03-23T14:58:03.696 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.689+0000 7f57fb7fe640 1 -- 172.21.15.164:0/1060823314 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58040aea30 con 0x7f581409def0 2024-03-23T14:58:03.696 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.689+0000 7f57fb7fe640 1 -- 172.21.15.164:0/1060823314 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58040aede0 con 0x7f581409def0 2024-03-23T14:58:03.696 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.689+0000 7f581978f640 1 -- 172.21.15.164:0/1060823314 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f581413d930 con 0x7f581409def0 2024-03-23T14:58:03.696 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.689+0000 7f581978f640 1 -- 172.21.15.164:0/1060823314 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f581406f2e0 con 0x7f581409def0 2024-03-23T14:58:03.696 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.689+0000 7f581978f640 1 -- 172.21.15.164:0/1060823314 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f58140aec50 con 0x7f581409def0 2024-03-23T14:58:03.696 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.689+0000 7f57fb7fe640 1 -- 172.21.15.164:0/1060823314 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7f58040b5070 con 0x7f581409def0 2024-03-23T14:58:03.697 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.689+0000 7f57fb7fe640 1 --2- 172.21.15.164:0/1060823314 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f58000672a0 0x7f5800069760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:03.698 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.689+0000 7f57fb7fe640 1 -- 172.21.15.164:0/1060823314 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4943+0+0 (secure 0 0 0) 0x7f580411ffd0 con 0x7f581409def0 2024-03-23T14:58:03.698 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.689+0000 7f58127fc640 1 --2- 172.21.15.164:0/1060823314 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f58000672a0 0x7f5800069760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:03.698 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.689+0000 7f57fb7fe640 1 --2- 172.21.15.164:0/1060823314 >> [v2:172.21.15.164:6816/1349657239,v1:172.21.15.164:6817/1349657239] conn(0x7f580006d160 0x7f580006f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:03.698 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.689+0000 7f57fb7fe640 1 -- 172.21.15.164:0/1060823314 --> [v2:172.21.15.164:6816/1349657239,v1:172.21.15.164:6817/1349657239] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f580006fc30 con 0x7f580006d160 2024-03-23T14:58:03.698 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.689+0000 7f57fb7fe640 1 -- 172.21.15.164:0/1060823314 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f5804120420 con 0x7f581409def0 2024-03-23T14:58:03.698 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.689+0000 7f58137fe640 1 --2- 172.21.15.164:0/1060823314 >> [v2:172.21.15.164:6816/1349657239,v1:172.21.15.164:6817/1349657239] conn(0x7f580006d160 0x7f580006f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:03.699 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.689+0000 7f58127fc640 1 --2- 172.21.15.164:0/1060823314 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f58000672a0 0x7f5800069760 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f58140b7e50 tx=0x7f5808009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:58:03.699 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.689+0000 7f58137fe640 1 --2- 172.21.15.164:0/1060823314 >> [v2:172.21.15.164:6816/1349657239,v1:172.21.15.164:6817/1349657239] conn(0x7f580006d160 0x7f580006f580 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:58:03.699 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.693+0000 7f57fb7fe640 1 -- 172.21.15.164:0/1060823314 <== osd.1 v2:172.21.15.164:6816/1349657239 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f580006fc30 con 0x7f580006d160 2024-03-23T14:58:03.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.705+0000 7f581978f640 1 -- 172.21.15.164:0/1060823314 --> [v2:172.21.15.164:6816/1349657239,v1:172.21.15.164:6817/1349657239] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f58140690a0 con 0x7f580006d160 2024-03-23T14:58:03.710 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.705+0000 7f57fb7fe640 1 -- 172.21.15.164:0/1060823314 <== osd.1 v2:172.21.15.164:6816/1349657239 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f58140690a0 con 0x7f580006d160 2024-03-23T14:58:03.710 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-03-23T14:58:03.710 INFO:teuthology.orchestra.run.smithi164.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-23T14:58:03.711 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-03-23T14:58:03.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.705+0000 7f581978f640 1 -- 172.21.15.164:0/1060823314 >> [v2:172.21.15.164:6816/1349657239,v1:172.21.15.164:6817/1349657239] conn(0x7f580006d160 msgr2=0x7f580006f580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:03.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.705+0000 7f581978f640 1 --2- 172.21.15.164:0/1060823314 >> [v2:172.21.15.164:6816/1349657239,v1:172.21.15.164:6817/1349657239] conn(0x7f580006d160 0x7f580006f580 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:03.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.705+0000 7f581978f640 1 -- 172.21.15.164:0/1060823314 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f58000672a0 msgr2=0x7f5800069760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:03.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.705+0000 7f581978f640 1 --2- 172.21.15.164:0/1060823314 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f58000672a0 0x7f5800069760 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f58140b7e50 tx=0x7f5808009040 comp rx=0 tx=0).stop 2024-03-23T14:58:03.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.705+0000 7f581978f640 1 -- 172.21.15.164:0/1060823314 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f581409def0 msgr2=0x7f58141401e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:03.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.705+0000 7f581978f640 1 --2- 172.21.15.164:0/1060823314 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f581409def0 0x7f58141401e0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f580400a1c0 tx=0x7f580401bfd0 comp rx=0 tx=0).stop 2024-03-23T14:58:03.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.705+0000 7f58137fe640 1 -- 172.21.15.164:0/1060823314 reap_dead start 2024-03-23T14:58:03.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.705+0000 7f581978f640 1 -- 172.21.15.164:0/1060823314 shutdown_connections 2024-03-23T14:58:03.712 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.705+0000 7f581978f640 1 -- 172.21.15.164:0/1060823314 >> 172.21.15.164:0/1060823314 conn(0x7f58140b8ca0 msgr2=0x7f5814145760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:58:03.713 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.705+0000 7f581978f640 1 -- 172.21.15.164:0/1060823314 shutdown_connections 2024-03-23T14:58:03.713 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.705+0000 7f581978f640 1 -- 172.21.15.164:0/1060823314 wait complete. 2024-03-23T14:58:03.724 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-03-23T14:58:03.829 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.825+0000 7fd6c2446640 1 Processor -- start 2024-03-23T14:58:03.830 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.825+0000 7fd6c2446640 1 -- start start 2024-03-23T14:58:03.832 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.825+0000 7fd6c2446640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fd6bc0b0230 0x7fd6bc0b34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:03.832 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.825+0000 7fd6c2446640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd6bc14db70 0x7fd6bc14ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:03.832 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.825+0000 7fd6c2446640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6bc0b51d0 con 0x7fd6bc148cf0 2024-03-23T14:58:03.832 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.825+0000 7fd6c2446640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 -- 0x7fd6bc0b0610 con 0x7fd6bc0b0230 2024-03-23T14:58:03.833 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.825+0000 7fd6c2446640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fd6bc0b1500 con 0x7fd6bc14db70 2024-03-23T14:58:03.833 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.825+0000 7fd6bb7fe640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fd6bc148cf0 0x7fd6bc149180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:37802/0 (socket says 172.21.15.164:37802) 2024-03-23T14:58:03.833 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.825+0000 7fd6bb7fe640 1 -- 172.21.15.164:0/663760443 learned_addr learned my addr 172.21.15.164:0/663760443 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-03-23T14:58:03.833 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.825+0000 7fd6c09e3640 1 --2- 172.21.15.164:0/663760443 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd6bc14db70 0x7fd6bc14ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:03.833 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.825+0000 7fd6bbfff640 1 --2- 172.21.15.164:0/663760443 >> v2:172.21.15.173:3300/0 conn(0x7fd6bc0b0230 0x7fd6bc0b34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:03.833 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.825+0000 7fd6baffd640 1 -- 172.21.15.164:0/663760443 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1152902736 0 0) 0x7fd6bc0b51d0 con 0x7fd6bc148cf0 2024-03-23T14:58:03.833 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.825+0000 7fd6baffd640 1 -- 172.21.15.164:0/663760443 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd6a8003530 con 0x7fd6bc148cf0 2024-03-23T14:58:03.833 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.825+0000 7fd6baffd640 1 -- 172.21.15.164:0/663760443 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3054543380 0 0) 0x7fd6a8003530 con 0x7fd6bc148cf0 2024-03-23T14:58:03.834 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.825+0000 7fd6baffd640 1 -- 172.21.15.164:0/663760443 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd6bc14db70 msgr2=0x7fd6bc14ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:58:03.834 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.825+0000 7fd6baffd640 1 --2- 172.21.15.164:0/663760443 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd6bc14db70 0x7fd6bc14ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:03.834 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.825+0000 7fd6baffd640 1 -- 172.21.15.164:0/663760443 >> v2:172.21.15.173:3300/0 conn(0x7fd6bc0b0230 msgr2=0x7fd6bc0b34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:58:03.834 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.825+0000 7fd6baffd640 1 --2- 172.21.15.164:0/663760443 >> v2:172.21.15.173:3300/0 conn(0x7fd6bc0b0230 0x7fd6bc0b34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:03.834 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.825+0000 7fd6baffd640 1 -- 172.21.15.164:0/663760443 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6bc06d250 con 0x7fd6bc148cf0 2024-03-23T14:58:03.834 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.825+0000 7fd6baffd640 1 -- 172.21.15.164:0/663760443 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2526390548 0 0) 0x7fd6ac001a00 con 0x7fd6bc148cf0 2024-03-23T14:58:03.834 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.825+0000 7fd6baffd640 1 -- 172.21.15.164:0/663760443 >> v1:172.21.15.164:6789/0 conn(0x7fd6bc148cf0 legacy=0x7fd6bc149180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:03.834 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.825+0000 7fd6baffd640 1 --2- 172.21.15.164:0/663760443 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd6a8003980 0x7fd6a8003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:03.835 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.825+0000 7fd6baffd640 1 --2- 172.21.15.164:0/663760443 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd6a80047d0 0x7fd6a8004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:03.835 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.825+0000 7fd6baffd640 1 --2- 172.21.15.164:0/663760443 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd6a8005190 0x7fd6a800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:03.835 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.825+0000 7fd6baffd640 1 -- 172.21.15.164:0/663760443 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd6bc0b0610 con 0x7fd6a8005190 2024-03-23T14:58:03.835 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.825+0000 7fd6baffd640 1 -- 172.21.15.164:0/663760443 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd6bc0b1500 con 0x7fd6a8003980 2024-03-23T14:58:03.835 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.825+0000 7fd6baffd640 1 -- 172.21.15.164:0/663760443 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fd6a800bf20 con 0x7fd6a80047d0 2024-03-23T14:58:03.835 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.825+0000 7fd6c09e3640 1 --2- 172.21.15.164:0/663760443 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd6bc14db70 0x7fd6bc14ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T14:58:03.835 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.825+0000 7fd6bb7fe640 1 --2- 172.21.15.164:0/663760443 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd6a80047d0 0x7fd6a8004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:03.835 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.825+0000 7fd6c09e3640 1 --2- 172.21.15.164:0/663760443 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd6a8005190 0x7fd6a800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:03.836 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.825+0000 7fd6bbfff640 1 --2- 172.21.15.164:0/663760443 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd6a8003980 0x7fd6a8003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:03.836 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.825+0000 7fd6c09e3640 1 -- 172.21.15.164:0/663760443 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd6a80047d0 msgr2=0x7fd6a8004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:58:03.836 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.825+0000 7fd6c09e3640 1 --2- 172.21.15.164:0/663760443 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd6a80047d0 0x7fd6a8004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:03.836 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.825+0000 7fd6c09e3640 1 -- 172.21.15.164:0/663760443 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd6a8003980 msgr2=0x7fd6a8003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:58:03.836 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.825+0000 7fd6c09e3640 1 --2- 172.21.15.164:0/663760443 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd6a8003980 0x7fd6a8003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:03.836 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.829+0000 7fd6c09e3640 1 -- 172.21.15.164:0/663760443 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6a8003530 con 0x7fd6a8005190 2024-03-23T14:58:03.836 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.829+0000 7fd6bbfff640 1 --2- 172.21.15.164:0/663760443 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd6a8003980 0x7fd6a8003da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T14:58:03.840 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.829+0000 7fd6c09e3640 1 --2- 172.21.15.164:0/663760443 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd6a8005190 0x7fd6a800b990 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fd6b4009b20 tx=0x7fd6b401cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbf6ebf4d021b3e1 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:58:03.840 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.829+0000 7fd6baffd640 1 -- 172.21.15.164:0/663760443 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6b4004030 con 0x7fd6a8005190 2024-03-23T14:58:03.840 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.829+0000 7fd6baffd640 1 -- 172.21.15.164:0/663760443 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6b4004190 con 0x7fd6a8005190 2024-03-23T14:58:03.840 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.829+0000 7fd6baffd640 1 -- 172.21.15.164:0/663760443 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6b4004520 con 0x7fd6a8005190 2024-03-23T14:58:03.840 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.829+0000 7fd6c2446640 1 -- 172.21.15.164:0/663760443 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd6a8005190 msgr2=0x7fd6a800b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:03.840 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.829+0000 7fd6c2446640 1 --2- 172.21.15.164:0/663760443 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd6a8005190 0x7fd6a800b990 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fd6b4009b20 tx=0x7fd6b401cab0 comp rx=0 tx=0).stop 2024-03-23T14:58:03.840 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.829+0000 7fd6c09e3640 1 -- 172.21.15.164:0/663760443 reap_dead start 2024-03-23T14:58:03.840 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.829+0000 7fd6c2446640 1 -- 172.21.15.164:0/663760443 shutdown_connections 2024-03-23T14:58:03.841 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.829+0000 7fd6c2446640 1 -- 172.21.15.164:0/663760443 >> 172.21.15.164:0/663760443 conn(0x7fd6bc0b8ca0 msgr2=0x7fd6bc0b90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:58:03.841 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.829+0000 7fd6c2446640 1 -- 172.21.15.164:0/663760443 shutdown_connections 2024-03-23T14:58:03.841 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.829+0000 7fd6c2446640 1 -- 172.21.15.164:0/663760443 wait complete. 2024-03-23T14:58:03.841 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.833+0000 7fd6c2446640 1 Processor -- start 2024-03-23T14:58:03.841 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.833+0000 7fd6c2446640 1 -- start start 2024-03-23T14:58:03.841 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.833+0000 7fd6c2446640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd6bc1447c0 0x7fd6bc13e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:03.841 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.833+0000 7fd6c2446640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd6bc13d190 0x7fd6bc13d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:03.841 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.833+0000 7fd6c2446640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd6bc0b68d0 0x7fd6bc0b6d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:03.842 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.833+0000 7fd6c2446640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd6bc13ccf0 con 0x7fd6bc1447c0 2024-03-23T14:58:03.842 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.833+0000 7fd6c2446640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd6bc13e160 con 0x7fd6bc13d190 2024-03-23T14:58:03.842 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.833+0000 7fd6c2446640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fd6bc13ca00 con 0x7fd6bc0b68d0 2024-03-23T14:58:03.842 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.833+0000 7fd6bbfff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd6bc1447c0 0x7fd6bc13e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:03.842 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.833+0000 7fd6bbfff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd6bc1447c0 0x7fd6bc13e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:48890/0 (socket says 172.21.15.164:48890) 2024-03-23T14:58:03.842 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.833+0000 7fd6bbfff640 1 -- 172.21.15.164:0/127245185 learned_addr learned my addr 172.21.15.164:0/127245185 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:58:03.842 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.833+0000 7fd6bb7fe640 1 --2- 172.21.15.164:0/127245185 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd6bc13d190 0x7fd6bc13d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:03.842 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.833+0000 7fd6c09e3640 1 --2- 172.21.15.164:0/127245185 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd6bc0b68d0 0x7fd6bc0b6d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:03.843 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.833+0000 7fd6bbfff640 1 -- 172.21.15.164:0/127245185 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd6bc0b68d0 msgr2=0x7fd6bc0b6d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:03.843 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.833+0000 7fd6bbfff640 1 --2- 172.21.15.164:0/127245185 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd6bc0b68d0 0x7fd6bc0b6d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:03.843 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.833+0000 7fd6bbfff640 1 -- 172.21.15.164:0/127245185 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd6bc13d190 msgr2=0x7fd6bc13d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:03.843 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.833+0000 7fd6bbfff640 1 --2- 172.21.15.164:0/127245185 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd6bc13d190 0x7fd6bc13d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:03.843 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.833+0000 7fd6bbfff640 1 -- 172.21.15.164:0/127245185 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6bc13db50 con 0x7fd6bc1447c0 2024-03-23T14:58:03.843 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.833+0000 7fd6bbfff640 1 --2- 172.21.15.164:0/127245185 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd6bc1447c0 0x7fd6bc13e3f0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fd6a40108c0 tx=0x7fd6a4010cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:58:03.843 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.833+0000 7fd6b8ff9640 1 -- 172.21.15.164:0/127245185 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6a4004220 con 0x7fd6bc1447c0 2024-03-23T14:58:03.843 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.833+0000 7fd6b8ff9640 1 -- 172.21.15.164:0/127245185 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6a4004380 con 0x7fd6bc1447c0 2024-03-23T14:58:03.844 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.833+0000 7fd6c2446640 1 -- 172.21.15.164:0/127245185 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6bc13eb50 con 0x7fd6bc1447c0 2024-03-23T14:58:03.844 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.833+0000 7fd6b8ff9640 1 -- 172.21.15.164:0/127245185 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6a400eb60 con 0x7fd6bc1447c0 2024-03-23T14:58:03.844 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.833+0000 7fd6c2446640 1 -- 172.21.15.164:0/127245185 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6bc0b7970 con 0x7fd6bc1447c0 2024-03-23T14:58:03.844 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.833+0000 7fd6c2446640 1 -- 172.21.15.164:0/127245185 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6bc02d370 con 0x7fd6bc1447c0 2024-03-23T14:58:03.844 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.833+0000 7fd6b8ff9640 1 -- 172.21.15.164:0/127245185 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7fd6a400ecc0 con 0x7fd6bc1447c0 2024-03-23T14:58:03.844 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.833+0000 7fd6b8ff9640 1 --2- 172.21.15.164:0/127245185 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fd69c0672d0 0x7fd69c069790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:03.844 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.833+0000 7fd6b8ff9640 1 -- 172.21.15.164:0/127245185 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4943+0+0 (secure 0 0 0) 0x7fd6a4084ac0 con 0x7fd6bc1447c0 2024-03-23T14:58:03.844 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.833+0000 7fd6bb7fe640 1 --2- 172.21.15.164:0/127245185 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fd69c0672d0 0x7fd69c069790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:03.844 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.833+0000 7fd6bb7fe640 1 --2- 172.21.15.164:0/127245185 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fd69c0672d0 0x7fd69c069790 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd6ac0013e0 tx=0x7fd6ac009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:58:03.845 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.837+0000 7fd6b8ff9640 1 -- 172.21.15.164:0/127245185 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7fd6a4051a00 con 0x7fd6bc1447c0 2024-03-23T14:58:03.989 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.985+0000 7fd6c2446640 1 -- 172.21.15.164:0/127245185 --> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7fd6bc0b3a10 con 0x7fd69c0672d0 2024-03-23T14:58:03.990 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.985+0000 7fd6b8ff9640 1 -- 172.21.15.164:0/127245185 <== mgr.4107 v2:172.21.15.164:6832/2139087985 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fd6ac005440 con 0x7fd69c0672d0 2024-03-23T14:58:03.990 INFO:teuthology.orchestra.run.smithi164.stderr:instructed osd(s) 1 to deep-scrub 2024-03-23T14:58:03.992 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.989+0000 7fd6c2446640 1 -- 172.21.15.164:0/127245185 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fd69c0672d0 msgr2=0x7fd69c069790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:03.992 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.989+0000 7fd6c2446640 1 --2- 172.21.15.164:0/127245185 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fd69c0672d0 0x7fd69c069790 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd6ac0013e0 tx=0x7fd6ac009040 comp rx=0 tx=0).stop 2024-03-23T14:58:03.993 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.989+0000 7fd6c2446640 1 -- 172.21.15.164:0/127245185 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd6bc1447c0 msgr2=0x7fd6bc13e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:03.993 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.989+0000 7fd6c2446640 1 --2- 172.21.15.164:0/127245185 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd6bc1447c0 0x7fd6bc13e3f0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fd6a40108c0 tx=0x7fd6a4010cc0 comp rx=0 tx=0).stop 2024-03-23T14:58:03.993 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.989+0000 7fd6c2446640 1 -- 172.21.15.164:0/127245185 shutdown_connections 2024-03-23T14:58:03.993 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.989+0000 7fd6c2446640 1 --2- 172.21.15.164:0/127245185 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd6bc0b68d0 0x7fd6bc0b6d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:03.993 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.989+0000 7fd6c2446640 1 --2- 172.21.15.164:0/127245185 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fd69c0672d0 0x7fd69c069790 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:03.993 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.989+0000 7fd6c2446640 1 --2- 172.21.15.164:0/127245185 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd6bc13d190 0x7fd6bc13d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:03.993 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.989+0000 7fd6c2446640 1 --2- 172.21.15.164:0/127245185 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd6bc1447c0 0x7fd6bc13e3f0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:03.993 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.989+0000 7fd6c2446640 1 -- 172.21.15.164:0/127245185 >> 172.21.15.164:0/127245185 conn(0x7fd6bc0b8ca0 msgr2=0x7fd6bc145bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:58:03.993 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.989+0000 7fd6c2446640 1 -- 172.21.15.164:0/127245185 shutdown_connections 2024-03-23T14:58:03.994 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:03.989+0000 7fd6c2446640 1 -- 172.21.15.164:0/127245185 wait complete. 2024-03-23T14:58:04.008 INFO:tasks.ceph:Scrubbing osd.2 2024-03-23T14:58:04.008 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-03-23T14:58:04.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.133+0000 7f2dd03d8640 1 Processor -- start 2024-03-23T14:58:04.137 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.133+0000 7f2dd03d8640 1 -- start start 2024-03-23T14:58:04.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.133+0000 7f2dd03d8640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2dc806bf70 0x7f2dc80ae070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:04.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.133+0000 7f2dd03d8640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2dc806c9c0 con 0x7f2dc814ad20 2024-03-23T14:58:04.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.133+0000 7f2dd03d8640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2dc80af460 con 0x7f2dc80adc90 2024-03-23T14:58:04.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.133+0000 7f2dd03d8640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f2dc80ae810 con 0x7f2dc806bf70 2024-03-23T14:58:04.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.133+0000 7f2dce975640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f2dc814ad20 0x7f2dc814b170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:37814/0 (socket says 172.21.15.164:37814) 2024-03-23T14:58:04.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.133+0000 7f2dce975640 1 -- 172.21.15.164:0/2062454433 learned_addr learned my addr 172.21.15.164:0/2062454433 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-03-23T14:58:04.140 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.133+0000 7f2dcd973640 1 --2- 172.21.15.164:0/2062454433 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2dc806bf70 0x7f2dc80ae070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:04.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.133+0000 7f2dcd172640 1 -- 172.21.15.164:0/2062454433 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 749285022 0 0) 0x7f2dc806c9c0 con 0x7f2dc814ad20 2024-03-23T14:58:04.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.133+0000 7f2dcd172640 1 -- 172.21.15.164:0/2062454433 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2db4003530 con 0x7f2dc814ad20 2024-03-23T14:58:04.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.133+0000 7f2dcd172640 1 -- 172.21.15.164:0/2062454433 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3592829452 0 0) 0x7f2db4003530 con 0x7f2dc814ad20 2024-03-23T14:58:04.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.133+0000 7f2dcd172640 1 -- 172.21.15.164:0/2062454433 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2dc806bf70 msgr2=0x7f2dc80ae070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:58:04.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.133+0000 7f2dcd172640 1 --2- 172.21.15.164:0/2062454433 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2dc806bf70 0x7f2dc80ae070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:04.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.133+0000 7f2dcd172640 1 -- 172.21.15.164:0/2062454433 >> v1:172.21.15.173:6789/0 conn(0x7f2dc80adc90 legacy=0x7f2dc80b4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:04.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.133+0000 7f2dcd172640 1 -- 172.21.15.164:0/2062454433 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2dc80b5170 con 0x7f2dc814ad20 2024-03-23T14:58:04.141 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.133+0000 7f2dcd172640 1 -- 172.21.15.164:0/2062454433 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2526390548 0 0) 0x7f2dc0001a00 con 0x7f2dc814ad20 2024-03-23T14:58:04.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.133+0000 7f2dcd172640 1 -- 172.21.15.164:0/2062454433 >> v1:172.21.15.164:6789/0 conn(0x7f2dc814ad20 legacy=0x7f2dc814b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:04.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.133+0000 7f2dcd172640 1 --2- 172.21.15.164:0/2062454433 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2db40039b0 0x7f2db4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:04.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.133+0000 7f2dcd172640 1 --2- 172.21.15.164:0/2062454433 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2db40047d0 0x7f2db4004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:04.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.133+0000 7f2dcd172640 1 --2- 172.21.15.164:0/2062454433 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2db4005200 0x7f2db400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:04.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.133+0000 7f2dcd172640 1 -- 172.21.15.164:0/2062454433 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2dc80ae810 con 0x7f2db40047d0 2024-03-23T14:58:04.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.133+0000 7f2dcd172640 1 -- 172.21.15.164:0/2062454433 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f2db400bf80 con 0x7f2db40039b0 2024-03-23T14:58:04.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.133+0000 7f2dcd172640 1 -- 172.21.15.164:0/2062454433 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f2db400c140 con 0x7f2db4005200 2024-03-23T14:58:04.142 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.133+0000 7f2dcd973640 1 --2- 172.21.15.164:0/2062454433 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2db40047d0 0x7f2db4004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:04.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.133+0000 7f2dce174640 1 --2- 172.21.15.164:0/2062454433 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2db40039b0 0x7f2db4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:04.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.133+0000 7f2dce975640 1 --2- 172.21.15.164:0/2062454433 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2db4005200 0x7f2db400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:04.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.133+0000 7f2dcd973640 1 -- 172.21.15.164:0/2062454433 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2db4005200 msgr2=0x7f2db400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:58:04.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.133+0000 7f2dcd973640 1 --2- 172.21.15.164:0/2062454433 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2db4005200 0x7f2db400b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:04.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.133+0000 7f2dcd973640 1 -- 172.21.15.164:0/2062454433 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2db40039b0 msgr2=0x7f2db4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:58:04.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.133+0000 7f2dcd973640 1 --2- 172.21.15.164:0/2062454433 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2db40039b0 0x7f2db4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:04.143 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.133+0000 7f2dcd973640 1 -- 172.21.15.164:0/2062454433 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2db4003530 con 0x7f2db40047d0 2024-03-23T14:58:04.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.133+0000 7f2dce174640 1 --2- 172.21.15.164:0/2062454433 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2db40039b0 0x7f2db4003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T14:58:04.148 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.137+0000 7f2dcd973640 1 --2- 172.21.15.164:0/2062454433 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2db40047d0 0x7f2db4004c50 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f2db8009ab0 tx=0x7f2db801ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dd3b2bc33a8e2b5 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:58:04.148 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.137+0000 7f2dcd172640 1 -- 172.21.15.164:0/2062454433 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2db80a5510 con 0x7f2db40047d0 2024-03-23T14:58:04.148 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.137+0000 7f2dcd172640 1 -- 172.21.15.164:0/2062454433 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2db80a5670 con 0x7f2db40047d0 2024-03-23T14:58:04.148 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.137+0000 7f2dcd172640 1 -- 172.21.15.164:0/2062454433 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2db80a59e0 con 0x7f2db40047d0 2024-03-23T14:58:04.148 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.137+0000 7f2dd03d8640 1 -- 172.21.15.164:0/2062454433 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2db40047d0 msgr2=0x7f2db4004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:04.148 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.137+0000 7f2dd03d8640 1 --2- 172.21.15.164:0/2062454433 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2db40047d0 0x7f2db4004c50 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f2db8009ab0 tx=0x7f2db801ca50 comp rx=0 tx=0).stop 2024-03-23T14:58:04.148 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.137+0000 7f2dce975640 1 -- 172.21.15.164:0/2062454433 reap_dead start 2024-03-23T14:58:04.148 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.137+0000 7f2dd03d8640 1 -- 172.21.15.164:0/2062454433 shutdown_connections 2024-03-23T14:58:04.149 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.137+0000 7f2dd03d8640 1 -- 172.21.15.164:0/2062454433 >> 172.21.15.164:0/2062454433 conn(0x7f2dc80b8ca0 msgr2=0x7f2dc80b90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:58:04.149 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.137+0000 7f2dd03d8640 1 -- 172.21.15.164:0/2062454433 shutdown_connections 2024-03-23T14:58:04.149 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.137+0000 7f2dd03d8640 1 -- 172.21.15.164:0/2062454433 wait complete. 2024-03-23T14:58:04.149 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.137+0000 7f2dd03d8640 1 Processor -- start 2024-03-23T14:58:04.149 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.137+0000 7f2dd03d8640 1 -- start start 2024-03-23T14:58:04.149 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.141+0000 7f2dd03d8640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2db40047d0 0x7f2dc809d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:04.149 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.141+0000 7f2dd03d8640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2dc809c1f0 0x7f2dc81e2e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:04.149 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.141+0000 7f2dd03d8640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2dc81e3390 0x7f2dc8147030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:04.149 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.141+0000 7f2dd03d8640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2dc809b710 con 0x7f2dc81e3390 2024-03-23T14:58:04.150 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.141+0000 7f2dd03d8640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f2dc809b9f0 con 0x7f2dc809c1f0 2024-03-23T14:58:04.150 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.141+0000 7f2dd03d8640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f2dc809d670 con 0x7f2db40047d0 2024-03-23T14:58:04.150 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.141+0000 7f2dce975640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2dc81e3390 0x7f2dc8147030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:04.150 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.141+0000 7f2dce975640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2dc81e3390 0x7f2dc8147030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:48910/0 (socket says 172.21.15.164:48910) 2024-03-23T14:58:04.150 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.141+0000 7f2dce975640 1 -- 172.21.15.164:0/102823721 learned_addr learned my addr 172.21.15.164:0/102823721 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:58:04.150 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.141+0000 7f2dcd973640 1 --2- 172.21.15.164:0/102823721 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2dc809c1f0 0x7f2dc81e2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:04.150 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.141+0000 7f2dce174640 1 --2- 172.21.15.164:0/102823721 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2db40047d0 0x7f2dc809d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:04.150 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.141+0000 7f2dce975640 1 -- 172.21.15.164:0/102823721 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2db40047d0 msgr2=0x7f2dc809d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:04.150 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.141+0000 7f2dce975640 1 --2- 172.21.15.164:0/102823721 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2db40047d0 0x7f2dc809d130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:04.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.141+0000 7f2dce975640 1 -- 172.21.15.164:0/102823721 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2dc809c1f0 msgr2=0x7f2dc81e2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:04.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.141+0000 7f2dce975640 1 --2- 172.21.15.164:0/102823721 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2dc809c1f0 0x7f2dc81e2e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:04.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.141+0000 7f2dce975640 1 -- 172.21.15.164:0/102823721 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2dc81e3820 con 0x7f2dc81e3390 2024-03-23T14:58:04.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.141+0000 7f2dce975640 1 --2- 172.21.15.164:0/102823721 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2dc81e3390 0x7f2dc8147030 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f2dc8072f80 tx=0x7f2dc000b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:58:04.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.141+0000 7f2dbeffd640 1 -- 172.21.15.164:0/102823721 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dc0004d70 con 0x7f2dc81e3390 2024-03-23T14:58:04.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.141+0000 7f2dbeffd640 1 -- 172.21.15.164:0/102823721 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2dc000bb80 con 0x7f2dc81e3390 2024-03-23T14:58:04.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.141+0000 7f2dd03d8640 1 -- 172.21.15.164:0/102823721 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2dc809d8d0 con 0x7f2dc81e3390 2024-03-23T14:58:04.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.141+0000 7f2dbeffd640 1 -- 172.21.15.164:0/102823721 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dc0007110 con 0x7f2dc81e3390 2024-03-23T14:58:04.151 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.141+0000 7f2dd03d8640 1 -- 172.21.15.164:0/102823721 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2dc809c9b0 con 0x7f2dc81e3390 2024-03-23T14:58:04.152 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.141+0000 7f2dd03d8640 1 -- 172.21.15.164:0/102823721 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2dc8152de0 con 0x7f2dc81e3390 2024-03-23T14:58:04.152 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.141+0000 7f2dbeffd640 1 -- 172.21.15.164:0/102823721 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7f2dc0007270 con 0x7f2dc81e3390 2024-03-23T14:58:04.152 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.141+0000 7f2dbeffd640 1 --2- 172.21.15.164:0/102823721 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f2dac0672a0 0x7f2dac069760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:04.152 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.141+0000 7f2dbeffd640 1 -- 172.21.15.164:0/102823721 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4943+0+0 (secure 0 0 0) 0x7f2dc00857b0 con 0x7f2dc81e3390 2024-03-23T14:58:04.152 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.141+0000 7f2dce174640 1 --2- 172.21.15.164:0/102823721 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f2dac0672a0 0x7f2dac069760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:04.152 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.141+0000 7f2dbeffd640 1 --2- 172.21.15.164:0/102823721 >> [v2:172.21.15.164:6808/887861252,v1:172.21.15.164:6809/887861252] conn(0x7f2dac06d160 0x7f2dac06f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:04.152 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.141+0000 7f2dbeffd640 1 -- 172.21.15.164:0/102823721 --> [v2:172.21.15.164:6808/887861252,v1:172.21.15.164:6809/887861252] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2dac06fc30 con 0x7f2dac06d160 2024-03-23T14:58:04.152 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.141+0000 7f2dbeffd640 1 -- 172.21.15.164:0/102823721 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f2dc0085c00 con 0x7f2dc81e3390 2024-03-23T14:58:04.152 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.141+0000 7f2dcd973640 1 --2- 172.21.15.164:0/102823721 >> [v2:172.21.15.164:6808/887861252,v1:172.21.15.164:6809/887861252] conn(0x7f2dac06d160 0x7f2dac06f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:04.153 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.141+0000 7f2dce174640 1 --2- 172.21.15.164:0/102823721 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f2dac0672a0 0x7f2dac069760 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f2dc4001640 tx=0x7f2dc4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:58:04.153 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.141+0000 7f2dcd973640 1 --2- 172.21.15.164:0/102823721 >> [v2:172.21.15.164:6808/887861252,v1:172.21.15.164:6809/887861252] conn(0x7f2dac06d160 0x7f2dac06f580 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:58:04.153 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.145+0000 7f2dbeffd640 1 -- 172.21.15.164:0/102823721 <== osd.2 v2:172.21.15.164:6808/887861252 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2dac06fc30 con 0x7f2dac06d160 2024-03-23T14:58:04.160 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.157+0000 7f2dd03d8640 1 -- 172.21.15.164:0/102823721 --> [v2:172.21.15.164:6808/887861252,v1:172.21.15.164:6809/887861252] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f2dc81539a0 con 0x7f2dac06d160 2024-03-23T14:58:04.160 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.157+0000 7f2dbeffd640 1 -- 172.21.15.164:0/102823721 <== osd.2 v2:172.21.15.164:6808/887861252 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f2db80a57e0 con 0x7f2dac06d160 2024-03-23T14:58:04.161 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-03-23T14:58:04.161 INFO:teuthology.orchestra.run.smithi164.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-23T14:58:04.161 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-03-23T14:58:04.162 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.157+0000 7f2dd03d8640 1 -- 172.21.15.164:0/102823721 >> [v2:172.21.15.164:6808/887861252,v1:172.21.15.164:6809/887861252] conn(0x7f2dac06d160 msgr2=0x7f2dac06f580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:04.162 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.157+0000 7f2dd03d8640 1 --2- 172.21.15.164:0/102823721 >> [v2:172.21.15.164:6808/887861252,v1:172.21.15.164:6809/887861252] conn(0x7f2dac06d160 0x7f2dac06f580 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:04.162 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.157+0000 7f2dd03d8640 1 -- 172.21.15.164:0/102823721 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f2dac0672a0 msgr2=0x7f2dac069760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:04.162 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.157+0000 7f2dd03d8640 1 --2- 172.21.15.164:0/102823721 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f2dac0672a0 0x7f2dac069760 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f2dc4001640 tx=0x7f2dc4009040 comp rx=0 tx=0).stop 2024-03-23T14:58:04.163 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.157+0000 7f2dd03d8640 1 -- 172.21.15.164:0/102823721 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2dc81e3390 msgr2=0x7f2dc8147030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:04.163 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.157+0000 7f2dd03d8640 1 --2- 172.21.15.164:0/102823721 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2dc81e3390 0x7f2dc8147030 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f2dc8072f80 tx=0x7f2dc000b4a0 comp rx=0 tx=0).stop 2024-03-23T14:58:04.163 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.157+0000 7f2dce975640 1 -- 172.21.15.164:0/102823721 reap_dead start 2024-03-23T14:58:04.163 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.157+0000 7f2dd03d8640 1 -- 172.21.15.164:0/102823721 shutdown_connections 2024-03-23T14:58:04.163 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.157+0000 7f2dd03d8640 1 -- 172.21.15.164:0/102823721 >> 172.21.15.164:0/102823721 conn(0x7f2dc80b8ca0 msgr2=0x7f2dc80b68d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:58:04.163 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.157+0000 7f2dd03d8640 1 -- 172.21.15.164:0/102823721 shutdown_connections 2024-03-23T14:58:04.163 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.157+0000 7f2dd03d8640 1 -- 172.21.15.164:0/102823721 wait complete. 2024-03-23T14:58:04.177 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-03-23T14:58:04.292 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.289+0000 7f0e5785e640 1 Processor -- start 2024-03-23T14:58:04.293 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.289+0000 7f0e5785e640 1 -- start start 2024-03-23T14:58:04.295 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.289+0000 7f0e5785e640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f0e5006e640 0x7f0e5006ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:04.295 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.289+0000 7f0e5785e640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e5002c160 con 0x7f0e5002cc40 2024-03-23T14:58:04.296 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.289+0000 7f0e5785e640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e5002c440 con 0x7f0e50070140 2024-03-23T14:58:04.296 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.289+0000 7f0e5785e640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 -- 0x7f0e5002c720 con 0x7f0e5006e640 2024-03-23T14:58:04.296 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.289+0000 7f0e54df9640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f0e5002cc40 0x7f0e5014ecc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:37826/0 (socket says 172.21.15.164:37826) 2024-03-23T14:58:04.296 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.289+0000 7f0e54df9640 1 -- 172.21.15.164:0/947411935 learned_addr learned my addr 172.21.15.164:0/947411935 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-03-23T14:58:04.296 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.289+0000 7f0e555fa640 1 --2- 172.21.15.164:0/947411935 >> v2:172.21.15.173:3300/0 conn(0x7f0e5006e640 0x7f0e5006ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:04.296 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.289+0000 7f0e4ffff640 1 -- 172.21.15.164:0/947411935 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 872950260 0 0) 0x7f0e5002c160 con 0x7f0e5002cc40 2024-03-23T14:58:04.296 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.289+0000 7f0e4ffff640 1 -- 172.21.15.164:0/947411935 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0e38003530 con 0x7f0e5002cc40 2024-03-23T14:58:04.296 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.289+0000 7f0e4ffff640 1 -- 172.21.15.164:0/947411935 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1260290971 0 0) 0x7f0e38003530 con 0x7f0e5002cc40 2024-03-23T14:58:04.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.289+0000 7f0e4ffff640 1 -- 172.21.15.164:0/947411935 >> v2:172.21.15.173:3300/0 conn(0x7f0e5006e640 msgr2=0x7f0e5006ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:58:04.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.289+0000 7f0e4ffff640 1 --2- 172.21.15.164:0/947411935 >> v2:172.21.15.173:3300/0 conn(0x7f0e5006e640 0x7f0e5006ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:04.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.289+0000 7f0e4ffff640 1 -- 172.21.15.164:0/947411935 >> v1:172.21.15.173:6789/0 conn(0x7f0e50070140 legacy=0x7f0e50148970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:04.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.289+0000 7f0e4ffff640 1 -- 172.21.15.164:0/947411935 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e50149080 con 0x7f0e5002cc40 2024-03-23T14:58:04.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.289+0000 7f0e4ffff640 1 -- 172.21.15.164:0/947411935 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2526390548 0 0) 0x7f0e44001680 con 0x7f0e5002cc40 2024-03-23T14:58:04.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.289+0000 7f0e4ffff640 1 -- 172.21.15.164:0/947411935 >> v1:172.21.15.164:6789/0 conn(0x7f0e5002cc40 legacy=0x7f0e5014ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:04.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.289+0000 7f0e4ffff640 1 --2- 172.21.15.164:0/947411935 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0e38003980 0x7f0e38003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:04.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.289+0000 7f0e4ffff640 1 --2- 172.21.15.164:0/947411935 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0e380047d0 0x7f0e38004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:04.297 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.289+0000 7f0e4ffff640 1 --2- 172.21.15.164:0/947411935 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0e38005190 0x7f0e3800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:04.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.289+0000 7f0e4ffff640 1 -- 172.21.15.164:0/947411935 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0e5002c720 con 0x7f0e380047d0 2024-03-23T14:58:04.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.289+0000 7f0e4ffff640 1 -- 172.21.15.164:0/947411935 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f0e3800c030 con 0x7f0e38005190 2024-03-23T14:58:04.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.289+0000 7f0e4ffff640 1 -- 172.21.15.164:0/947411935 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f0e3800c1f0 con 0x7f0e38003980 2024-03-23T14:58:04.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.289+0000 7f0e54df9640 1 --2- 172.21.15.164:0/947411935 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0e380047d0 0x7f0e38004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:04.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.289+0000 7f0e555fa640 1 --2- 172.21.15.164:0/947411935 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0e38003980 0x7f0e38003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:04.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.289+0000 7f0e55dfb640 1 --2- 172.21.15.164:0/947411935 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0e38005190 0x7f0e3800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:04.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.289+0000 7f0e54df9640 1 -- 172.21.15.164:0/947411935 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0e38003980 msgr2=0x7f0e38003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:58:04.298 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.289+0000 7f0e54df9640 1 --2- 172.21.15.164:0/947411935 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0e38003980 0x7f0e38003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:04.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.289+0000 7f0e54df9640 1 -- 172.21.15.164:0/947411935 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0e38005190 msgr2=0x7f0e3800b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:58:04.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.289+0000 7f0e54df9640 1 --2- 172.21.15.164:0/947411935 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0e38005190 0x7f0e3800b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:04.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.289+0000 7f0e54df9640 1 -- 172.21.15.164:0/947411935 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e38003530 con 0x7f0e380047d0 2024-03-23T14:58:04.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.289+0000 7f0e555fa640 1 --2- 172.21.15.164:0/947411935 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0e38003980 0x7f0e38003da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T14:58:04.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.289+0000 7f0e55dfb640 1 --2- 172.21.15.164:0/947411935 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0e38005190 0x7f0e3800b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T14:58:04.299 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.289+0000 7f0e55dfb640 1 -- 172.21.15.164:0/947411935 reap_dead start 2024-03-23T14:58:04.302 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.293+0000 7f0e54df9640 1 --2- 172.21.15.164:0/947411935 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0e380047d0 0x7f0e38004c50 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f0e4400c240 tx=0x7f0e4401f3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1aa67ae4e503122f server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:58:04.302 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.293+0000 7f0e4ffff640 1 -- 172.21.15.164:0/947411935 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e44005ae0 con 0x7f0e380047d0 2024-03-23T14:58:04.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.293+0000 7f0e4ffff640 1 -- 172.21.15.164:0/947411935 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e440ab070 con 0x7f0e380047d0 2024-03-23T14:58:04.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.293+0000 7f0e4ffff640 1 -- 172.21.15.164:0/947411935 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e44005ea0 con 0x7f0e380047d0 2024-03-23T14:58:04.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.293+0000 7f0e5785e640 1 -- 172.21.15.164:0/947411935 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0e380047d0 msgr2=0x7f0e38004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:04.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.293+0000 7f0e5785e640 1 --2- 172.21.15.164:0/947411935 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0e380047d0 0x7f0e38004c50 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f0e4400c240 tx=0x7f0e4401f3d0 comp rx=0 tx=0).stop 2024-03-23T14:58:04.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.293+0000 7f0e5785e640 1 -- 172.21.15.164:0/947411935 shutdown_connections 2024-03-23T14:58:04.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.293+0000 7f0e5785e640 1 --2- 172.21.15.164:0/947411935 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0e380047d0 0x7f0e38004c50 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:04.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.293+0000 7f0e5785e640 1 -- 172.21.15.164:0/947411935 >> 172.21.15.164:0/947411935 conn(0x7f0e500b7970 msgr2=0x7f0e500b7d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:58:04.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.293+0000 7f0e5785e640 1 -- 172.21.15.164:0/947411935 shutdown_connections 2024-03-23T14:58:04.303 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.293+0000 7f0e5785e640 1 -- 172.21.15.164:0/947411935 wait complete. 2024-03-23T14:58:04.304 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.293+0000 7f0e5785e640 1 Processor -- start 2024-03-23T14:58:04.304 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.293+0000 7f0e5785e640 1 -- start start 2024-03-23T14:58:04.304 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.293+0000 7f0e5785e640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0e380047d0 0x7f0e500a9b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:04.304 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.293+0000 7f0e5785e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0e5014c080 0x7f0e501e0970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:04.304 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.293+0000 7f0e5785e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0e501e0eb0 0x7f0e50145460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:04.304 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.293+0000 7f0e5785e640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0e500b35d0 con 0x7f0e501e0eb0 2024-03-23T14:58:04.304 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.293+0000 7f0e5785e640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f0e5014c500 con 0x7f0e5014c080 2024-03-23T14:58:04.304 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.293+0000 7f0e5785e640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f0e500b2cc0 con 0x7f0e380047d0 2024-03-23T14:58:04.305 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.293+0000 7f0e55dfb640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0e501e0eb0 0x7f0e50145460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:04.305 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.293+0000 7f0e55dfb640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0e501e0eb0 0x7f0e50145460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:48928/0 (socket says 172.21.15.164:48928) 2024-03-23T14:58:04.305 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.293+0000 7f0e55dfb640 1 -- 172.21.15.164:0/1902421913 learned_addr learned my addr 172.21.15.164:0/1902421913 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:58:04.305 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.293+0000 7f0e555fa640 1 --2- 172.21.15.164:0/1902421913 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0e380047d0 0x7f0e500a9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:04.305 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.297+0000 7f0e55dfb640 1 -- 172.21.15.164:0/1902421913 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0e380047d0 msgr2=0x7f0e500a9b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:04.305 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.297+0000 7f0e55dfb640 1 --2- 172.21.15.164:0/1902421913 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0e380047d0 0x7f0e500a9b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:04.305 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.297+0000 7f0e55dfb640 1 -- 172.21.15.164:0/1902421913 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0e5014c080 msgr2=0x7f0e501e0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:04.305 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.297+0000 7f0e55dfb640 1 --2- 172.21.15.164:0/1902421913 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0e5014c080 0x7f0e501e0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:04.305 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.297+0000 7f0e55dfb640 1 -- 172.21.15.164:0/1902421913 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e500b2f10 con 0x7f0e501e0eb0 2024-03-23T14:58:04.306 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.297+0000 7f0e555fa640 1 --2- 172.21.15.164:0/1902421913 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0e380047d0 0x7f0e500a9b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T14:58:04.306 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.297+0000 7f0e55dfb640 1 --2- 172.21.15.164:0/1902421913 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0e501e0eb0 0x7f0e50145460 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f0e5014c880 tx=0x7f0e4000d660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:58:04.306 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.297+0000 7f0e4dffb640 1 -- 172.21.15.164:0/1902421913 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e4000b070 con 0x7f0e501e0eb0 2024-03-23T14:58:04.306 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.297+0000 7f0e4dffb640 1 -- 172.21.15.164:0/1902421913 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e4000de90 con 0x7f0e501e0eb0 2024-03-23T14:58:04.306 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.297+0000 7f0e5785e640 1 -- 172.21.15.164:0/1902421913 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e500b31f0 con 0x7f0e501e0eb0 2024-03-23T14:58:04.306 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.297+0000 7f0e4dffb640 1 -- 172.21.15.164:0/1902421913 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e40005500 con 0x7f0e501e0eb0 2024-03-23T14:58:04.306 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.297+0000 7f0e5785e640 1 -- 172.21.15.164:0/1902421913 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e500b2120 con 0x7f0e501e0eb0 2024-03-23T14:58:04.306 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.297+0000 7f0e5785e640 1 -- 172.21.15.164:0/1902421913 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e5002d260 con 0x7f0e501e0eb0 2024-03-23T14:58:04.306 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.297+0000 7f0e4dffb640 1 -- 172.21.15.164:0/1902421913 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7f0e40003840 con 0x7f0e501e0eb0 2024-03-23T14:58:04.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.297+0000 7f0e4dffb640 1 --2- 172.21.15.164:0/1902421913 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f0e3c0672a0 0x7f0e3c069760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:04.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.297+0000 7f0e555fa640 1 --2- 172.21.15.164:0/1902421913 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f0e3c0672a0 0x7f0e3c069760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:04.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.297+0000 7f0e4dffb640 1 -- 172.21.15.164:0/1902421913 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4943+0+0 (secure 0 0 0) 0x7f0e40084260 con 0x7f0e501e0eb0 2024-03-23T14:58:04.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.297+0000 7f0e555fa640 1 --2- 172.21.15.164:0/1902421913 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f0e3c0672a0 0x7f0e3c069760 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f0e500b4fd0 tx=0x7f0e4800b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:58:04.307 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.301+0000 7f0e4dffb640 1 -- 172.21.15.164:0/1902421913 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7f0e40051100 con 0x7f0e501e0eb0 2024-03-23T14:58:04.465 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.461+0000 7f0e5785e640 1 -- 172.21.15.164:0/1902421913 --> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7f0e500b5f40 con 0x7f0e3c0672a0 2024-03-23T14:58:04.466 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.461+0000 7f0e4dffb640 1 -- 172.21.15.164:0/1902421913 <== mgr.4107 v2:172.21.15.164:6832/2139087985 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f0e500b5f40 con 0x7f0e3c0672a0 2024-03-23T14:58:04.466 INFO:teuthology.orchestra.run.smithi164.stderr:instructed osd(s) 2 to deep-scrub 2024-03-23T14:58:04.468 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.465+0000 7f0e5785e640 1 -- 172.21.15.164:0/1902421913 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f0e3c0672a0 msgr2=0x7f0e3c069760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:04.469 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.465+0000 7f0e5785e640 1 --2- 172.21.15.164:0/1902421913 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f0e3c0672a0 0x7f0e3c069760 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f0e500b4fd0 tx=0x7f0e4800b040 comp rx=0 tx=0).stop 2024-03-23T14:58:04.469 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.465+0000 7f0e5785e640 1 -- 172.21.15.164:0/1902421913 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0e501e0eb0 msgr2=0x7f0e50145460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:04.469 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.465+0000 7f0e5785e640 1 --2- 172.21.15.164:0/1902421913 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0e501e0eb0 0x7f0e50145460 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f0e5014c880 tx=0x7f0e4000d660 comp rx=0 tx=0).stop 2024-03-23T14:58:04.469 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.465+0000 7f0e5785e640 1 -- 172.21.15.164:0/1902421913 shutdown_connections 2024-03-23T14:58:04.469 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.465+0000 7f0e5785e640 1 --2- 172.21.15.164:0/1902421913 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0e501e0eb0 0x7f0e50145460 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:04.469 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.465+0000 7f0e5785e640 1 --2- 172.21.15.164:0/1902421913 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f0e3c0672a0 0x7f0e3c069760 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:04.469 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.465+0000 7f0e5785e640 1 --2- 172.21.15.164:0/1902421913 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0e5014c080 0x7f0e501e0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:04.470 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.465+0000 7f0e5785e640 1 --2- 172.21.15.164:0/1902421913 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0e380047d0 0x7f0e500a9b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:04.470 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.465+0000 7f0e5785e640 1 -- 172.21.15.164:0/1902421913 >> 172.21.15.164:0/1902421913 conn(0x7f0e500b7970 msgr2=0x7f0e500b7d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:58:04.470 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.465+0000 7f0e5785e640 1 -- 172.21.15.164:0/1902421913 shutdown_connections 2024-03-23T14:58:04.470 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.465+0000 7f0e5785e640 1 -- 172.21.15.164:0/1902421913 wait complete. 2024-03-23T14:58:04.485 INFO:tasks.ceph:Scrubbing osd.3 2024-03-23T14:58:04.485 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-03-23T14:58:04.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.609+0000 7f97a849b640 1 Processor -- start 2024-03-23T14:58:04.613 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.609+0000 7f97a849b640 1 -- start start 2024-03-23T14:58:04.614 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.609+0000 7f97a849b640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f97a00b9e60 0x7f97a0147400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:04.614 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.609+0000 7f97a849b640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f97a0141890 0x7f97a0151ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:04.614 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.609+0000 7f97a849b640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97a00b3e20 con 0x7f97a00b5ab0 2024-03-23T14:58:04.614 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.609+0000 7f97a849b640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f97a00b6070 con 0x7f97a0141890 2024-03-23T14:58:04.615 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.609+0000 7f97a849b640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f97a00b3b80 con 0x7f97a00b9e60 2024-03-23T14:58:04.615 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.609+0000 7f97a6a38640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f97a0141890 0x7f97a0151ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:04.615 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.609+0000 7f97a5a36640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f97a00b5ab0 0x7f97a0141180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:37836/0 (socket says 172.21.15.164:37836) 2024-03-23T14:58:04.615 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.609+0000 7f97a5a36640 1 -- 172.21.15.164:0/2675889926 learned_addr learned my addr 172.21.15.164:0/2675889926 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-03-23T14:58:04.615 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.609+0000 7f97a6a38640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f97a0141890 0x7f97a0151ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:48932/0 (socket says 172.21.15.164:48932) 2024-03-23T14:58:04.615 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.609+0000 7f97a6237640 1 --2- 172.21.15.164:0/2675889926 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f97a00b9e60 0x7f97a0147400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:04.615 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.609+0000 7f97a6a38640 1 -- 172.21.15.164:0/2675889926 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f97a00b9e60 msgr2=0x7f97a0147400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:58:04.615 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.609+0000 7f97a6a38640 1 --2- 172.21.15.164:0/2675889926 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f97a00b9e60 0x7f97a0147400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:04.615 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.609+0000 7f97a6a38640 1 -- 172.21.15.164:0/2675889926 >> v1:172.21.15.164:6789/0 conn(0x7f97a00b5ab0 legacy=0x7f97a0141180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:04.616 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.609+0000 7f97a6a38640 1 -- 172.21.15.164:0/2675889926 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97a00ba240 con 0x7f97a0141890 2024-03-23T14:58:04.616 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.609+0000 7f97a5235640 1 -- 172.21.15.164:0/2675889926 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2180593155 0 0) 0x7f97a00b3e20 con 0x7f97a00b5ab0 2024-03-23T14:58:04.616 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.613+0000 7f97a6a38640 1 --2- 172.21.15.164:0/2675889926 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f97a0141890 0x7f97a0151ce0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f9798009d90 tx=0x7f979801c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0651d0e1b1b41c3 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:58:04.616 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.613+0000 7f97a5235640 1 -- 172.21.15.164:0/2675889926 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97980a5900 con 0x7f97a0141890 2024-03-23T14:58:04.617 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.613+0000 7f97a5235640 1 -- 172.21.15.164:0/2675889926 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97980a5a60 con 0x7f97a0141890 2024-03-23T14:58:04.617 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.613+0000 7f97a5235640 1 -- 172.21.15.164:0/2675889926 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97980a5df0 con 0x7f97a0141890 2024-03-23T14:58:04.617 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.613+0000 7f97a849b640 1 -- 172.21.15.164:0/2675889926 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f97a0141890 msgr2=0x7f97a0151ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:04.618 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.613+0000 7f97a849b640 1 --2- 172.21.15.164:0/2675889926 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f97a0141890 0x7f97a0151ce0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f9798009d90 tx=0x7f979801c820 comp rx=0 tx=0).stop 2024-03-23T14:58:04.618 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.613+0000 7f97a849b640 1 -- 172.21.15.164:0/2675889926 shutdown_connections 2024-03-23T14:58:04.618 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.613+0000 7f97a849b640 1 --2- 172.21.15.164:0/2675889926 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f97a0141890 0x7f97a0151ce0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:04.618 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.613+0000 7f97a849b640 1 --2- 172.21.15.164:0/2675889926 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f97a00b9e60 0x7f97a0147400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:04.618 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.613+0000 7f97a849b640 1 -- 172.21.15.164:0/2675889926 >> 172.21.15.164:0/2675889926 conn(0x7f97a0072cd0 msgr2=0x7f97a00730e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:58:04.619 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.613+0000 7f97a849b640 1 -- 172.21.15.164:0/2675889926 shutdown_connections 2024-03-23T14:58:04.619 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.613+0000 7f97a849b640 1 -- 172.21.15.164:0/2675889926 wait complete. 2024-03-23T14:58:04.621 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.613+0000 7f97a849b640 1 Processor -- start 2024-03-23T14:58:04.621 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.613+0000 7f97a849b640 1 -- start start 2024-03-23T14:58:04.621 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.613+0000 7f97a849b640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f97a00b5ab0 0x7f97a014dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:04.621 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.613+0000 7f97a849b640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f97a00b9e60 0x7f97a014a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:04.621 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.613+0000 7f97a849b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f97a0141890 0x7f97a014aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:04.621 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.613+0000 7f97a849b640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f97a013ee10 con 0x7f97a0141890 2024-03-23T14:58:04.621 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.613+0000 7f97a849b640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f97a01e2f00 con 0x7f97a00b5ab0 2024-03-23T14:58:04.622 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.613+0000 7f97a849b640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f97a014f440 con 0x7f97a00b9e60 2024-03-23T14:58:04.622 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.613+0000 7f97a6a38640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f97a0141890 0x7f97a014aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:04.622 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.613+0000 7f97a6a38640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f97a0141890 0x7f97a014aea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:48934/0 (socket says 172.21.15.164:48934) 2024-03-23T14:58:04.622 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.613+0000 7f97a6a38640 1 -- 172.21.15.164:0/3064153484 learned_addr learned my addr 172.21.15.164:0/3064153484 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:58:04.622 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.613+0000 7f97a6237640 1 --2- 172.21.15.164:0/3064153484 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f97a00b5ab0 0x7f97a014dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:04.622 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.613+0000 7f97a5a36640 1 --2- 172.21.15.164:0/3064153484 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f97a00b9e60 0x7f97a014a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:04.622 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.617+0000 7f97a6a38640 1 -- 172.21.15.164:0/3064153484 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f97a00b9e60 msgr2=0x7f97a014a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:04.622 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.617+0000 7f97a6a38640 1 --2- 172.21.15.164:0/3064153484 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f97a00b9e60 0x7f97a014a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:04.623 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.617+0000 7f97a6a38640 1 -- 172.21.15.164:0/3064153484 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f97a00b5ab0 msgr2=0x7f97a014dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:04.623 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.617+0000 7f97a6a38640 1 --2- 172.21.15.164:0/3064153484 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f97a00b5ab0 0x7f97a014dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:04.623 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.617+0000 7f97a6a38640 1 -- 172.21.15.164:0/3064153484 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9798009a70 con 0x7f97a0141890 2024-03-23T14:58:04.623 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.617+0000 7f97a6a38640 1 --2- 172.21.15.164:0/3064153484 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f97a0141890 0x7f97a014aea0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f979801aed0 tx=0x7f97980195c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:58:04.623 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.617+0000 7f9796ffd640 1 -- 172.21.15.164:0/3064153484 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97980ad850 con 0x7f97a0141890 2024-03-23T14:58:04.623 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.617+0000 7f9796ffd640 1 -- 172.21.15.164:0/3064153484 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97980a58f0 con 0x7f97a0141890 2024-03-23T14:58:04.623 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.617+0000 7f9796ffd640 1 -- 172.21.15.164:0/3064153484 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97980adbb0 con 0x7f97a0141890 2024-03-23T14:58:04.623 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.617+0000 7f97a849b640 1 -- 172.21.15.164:0/3064153484 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97a014f720 con 0x7f97a0141890 2024-03-23T14:58:04.624 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.617+0000 7f97a849b640 1 -- 172.21.15.164:0/3064153484 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97a014bcc0 con 0x7f97a0141890 2024-03-23T14:58:04.624 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.617+0000 7f97a849b640 1 -- 172.21.15.164:0/3064153484 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f97a00b4930 con 0x7f97a0141890 2024-03-23T14:58:04.625 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.617+0000 7f9796ffd640 1 -- 172.21.15.164:0/3064153484 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7f97980b4050 con 0x7f97a0141890 2024-03-23T14:58:04.625 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.617+0000 7f9796ffd640 1 --2- 172.21.15.164:0/3064153484 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f97840672d0 0x7f9784069790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:04.625 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.617+0000 7f9796ffd640 1 -- 172.21.15.164:0/3064153484 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4943+0+0 (secure 0 0 0) 0x7f9798120010 con 0x7f97a0141890 2024-03-23T14:58:04.625 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.617+0000 7f97a6237640 1 --2- 172.21.15.164:0/3064153484 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f97840672d0 0x7f9784069790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:04.626 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.617+0000 7f9796ffd640 1 --2- 172.21.15.164:0/3064153484 >> [v2:172.21.15.164:6824/812540271,v1:172.21.15.164:6825/812540271] conn(0x7f978406d190 0x7f978406f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:04.626 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.617+0000 7f9796ffd640 1 -- 172.21.15.164:0/3064153484 --> [v2:172.21.15.164:6824/812540271,v1:172.21.15.164:6825/812540271] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f978406fc60 con 0x7f978406d190 2024-03-23T14:58:04.626 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.617+0000 7f9796ffd640 1 -- 172.21.15.164:0/3064153484 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f9798120460 con 0x7f97a0141890 2024-03-23T14:58:04.626 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.617+0000 7f97a5a36640 1 --2- 172.21.15.164:0/3064153484 >> [v2:172.21.15.164:6824/812540271,v1:172.21.15.164:6825/812540271] conn(0x7f978406d190 0x7f978406f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:04.626 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.617+0000 7f97a6237640 1 --2- 172.21.15.164:0/3064153484 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f97840672d0 0x7f9784069790 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f97a0145340 tx=0x7f979c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:58:04.626 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.617+0000 7f97a5a36640 1 --2- 172.21.15.164:0/3064153484 >> [v2:172.21.15.164:6824/812540271,v1:172.21.15.164:6825/812540271] conn(0x7f978406d190 0x7f978406f5b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:58:04.626 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.617+0000 7f9796ffd640 1 -- 172.21.15.164:0/3064153484 <== osd.3 v2:172.21.15.164:6824/812540271 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f978406fc60 con 0x7f978406d190 2024-03-23T14:58:04.636 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.633+0000 7f97a849b640 1 -- 172.21.15.164:0/3064153484 --> [v2:172.21.15.164:6824/812540271,v1:172.21.15.164:6825/812540271] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f97a01e2d70 con 0x7f978406d190 2024-03-23T14:58:04.637 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.633+0000 7f9796ffd640 1 -- 172.21.15.164:0/3064153484 <== osd.3 v2:172.21.15.164:6824/812540271 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f97a01e2d70 con 0x7f978406d190 2024-03-23T14:58:04.637 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-03-23T14:58:04.637 INFO:teuthology.orchestra.run.smithi164.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-23T14:58:04.637 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-03-23T14:58:04.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.633+0000 7f97a849b640 1 -- 172.21.15.164:0/3064153484 >> [v2:172.21.15.164:6824/812540271,v1:172.21.15.164:6825/812540271] conn(0x7f978406d190 msgr2=0x7f978406f5b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:04.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.633+0000 7f97a849b640 1 --2- 172.21.15.164:0/3064153484 >> [v2:172.21.15.164:6824/812540271,v1:172.21.15.164:6825/812540271] conn(0x7f978406d190 0x7f978406f5b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:04.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.633+0000 7f97a849b640 1 -- 172.21.15.164:0/3064153484 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f97840672d0 msgr2=0x7f9784069790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:04.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.633+0000 7f97a849b640 1 --2- 172.21.15.164:0/3064153484 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f97840672d0 0x7f9784069790 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f97a0145340 tx=0x7f979c008040 comp rx=0 tx=0).stop 2024-03-23T14:58:04.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.633+0000 7f97a849b640 1 -- 172.21.15.164:0/3064153484 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f97a0141890 msgr2=0x7f97a014aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:04.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.633+0000 7f97a849b640 1 --2- 172.21.15.164:0/3064153484 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f97a0141890 0x7f97a014aea0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f979801aed0 tx=0x7f97980195c0 comp rx=0 tx=0).stop 2024-03-23T14:58:04.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.633+0000 7f97a6a38640 1 -- 172.21.15.164:0/3064153484 reap_dead start 2024-03-23T14:58:04.639 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.633+0000 7f97a849b640 1 -- 172.21.15.164:0/3064153484 shutdown_connections 2024-03-23T14:58:04.640 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.633+0000 7f97a849b640 1 -- 172.21.15.164:0/3064153484 >> 172.21.15.164:0/3064153484 conn(0x7f97a0072cd0 msgr2=0x7f97a00bbcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:58:04.640 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.633+0000 7f97a849b640 1 -- 172.21.15.164:0/3064153484 shutdown_connections 2024-03-23T14:58:04.640 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.633+0000 7f97a849b640 1 -- 172.21.15.164:0/3064153484 wait complete. 2024-03-23T14:58:04.653 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-03-23T14:58:04.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.769+0000 7f506a414640 1 Processor -- start 2024-03-23T14:58:04.773 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.769+0000 7f506a414640 1 -- start start 2024-03-23T14:58:04.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.769+0000 7f506a414640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f50640b9ac0 0x7f506413f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:04.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.769+0000 7f506a414640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f506406f8e0 0x7f50641459b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:04.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.769+0000 7f506a414640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5064145ef0 0x7f5064148810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:04.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.769+0000 7f506a414640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f50640b3b50 con 0x7f506406f8e0 2024-03-23T14:58:04.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.769+0000 7f506a414640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 -- 0x7f5064145240 con 0x7f50640b9ac0 2024-03-23T14:58:04.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.769+0000 7f506a414640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f5064070050 con 0x7f5064145ef0 2024-03-23T14:58:04.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.769+0000 7f50637fe640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f506406f8e0 0x7f50641459b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:04.775 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.769+0000 7f50637fe640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f506406f8e0 0x7f50641459b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:48942/0 (socket says 172.21.15.164:48942) 2024-03-23T14:58:04.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.769+0000 7f50637fe640 1 -- 172.21.15.164:0/3146138781 learned_addr learned my addr 172.21.15.164:0/3146138781 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:58:04.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.769+0000 7f50689b1640 1 --2- 172.21.15.164:0/3146138781 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5064145ef0 0x7f5064148810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:04.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.769+0000 7f5063fff640 1 --2- 172.21.15.164:0/3146138781 >> v2:172.21.15.173:3300/0 conn(0x7f50640b9ac0 0x7f506413f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:04.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.769+0000 7f50637fe640 1 -- 172.21.15.164:0/3146138781 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5064145ef0 msgr2=0x7f5064148810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:58:04.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.769+0000 7f50637fe640 1 --2- 172.21.15.164:0/3146138781 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5064145ef0 0x7f5064148810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:04.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.769+0000 7f50637fe640 1 -- 172.21.15.164:0/3146138781 >> v2:172.21.15.173:3300/0 conn(0x7f50640b9ac0 msgr2=0x7f506413f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:58:04.776 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.769+0000 7f50637fe640 1 --2- 172.21.15.164:0/3146138781 >> v2:172.21.15.173:3300/0 conn(0x7f50640b9ac0 0x7f506413f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:04.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.769+0000 7f50637fe640 1 -- 172.21.15.164:0/3146138781 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50640b9ea0 con 0x7f506406f8e0 2024-03-23T14:58:04.777 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.769+0000 7f5063fff640 1 --2- 172.21.15.164:0/3146138781 >> v2:172.21.15.173:3300/0 conn(0x7f50640b9ac0 0x7f506413f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T14:58:04.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.773+0000 7f50637fe640 1 --2- 172.21.15.164:0/3146138781 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f506406f8e0 0x7f50641459b0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f50640ba090 tx=0x7f505801c1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb539d464faa0bc1 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:58:04.778 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.773+0000 7f5062ffd640 1 -- 172.21.15.164:0/3146138781 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50580ab600 con 0x7f506406f8e0 2024-03-23T14:58:04.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.773+0000 7f5062ffd640 1 -- 172.21.15.164:0/3146138781 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50580a7050 con 0x7f506406f8e0 2024-03-23T14:58:04.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.773+0000 7f5062ffd640 1 -- 172.21.15.164:0/3146138781 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50580ab980 con 0x7f506406f8e0 2024-03-23T14:58:04.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.773+0000 7f506a414640 1 -- 172.21.15.164:0/3146138781 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f506406f8e0 msgr2=0x7f50641459b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:04.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.773+0000 7f506a414640 1 --2- 172.21.15.164:0/3146138781 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f506406f8e0 0x7f50641459b0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f50640ba090 tx=0x7f505801c1a0 comp rx=0 tx=0).stop 2024-03-23T14:58:04.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.773+0000 7f506a414640 1 -- 172.21.15.164:0/3146138781 shutdown_connections 2024-03-23T14:58:04.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.773+0000 7f506a414640 1 --2- 172.21.15.164:0/3146138781 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5064145ef0 0x7f5064148810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:04.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.773+0000 7f506a414640 1 --2- 172.21.15.164:0/3146138781 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f506406f8e0 0x7f50641459b0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:04.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.773+0000 7f506a414640 1 --2- 172.21.15.164:0/3146138781 >> v2:172.21.15.173:3300/0 conn(0x7f50640b9ac0 0x7f506413f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:04.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.773+0000 7f506a414640 1 -- 172.21.15.164:0/3146138781 >> 172.21.15.164:0/3146138781 conn(0x7f506406b930 msgr2=0x7f50640b9150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:58:04.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.773+0000 7f506a414640 1 -- 172.21.15.164:0/3146138781 shutdown_connections 2024-03-23T14:58:04.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.773+0000 7f506a414640 1 -- 172.21.15.164:0/3146138781 wait complete. 2024-03-23T14:58:04.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.773+0000 7f506a414640 1 Processor -- start 2024-03-23T14:58:04.780 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.773+0000 7f506a414640 1 -- start start 2024-03-23T14:58:04.783 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.773+0000 7f506a414640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f50640b9ac0 0x7f506413dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:04.783 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.773+0000 7f506a414640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5064145ef0 0x7f506413cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:04.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.773+0000 7f506a414640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f506413d500 0x7f5064135230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:04.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.773+0000 7f506a414640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5064072f60 con 0x7f5064145ef0 2024-03-23T14:58:04.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.773+0000 7f506a414640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f50640b8710 con 0x7f506413d500 2024-03-23T14:58:04.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.773+0000 7f506a414640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f50640b8430 con 0x7f50640b9ac0 2024-03-23T14:58:04.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.773+0000 7f50637fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5064145ef0 0x7f506413cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:04.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.773+0000 7f50637fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5064145ef0 0x7f506413cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:48948/0 (socket says 172.21.15.164:48948) 2024-03-23T14:58:04.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.773+0000 7f50637fe640 1 -- 172.21.15.164:0/4291868118 learned_addr learned my addr 172.21.15.164:0/4291868118 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:58:04.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.777+0000 7f5063fff640 1 --2- 172.21.15.164:0/4291868118 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f50640b9ac0 0x7f506413dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:04.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.777+0000 7f50689b1640 1 --2- 172.21.15.164:0/4291868118 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f506413d500 0x7f5064135230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:04.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.777+0000 7f50637fe640 1 -- 172.21.15.164:0/4291868118 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f50640b9ac0 msgr2=0x7f506413dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:04.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.777+0000 7f50637fe640 1 --2- 172.21.15.164:0/4291868118 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f50640b9ac0 0x7f506413dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:04.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.777+0000 7f50637fe640 1 -- 172.21.15.164:0/4291868118 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f506413d500 msgr2=0x7f5064135230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:04.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.777+0000 7f50637fe640 1 --2- 172.21.15.164:0/4291868118 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f506413d500 0x7f5064135230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:04.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.777+0000 7f50637fe640 1 -- 172.21.15.164:0/4291868118 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50580094d0 con 0x7f5064145ef0 2024-03-23T14:58:04.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.777+0000 7f50689b1640 1 --2- 172.21.15.164:0/4291868118 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f506413d500 0x7f5064135230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T14:58:04.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.777+0000 7f5063fff640 1 --2- 172.21.15.164:0/4291868118 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f50640b9ac0 0x7f506413dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T14:58:04.785 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.777+0000 7f50637fe640 1 --2- 172.21.15.164:0/4291868118 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5064145ef0 0x7f506413cb90 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f50580ae040 tx=0x7f5058018f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:58:04.786 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.777+0000 7f5060ff9640 1 -- 172.21.15.164:0/4291868118 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50580abde0 con 0x7f5064145ef0 2024-03-23T14:58:04.786 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.777+0000 7f5060ff9640 1 -- 172.21.15.164:0/4291868118 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50580b4440 con 0x7f5064145ef0 2024-03-23T14:58:04.786 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.777+0000 7f506a414640 1 -- 172.21.15.164:0/4291868118 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5064070af0 con 0x7f5064145ef0 2024-03-23T14:58:04.786 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.777+0000 7f5060ff9640 1 -- 172.21.15.164:0/4291868118 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50580b47d0 con 0x7f5064145ef0 2024-03-23T14:58:04.786 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.777+0000 7f506a414640 1 -- 172.21.15.164:0/4291868118 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5064139270 con 0x7f5064145ef0 2024-03-23T14:58:04.786 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.777+0000 7f506a414640 1 -- 172.21.15.164:0/4291868118 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50640690a0 con 0x7f5064145ef0 2024-03-23T14:58:04.786 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.777+0000 7f5060ff9640 1 -- 172.21.15.164:0/4291868118 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7f50580b2070 con 0x7f5064145ef0 2024-03-23T14:58:04.786 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.777+0000 7f5060ff9640 1 --2- 172.21.15.164:0/4291868118 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f50500672a0 0x7f5050069760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:04.787 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.777+0000 7f5063fff640 1 --2- 172.21.15.164:0/4291868118 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f50500672a0 0x7f5050069760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:04.787 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.777+0000 7f5060ff9640 1 -- 172.21.15.164:0/4291868118 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4943+0+0 (secure 0 0 0) 0x7f505811eb20 con 0x7f5064145ef0 2024-03-23T14:58:04.787 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.777+0000 7f5063fff640 1 --2- 172.21.15.164:0/4291868118 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f50500672a0 0x7f5050069760 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f505c007130 tx=0x7f505c007670 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:58:04.787 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.781+0000 7f5060ff9640 1 -- 172.21.15.164:0/4291868118 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7f50580eba60 con 0x7f5064145ef0 2024-03-23T14:58:04.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.945+0000 7f506a414640 1 -- 172.21.15.164:0/4291868118 --> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7f506414dd50 con 0x7f50500672a0 2024-03-23T14:58:04.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.945+0000 7f5060ff9640 1 -- 172.21.15.164:0/4291868118 <== mgr.4107 v2:172.21.15.164:6832/2139087985 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f506414dd50 con 0x7f50500672a0 2024-03-23T14:58:04.952 INFO:teuthology.orchestra.run.smithi164.stderr:instructed osd(s) 3 to deep-scrub 2024-03-23T14:58:04.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.949+0000 7f506a414640 1 -- 172.21.15.164:0/4291868118 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f50500672a0 msgr2=0x7f5050069760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:04.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.949+0000 7f506a414640 1 --2- 172.21.15.164:0/4291868118 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f50500672a0 0x7f5050069760 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f505c007130 tx=0x7f505c007670 comp rx=0 tx=0).stop 2024-03-23T14:58:04.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.949+0000 7f506a414640 1 -- 172.21.15.164:0/4291868118 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5064145ef0 msgr2=0x7f506413cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:04.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.949+0000 7f506a414640 1 --2- 172.21.15.164:0/4291868118 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5064145ef0 0x7f506413cb90 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f50580ae040 tx=0x7f5058018f40 comp rx=0 tx=0).stop 2024-03-23T14:58:04.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.949+0000 7f506a414640 1 -- 172.21.15.164:0/4291868118 shutdown_connections 2024-03-23T14:58:04.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.949+0000 7f506a414640 1 --2- 172.21.15.164:0/4291868118 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f50500672a0 0x7f5050069760 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:04.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.949+0000 7f506a414640 1 --2- 172.21.15.164:0/4291868118 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f506413d500 0x7f5064135230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:04.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.949+0000 7f506a414640 1 --2- 172.21.15.164:0/4291868118 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5064145ef0 0x7f506413cb90 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:04.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.949+0000 7f506a414640 1 --2- 172.21.15.164:0/4291868118 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f50640b9ac0 0x7f506413dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:04.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.949+0000 7f506a414640 1 -- 172.21.15.164:0/4291868118 >> 172.21.15.164:0/4291868118 conn(0x7f506406b930 msgr2=0x7f50640b3ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:58:04.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.949+0000 7f506a414640 1 -- 172.21.15.164:0/4291868118 shutdown_connections 2024-03-23T14:58:04.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:04.949+0000 7f506a414640 1 -- 172.21.15.164:0/4291868118 wait complete. 2024-03-23T14:58:04.970 INFO:tasks.ceph:Scrubbing osd.4 2024-03-23T14:58:04.970 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-03-23T14:58:05.097 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.093+0000 7f9ffcc6b640 1 Processor -- start 2024-03-23T14:58:05.097 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.093+0000 7f9ffcc6b640 1 -- start start 2024-03-23T14:58:05.098 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.093+0000 7f9ffcc6b640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f9ff80b5790 0x7f9ff8141320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:05.099 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.093+0000 7f9ffcc6b640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f9ff80b82b0 0x7f9ff814acf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:05.099 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.093+0000 7f9ffcc6b640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ff806ca90 con 0x7f9ff80b5b70 2024-03-23T14:58:05.099 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.093+0000 7f9ffcc6b640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f9ff80b7110 con 0x7f9ff80b82b0 2024-03-23T14:58:05.099 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.093+0000 7f9ffcc6b640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 -- 0x7f9ff806d220 con 0x7f9ff80b5790 2024-03-23T14:58:05.099 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.093+0000 7f9ff5d9b640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f9ff80b82b0 0x7f9ff814acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:05.099 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.093+0000 7f9ff6d9d640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f9ff80b5b70 0x7f9ff80b6050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:37850/0 (socket says 172.21.15.164:37850) 2024-03-23T14:58:05.099 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.093+0000 7f9ff6d9d640 1 -- 172.21.15.164:0/466110674 learned_addr learned my addr 172.21.15.164:0/466110674 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-03-23T14:58:05.099 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.093+0000 7f9ff659c640 1 --2- 172.21.15.164:0/466110674 >> v2:172.21.15.173:3300/0 conn(0x7f9ff80b5790 0x7f9ff8141320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:05.100 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.093+0000 7f9ff559a640 1 -- 172.21.15.164:0/466110674 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4222367667 0 0) 0x7f9ff806ca90 con 0x7f9ff80b5b70 2024-03-23T14:58:05.100 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.093+0000 7f9ff559a640 1 -- 172.21.15.164:0/466110674 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9fe0003530 con 0x7f9ff80b5b70 2024-03-23T14:58:05.100 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.093+0000 7f9ff5d9b640 1 -- 172.21.15.164:0/466110674 >> v2:172.21.15.173:3300/0 conn(0x7f9ff80b5790 msgr2=0x7f9ff8141320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:58:05.100 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.093+0000 7f9ff5d9b640 1 --2- 172.21.15.164:0/466110674 >> v2:172.21.15.173:3300/0 conn(0x7f9ff80b5790 0x7f9ff8141320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:05.100 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.093+0000 7f9ff5d9b640 1 -- 172.21.15.164:0/466110674 >> v1:172.21.15.164:6789/0 conn(0x7f9ff80b5b70 legacy=0x7f9ff80b6050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:05.100 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.093+0000 7f9ff5d9b640 1 -- 172.21.15.164:0/466110674 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ff806c7b0 con 0x7f9ff80b82b0 2024-03-23T14:58:05.101 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.097+0000 7f9ff5d9b640 1 --2- 172.21.15.164:0/466110674 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9ff80b82b0 0x7f9ff814acf0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f9ff806f350 tx=0x7f9fec01c2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e7114508a176e6c server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:58:05.101 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.097+0000 7f9ff559a640 1 -- 172.21.15.164:0/466110674 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fec0ab600 con 0x7f9ff80b82b0 2024-03-23T14:58:05.101 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.097+0000 7f9ff559a640 1 -- 172.21.15.164:0/466110674 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fec0a7050 con 0x7f9ff80b82b0 2024-03-23T14:58:05.101 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.097+0000 7f9ff559a640 1 -- 172.21.15.164:0/466110674 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fec0ab980 con 0x7f9ff80b82b0 2024-03-23T14:58:05.102 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.097+0000 7f9ffcc6b640 1 -- 172.21.15.164:0/466110674 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9ff80b82b0 msgr2=0x7f9ff814acf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:05.103 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.097+0000 7f9ffcc6b640 1 --2- 172.21.15.164:0/466110674 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9ff80b82b0 0x7f9ff814acf0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f9ff806f350 tx=0x7f9fec01c2a0 comp rx=0 tx=0).stop 2024-03-23T14:58:05.103 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.097+0000 7f9ffcc6b640 1 -- 172.21.15.164:0/466110674 shutdown_connections 2024-03-23T14:58:05.103 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.097+0000 7f9ffcc6b640 1 --2- 172.21.15.164:0/466110674 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9ff80b82b0 0x7f9ff814acf0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:05.103 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.097+0000 7f9ffcc6b640 1 --2- 172.21.15.164:0/466110674 >> v2:172.21.15.173:3300/0 conn(0x7f9ff80b5790 0x7f9ff8141320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:05.103 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.097+0000 7f9ffcc6b640 1 -- 172.21.15.164:0/466110674 >> 172.21.15.164:0/466110674 conn(0x7f9ff80b7630 msgr2=0x7f9ff80b7a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:58:05.103 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.097+0000 7f9ffcc6b640 1 -- 172.21.15.164:0/466110674 shutdown_connections 2024-03-23T14:58:05.103 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.097+0000 7f9ffcc6b640 1 -- 172.21.15.164:0/466110674 wait complete. 2024-03-23T14:58:05.103 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.097+0000 7f9ffcc6b640 1 Processor -- start 2024-03-23T14:58:05.104 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.097+0000 7f9ffcc6b640 1 -- start start 2024-03-23T14:58:05.106 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.097+0000 7f9ffcc6b640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9ff80b5790 0x7f9ff81487d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:05.106 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.097+0000 7f9ffcc6b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9ff80b5b70 0x7f9ff8149700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:05.106 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.097+0000 7f9ffcc6b640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9ff81e0940 0x7f9ff8149c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:05.106 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.097+0000 7f9ffcc6b640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff806c5d0 con 0x7f9ff80b5b70 2024-03-23T14:58:05.106 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.097+0000 7f9ffcc6b640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff81e0e90 con 0x7f9ff81e0940 2024-03-23T14:58:05.106 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.097+0000 7f9ffcc6b640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f9ff81e1170 con 0x7f9ff80b5790 2024-03-23T14:58:05.106 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.097+0000 7f9ff5d9b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9ff80b5b70 0x7f9ff8149700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:05.107 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.097+0000 7f9ff5d9b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9ff80b5b70 0x7f9ff8149700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:48968/0 (socket says 172.21.15.164:48968) 2024-03-23T14:58:05.107 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.097+0000 7f9ff5d9b640 1 -- 172.21.15.164:0/1063226573 learned_addr learned my addr 172.21.15.164:0/1063226573 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:58:05.107 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.101+0000 7f9ff6d9d640 1 --2- 172.21.15.164:0/1063226573 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9ff81e0940 0x7f9ff8149c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:05.107 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.101+0000 7f9ff659c640 1 --2- 172.21.15.164:0/1063226573 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9ff80b5790 0x7f9ff81487d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:05.107 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.101+0000 7f9ff5d9b640 1 -- 172.21.15.164:0/1063226573 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9ff80b5790 msgr2=0x7f9ff81487d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:05.107 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.101+0000 7f9ff5d9b640 1 --2- 172.21.15.164:0/1063226573 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9ff80b5790 0x7f9ff81487d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:05.107 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.101+0000 7f9ff5d9b640 1 -- 172.21.15.164:0/1063226573 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9ff81e0940 msgr2=0x7f9ff8149c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:05.108 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.101+0000 7f9ff5d9b640 1 --2- 172.21.15.164:0/1063226573 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9ff81e0940 0x7f9ff8149c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:05.108 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.101+0000 7f9ff5d9b640 1 -- 172.21.15.164:0/1063226573 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fec009420 con 0x7f9ff80b5b70 2024-03-23T14:58:05.108 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.101+0000 7f9ff5d9b640 1 --2- 172.21.15.164:0/1063226573 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9ff80b5b70 0x7f9ff8149700 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f9fec0ae040 tx=0x7f9fec019040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:58:05.108 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.101+0000 7f9fe6ffd640 1 -- 172.21.15.164:0/1063226573 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fec0b4430 con 0x7f9ff80b5b70 2024-03-23T14:58:05.108 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.101+0000 7f9fe6ffd640 1 -- 172.21.15.164:0/1063226573 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fec0b45b0 con 0x7f9ff80b5b70 2024-03-23T14:58:05.108 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.101+0000 7f9ffcc6b640 1 -- 172.21.15.164:0/1063226573 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ff8140c80 con 0x7f9ff80b5b70 2024-03-23T14:58:05.108 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.101+0000 7f9fe6ffd640 1 -- 172.21.15.164:0/1063226573 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fec0b4960 con 0x7f9ff80b5b70 2024-03-23T14:58:05.109 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.101+0000 7f9ffcc6b640 1 -- 172.21.15.164:0/1063226573 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ff8148ee0 con 0x7f9ff80b5b70 2024-03-23T14:58:05.109 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.101+0000 7f9ffcc6b640 1 -- 172.21.15.164:0/1063226573 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9ff80b7a10 con 0x7f9ff80b5b70 2024-03-23T14:58:05.109 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.101+0000 7f9fe6ffd640 1 -- 172.21.15.164:0/1063226573 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7f9fec0b2040 con 0x7f9ff80b5b70 2024-03-23T14:58:05.109 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.101+0000 7f9fe6ffd640 1 --2- 172.21.15.164:0/1063226573 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f9fd80672a0 0x7f9fd8069760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:05.110 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.101+0000 7f9fe6ffd640 1 -- 172.21.15.164:0/1063226573 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4943+0+0 (secure 0 0 0) 0x7f9fec0a4e40 con 0x7f9ff80b5b70 2024-03-23T14:58:05.110 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.101+0000 7f9ff659c640 1 --2- 172.21.15.164:0/1063226573 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f9fd80672a0 0x7f9fd8069760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:05.110 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.101+0000 7f9fe6ffd640 1 --2- 172.21.15.164:0/1063226573 >> [v2:172.21.15.173:6824/2656857421,v1:172.21.15.173:6825/2656857421] conn(0x7f9fd806d160 0x7f9fd806f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:05.111 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.101+0000 7f9fe6ffd640 1 -- 172.21.15.164:0/1063226573 --> [v2:172.21.15.173:6824/2656857421,v1:172.21.15.173:6825/2656857421] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9fd806fc30 con 0x7f9fd806d160 2024-03-23T14:58:05.111 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.101+0000 7f9fe6ffd640 1 -- 172.21.15.164:0/1063226573 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f9fec11f830 con 0x7f9ff80b5b70 2024-03-23T14:58:05.111 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.101+0000 7f9ff659c640 1 --2- 172.21.15.164:0/1063226573 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f9fd80672a0 0x7f9fd8069760 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f9ff0003fd0 tx=0x7f9ff00039a0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:58:05.111 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.101+0000 7f9ff6d9d640 1 --2- 172.21.15.164:0/1063226573 >> [v2:172.21.15.173:6824/2656857421,v1:172.21.15.173:6825/2656857421] conn(0x7f9fd806d160 0x7f9fd806f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:05.111 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.101+0000 7f9ff6d9d640 1 --2- 172.21.15.164:0/1063226573 >> [v2:172.21.15.173:6824/2656857421,v1:172.21.15.173:6825/2656857421] conn(0x7f9fd806d160 0x7f9fd806f580 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:58:05.111 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.105+0000 7f9fe6ffd640 1 -- 172.21.15.164:0/1063226573 <== osd.4 v2:172.21.15.173:6824/2656857421 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9fd806fc30 con 0x7f9fd806d160 2024-03-23T14:58:05.122 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.117+0000 7f9ffcc6b640 1 -- 172.21.15.164:0/1063226573 --> [v2:172.21.15.173:6824/2656857421,v1:172.21.15.173:6825/2656857421] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f9ff806c100 con 0x7f9fd806d160 2024-03-23T14:58:05.123 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.117+0000 7f9fe6ffd640 1 -- 172.21.15.164:0/1063226573 <== osd.4 v2:172.21.15.173:6824/2656857421 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f9ff806c100 con 0x7f9fd806d160 2024-03-23T14:58:05.123 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-03-23T14:58:05.123 INFO:teuthology.orchestra.run.smithi164.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-23T14:58:05.123 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-03-23T14:58:05.124 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.117+0000 7f9ffcc6b640 1 -- 172.21.15.164:0/1063226573 >> [v2:172.21.15.173:6824/2656857421,v1:172.21.15.173:6825/2656857421] conn(0x7f9fd806d160 msgr2=0x7f9fd806f580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:05.124 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.117+0000 7f9ffcc6b640 1 --2- 172.21.15.164:0/1063226573 >> [v2:172.21.15.173:6824/2656857421,v1:172.21.15.173:6825/2656857421] conn(0x7f9fd806d160 0x7f9fd806f580 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:05.124 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.117+0000 7f9ffcc6b640 1 -- 172.21.15.164:0/1063226573 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f9fd80672a0 msgr2=0x7f9fd8069760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:05.125 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.117+0000 7f9ffcc6b640 1 --2- 172.21.15.164:0/1063226573 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f9fd80672a0 0x7f9fd8069760 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f9ff0003fd0 tx=0x7f9ff00039a0 comp rx=0 tx=0).stop 2024-03-23T14:58:05.125 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.117+0000 7f9ffcc6b640 1 -- 172.21.15.164:0/1063226573 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9ff80b5b70 msgr2=0x7f9ff8149700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:05.125 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.117+0000 7f9ffcc6b640 1 --2- 172.21.15.164:0/1063226573 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9ff80b5b70 0x7f9ff8149700 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f9fec0ae040 tx=0x7f9fec019040 comp rx=0 tx=0).stop 2024-03-23T14:58:05.125 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.117+0000 7f9ff6d9d640 1 -- 172.21.15.164:0/1063226573 reap_dead start 2024-03-23T14:58:05.125 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.117+0000 7f9ffcc6b640 1 -- 172.21.15.164:0/1063226573 shutdown_connections 2024-03-23T14:58:05.125 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.117+0000 7f9ffcc6b640 1 -- 172.21.15.164:0/1063226573 >> 172.21.15.164:0/1063226573 conn(0x7f9ff80b7630 msgr2=0x7f9ff813d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:58:05.125 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.117+0000 7f9ffcc6b640 1 -- 172.21.15.164:0/1063226573 shutdown_connections 2024-03-23T14:58:05.125 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.117+0000 7f9ffcc6b640 1 -- 172.21.15.164:0/1063226573 wait complete. 2024-03-23T14:58:05.139 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-03-23T14:58:05.248 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.245+0000 7f31eacda640 1 Processor -- start 2024-03-23T14:58:05.248 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.245+0000 7f31eacda640 1 -- start start 2024-03-23T14:58:05.249 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.245+0000 7f31eacda640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f31e409def0 0x7f31e40b45e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:05.250 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.245+0000 7f31eacda640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f31e40add50 0x7f31e406bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:05.250 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.245+0000 7f31eacda640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31e406d650 con 0x7f31e40b4b20 2024-03-23T14:58:05.250 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.245+0000 7f31eacda640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f31e40af4a0 con 0x7f31e409def0 2024-03-23T14:58:05.250 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.245+0000 7f31eacda640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f31e409e2d0 con 0x7f31e40add50 2024-03-23T14:58:05.250 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.245+0000 7f31e8a76640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f31e409def0 0x7f31e40b45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:05.250 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.245+0000 7f31e8a76640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f31e409def0 0x7f31e40b45e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:48978/0 (socket says 172.21.15.164:48978) 2024-03-23T14:58:05.250 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.245+0000 7f31e8a76640 1 -- 172.21.15.164:0/3966529335 learned_addr learned my addr 172.21.15.164:0/3966529335 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:58:05.251 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.245+0000 7f31e8a76640 1 -- 172.21.15.164:0/3966529335 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f31e40add50 msgr2=0x7f31e406bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:58:05.251 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.245+0000 7f31e8a76640 1 --2- 172.21.15.164:0/3966529335 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f31e40add50 0x7f31e406bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:05.251 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.245+0000 7f31e8a76640 1 -- 172.21.15.164:0/3966529335 >> v1:172.21.15.164:6789/0 conn(0x7f31e40b4b20 legacy=0x7f31e406c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:05.251 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.245+0000 7f31e8a76640 1 -- 172.21.15.164:0/3966529335 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31e40aeca0 con 0x7f31e409def0 2024-03-23T14:58:05.251 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.245+0000 7f31e37fe640 1 -- 172.21.15.164:0/3966529335 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4051890605 0 0) 0x7f31e406d650 con 0x7f31e40b4b20 2024-03-23T14:58:05.251 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.249+0000 7f31e8a76640 1 --2- 172.21.15.164:0/3966529335 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f31e409def0 0x7f31e40b45e0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f31d400bb10 tx=0x7f31d401eb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3de5ed6d0cf66cf2 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:58:05.252 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.249+0000 7f31e37fe640 1 -- 172.21.15.164:0/3966529335 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31d40a7a40 con 0x7f31e409def0 2024-03-23T14:58:05.252 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.249+0000 7f31e37fe640 1 -- 172.21.15.164:0/3966529335 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31d40a7ba0 con 0x7f31e409def0 2024-03-23T14:58:05.252 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.249+0000 7f31e37fe640 1 -- 172.21.15.164:0/3966529335 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31d40af440 con 0x7f31e409def0 2024-03-23T14:58:05.254 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.249+0000 7f31eacda640 1 -- 172.21.15.164:0/3966529335 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f31e409def0 msgr2=0x7f31e40b45e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:05.254 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.249+0000 7f31eacda640 1 --2- 172.21.15.164:0/3966529335 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f31e409def0 0x7f31e40b45e0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f31d400bb10 tx=0x7f31d401eb10 comp rx=0 tx=0).stop 2024-03-23T14:58:05.254 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.249+0000 7f31eacda640 1 -- 172.21.15.164:0/3966529335 shutdown_connections 2024-03-23T14:58:05.254 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.249+0000 7f31eacda640 1 --2- 172.21.15.164:0/3966529335 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f31e40add50 0x7f31e406bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:05.254 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.249+0000 7f31eacda640 1 --2- 172.21.15.164:0/3966529335 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f31e409def0 0x7f31e40b45e0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:05.254 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.249+0000 7f31eacda640 1 -- 172.21.15.164:0/3966529335 >> 172.21.15.164:0/3966529335 conn(0x7f31e40b8ca0 msgr2=0x7f31e40b90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:58:05.254 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.249+0000 7f31eacda640 1 -- 172.21.15.164:0/3966529335 shutdown_connections 2024-03-23T14:58:05.254 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.249+0000 7f31eacda640 1 -- 172.21.15.164:0/3966529335 wait complete. 2024-03-23T14:58:05.255 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.249+0000 7f31eacda640 1 Processor -- start 2024-03-23T14:58:05.255 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.249+0000 7f31eacda640 1 -- start start 2024-03-23T14:58:05.257 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.249+0000 7f31eacda640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f31e409def0 0x7f31e413c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:05.257 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.249+0000 7f31eacda640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f31e40add50 0x7f31e409c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:05.257 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.249+0000 7f31eacda640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f31e40b4b20 0x7f31e40b1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:05.257 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.249+0000 7f31eacda640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f31e409bad0 con 0x7f31e40add50 2024-03-23T14:58:05.257 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.249+0000 7f31eacda640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f31e409c530 con 0x7f31e40b4b20 2024-03-23T14:58:05.257 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.249+0000 7f31eacda640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f31e413e960 con 0x7f31e409def0 2024-03-23T14:58:05.257 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.249+0000 7f31e3fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f31e40add50 0x7f31e409c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:05.258 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.249+0000 7f31e3fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f31e40add50 0x7f31e409c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:48982/0 (socket says 172.21.15.164:48982) 2024-03-23T14:58:05.258 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.249+0000 7f31e3fff640 1 -- 172.21.15.164:0/4084320455 learned_addr learned my addr 172.21.15.164:0/4084320455 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:58:05.258 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.249+0000 7f31e8a76640 1 --2- 172.21.15.164:0/4084320455 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f31e409def0 0x7f31e413c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:05.258 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.249+0000 7f31e9277640 1 --2- 172.21.15.164:0/4084320455 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f31e40b4b20 0x7f31e40b1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:05.258 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.249+0000 7f31e3fff640 1 -- 172.21.15.164:0/4084320455 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f31e409def0 msgr2=0x7f31e413c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:05.258 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.249+0000 7f31e3fff640 1 --2- 172.21.15.164:0/4084320455 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f31e409def0 0x7f31e413c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:05.258 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.249+0000 7f31e3fff640 1 -- 172.21.15.164:0/4084320455 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f31e40b4b20 msgr2=0x7f31e40b1ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:05.258 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.249+0000 7f31e3fff640 1 --2- 172.21.15.164:0/4084320455 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f31e40b4b20 0x7f31e40b1ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:05.259 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.249+0000 7f31e3fff640 1 -- 172.21.15.164:0/4084320455 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31d400b770 con 0x7f31e40add50 2024-03-23T14:58:05.259 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.249+0000 7f31e9277640 1 --2- 172.21.15.164:0/4084320455 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f31e40b4b20 0x7f31e40b1ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T14:58:05.259 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.249+0000 7f31e3fff640 1 --2- 172.21.15.164:0/4084320455 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f31e40add50 0x7f31e409c9c0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f31d800b550 tx=0x7f31d800ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:58:05.259 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.249+0000 7f31e17fa640 1 -- 172.21.15.164:0/4084320455 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31d8010870 con 0x7f31e40add50 2024-03-23T14:58:05.259 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.249+0000 7f31e17fa640 1 -- 172.21.15.164:0/4084320455 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31d80109f0 con 0x7f31e40add50 2024-03-23T14:58:05.259 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.249+0000 7f31e17fa640 1 -- 172.21.15.164:0/4084320455 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31d8010da0 con 0x7f31e40add50 2024-03-23T14:58:05.259 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.253+0000 7f31eacda640 1 -- 172.21.15.164:0/4084320455 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31e409bce0 con 0x7f31e40add50 2024-03-23T14:58:05.259 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.253+0000 7f31eacda640 1 -- 172.21.15.164:0/4084320455 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31e413cb30 con 0x7f31e40add50 2024-03-23T14:58:05.259 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.253+0000 7f31eacda640 1 -- 172.21.15.164:0/4084320455 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31e40690a0 con 0x7f31e40add50 2024-03-23T14:58:05.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.253+0000 7f31e17fa640 1 -- 172.21.15.164:0/4084320455 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7f31d8023480 con 0x7f31e40add50 2024-03-23T14:58:05.260 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.253+0000 7f31e17fa640 1 --2- 172.21.15.164:0/4084320455 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f31d00672d0 0x7f31d0069790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:05.261 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.253+0000 7f31e17fa640 1 -- 172.21.15.164:0/4084320455 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4943+0+0 (secure 0 0 0) 0x7f31d8084fb0 con 0x7f31e40add50 2024-03-23T14:58:05.261 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.253+0000 7f31e8a76640 1 --2- 172.21.15.164:0/4084320455 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f31d00672d0 0x7f31d0069790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:05.261 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.253+0000 7f31e8a76640 1 --2- 172.21.15.164:0/4084320455 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f31d00672d0 0x7f31d0069790 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f31d40b1000 tx=0x7f31d40a9070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:58:05.262 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.257+0000 7f31e17fa640 1 -- 172.21.15.164:0/4084320455 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7f31d8051fd0 con 0x7f31e40add50 2024-03-23T14:58:05.408 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.405+0000 7f31eacda640 1 -- 172.21.15.164:0/4084320455 --> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7f31e40b3ce0 con 0x7f31d00672d0 2024-03-23T14:58:05.409 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.405+0000 7f31e17fa640 1 -- 172.21.15.164:0/4084320455 <== mgr.4107 v2:172.21.15.164:6832/2139087985 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f31d40b9440 con 0x7f31d00672d0 2024-03-23T14:58:05.409 INFO:teuthology.orchestra.run.smithi164.stderr:instructed osd(s) 4 to deep-scrub 2024-03-23T14:58:05.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.405+0000 7f31eacda640 1 -- 172.21.15.164:0/4084320455 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f31d00672d0 msgr2=0x7f31d0069790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:05.411 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.405+0000 7f31eacda640 1 --2- 172.21.15.164:0/4084320455 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f31d00672d0 0x7f31d0069790 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f31d40b1000 tx=0x7f31d40a9070 comp rx=0 tx=0).stop 2024-03-23T14:58:05.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.405+0000 7f31eacda640 1 -- 172.21.15.164:0/4084320455 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f31e40add50 msgr2=0x7f31e409c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:05.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.405+0000 7f31eacda640 1 --2- 172.21.15.164:0/4084320455 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f31e40add50 0x7f31e409c9c0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f31d800b550 tx=0x7f31d800ba10 comp rx=0 tx=0).stop 2024-03-23T14:58:05.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.405+0000 7f31eacda640 1 -- 172.21.15.164:0/4084320455 shutdown_connections 2024-03-23T14:58:05.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.405+0000 7f31eacda640 1 --2- 172.21.15.164:0/4084320455 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f31d00672d0 0x7f31d0069790 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:05.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.405+0000 7f31eacda640 1 --2- 172.21.15.164:0/4084320455 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f31e40b4b20 0x7f31e40b1ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:05.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.405+0000 7f31eacda640 1 --2- 172.21.15.164:0/4084320455 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f31e40add50 0x7f31e409c9c0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:05.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.405+0000 7f31eacda640 1 --2- 172.21.15.164:0/4084320455 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f31e409def0 0x7f31e413c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:05.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.405+0000 7f31eacda640 1 -- 172.21.15.164:0/4084320455 >> 172.21.15.164:0/4084320455 conn(0x7f31e40b8ca0 msgr2=0x7f31e413d750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:58:05.412 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.405+0000 7f31eacda640 1 -- 172.21.15.164:0/4084320455 shutdown_connections 2024-03-23T14:58:05.413 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.405+0000 7f31eacda640 1 -- 172.21.15.164:0/4084320455 wait complete. 2024-03-23T14:58:05.426 INFO:tasks.ceph:Scrubbing osd.5 2024-03-23T14:58:05.427 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-03-23T14:58:05.538 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.533+0000 7f9fa1ba2640 1 Processor -- start 2024-03-23T14:58:05.538 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.533+0000 7f9fa1ba2640 1 -- start start 2024-03-23T14:58:05.539 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.533+0000 7f9fa1ba2640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9f9c0b56f0 0x7f9f9c14a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:05.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.533+0000 7f9fa1ba2640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f9f9c0b4a70 0x7f9f9c140f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:05.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.533+0000 7f9fa1ba2640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f9f9c141470 0x7f9f9c0b75f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:05.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.533+0000 7f9fa1ba2640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f9f9c0b5fa0 con 0x7f9f9c141470 2024-03-23T14:58:05.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.533+0000 7f9fa1ba2640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 -- 0x7f9f9c0b4300 con 0x7f9f9c0b4a70 2024-03-23T14:58:05.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.533+0000 7f9fa1ba2640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f9f9c0b45e0 con 0x7f9f9c0b56f0 2024-03-23T14:58:05.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.533+0000 7f9f9bfff640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f9f9c141470 0x7f9f9c0b75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:05.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.533+0000 7f9f9bfff640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f9f9c141470 0x7f9f9c0b75f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:48998/0 (socket says 172.21.15.164:48998) 2024-03-23T14:58:05.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.533+0000 7f9f9bfff640 1 -- 172.21.15.164:0/3035103892 learned_addr learned my addr 172.21.15.164:0/3035103892 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:58:05.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.533+0000 7f9f9b7fe640 1 --2- 172.21.15.164:0/3035103892 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9f9c0b56f0 0x7f9f9c14a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:05.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.533+0000 7f9f93fff640 1 --2- 172.21.15.164:0/3035103892 >> v2:172.21.15.173:3300/0 conn(0x7f9f9c0b4a70 0x7f9f9c140f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:05.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.533+0000 7f9f9bfff640 1 -- 172.21.15.164:0/3035103892 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9f9c0b56f0 msgr2=0x7f9f9c14a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:58:05.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.533+0000 7f9f9bfff640 1 --2- 172.21.15.164:0/3035103892 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9f9c0b56f0 0x7f9f9c14a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:05.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.533+0000 7f9f9bfff640 1 -- 172.21.15.164:0/3035103892 >> v2:172.21.15.173:3300/0 conn(0x7f9f9c0b4a70 msgr2=0x7f9f9c140f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:58:05.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.533+0000 7f9f9bfff640 1 --2- 172.21.15.164:0/3035103892 >> v2:172.21.15.173:3300/0 conn(0x7f9f9c0b4a70 0x7f9f9c140f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:05.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.533+0000 7f9f9bfff640 1 -- 172.21.15.164:0/3035103892 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f9c14ade0 con 0x7f9f9c141470 2024-03-23T14:58:05.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.533+0000 7f9f9b7fe640 1 --2- 172.21.15.164:0/3035103892 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9f9c0b56f0 0x7f9f9c14a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T14:58:05.543 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.537+0000 7f9f9bfff640 1 --2- 172.21.15.164:0/3035103892 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9f9c141470 0x7f9f9c0b75f0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f9f9c0ba7b0 tx=0x7f9f9401c710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2a2bddd6a35fef4 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:58:05.543 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.537+0000 7f9f9affd640 1 -- 172.21.15.164:0/3035103892 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f940a5590 con 0x7f9f9c141470 2024-03-23T14:58:05.543 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.537+0000 7f9f9affd640 1 -- 172.21.15.164:0/3035103892 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f940a56f0 con 0x7f9f9c141470 2024-03-23T14:58:05.543 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.537+0000 7f9f9affd640 1 -- 172.21.15.164:0/3035103892 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f940a5a80 con 0x7f9f9c141470 2024-03-23T14:58:05.543 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.537+0000 7f9fa1ba2640 1 -- 172.21.15.164:0/3035103892 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9f9c141470 msgr2=0x7f9f9c0b75f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:05.543 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.537+0000 7f9fa1ba2640 1 --2- 172.21.15.164:0/3035103892 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9f9c141470 0x7f9f9c0b75f0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f9f9c0ba7b0 tx=0x7f9f9401c710 comp rx=0 tx=0).stop 2024-03-23T14:58:05.544 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.537+0000 7f9fa1ba2640 1 -- 172.21.15.164:0/3035103892 shutdown_connections 2024-03-23T14:58:05.544 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.537+0000 7f9fa1ba2640 1 --2- 172.21.15.164:0/3035103892 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9f9c141470 0x7f9f9c0b75f0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:05.544 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.537+0000 7f9fa1ba2640 1 --2- 172.21.15.164:0/3035103892 >> v2:172.21.15.173:3300/0 conn(0x7f9f9c0b4a70 0x7f9f9c140f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:05.544 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.537+0000 7f9fa1ba2640 1 --2- 172.21.15.164:0/3035103892 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9f9c0b56f0 0x7f9f9c14a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:05.544 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.537+0000 7f9fa1ba2640 1 -- 172.21.15.164:0/3035103892 >> 172.21.15.164:0/3035103892 conn(0x7f9f9c02c3c0 msgr2=0x7f9f9c02c7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:58:05.544 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.537+0000 7f9fa1ba2640 1 -- 172.21.15.164:0/3035103892 shutdown_connections 2024-03-23T14:58:05.544 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.537+0000 7f9fa1ba2640 1 -- 172.21.15.164:0/3035103892 wait complete. 2024-03-23T14:58:05.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.541+0000 7f9fa1ba2640 1 Processor -- start 2024-03-23T14:58:05.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.541+0000 7f9fa1ba2640 1 -- start start 2024-03-23T14:58:05.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.541+0000 7f9fa1ba2640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9f9c0b4a70 0x7f9f9c149930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:05.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.541+0000 7f9fa1ba2640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9f9c0b56f0 0x7f9f9c143390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:05.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.541+0000 7f9fa1ba2640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9f9c141470 0x7f9f9c1438d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:05.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.541+0000 7f9fa1ba2640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9f9c140390 con 0x7f9f9c0b56f0 2024-03-23T14:58:05.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.541+0000 7f9fa1ba2640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f9f9c1e0920 con 0x7f9f9c0b4a70 2024-03-23T14:58:05.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.541+0000 7f9fa1ba2640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f9f9c143e10 con 0x7f9f9c141470 2024-03-23T14:58:05.547 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.541+0000 7f9f93fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9f9c0b56f0 0x7f9f9c143390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:05.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.541+0000 7f9f93fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9f9c0b56f0 0x7f9f9c143390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:49002/0 (socket says 172.21.15.164:49002) 2024-03-23T14:58:05.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.541+0000 7f9f93fff640 1 -- 172.21.15.164:0/1881151653 learned_addr learned my addr 172.21.15.164:0/1881151653 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:58:05.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.541+0000 7f9f9b7fe640 1 --2- 172.21.15.164:0/1881151653 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9f9c0b4a70 0x7f9f9c149930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:05.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.541+0000 7f9f9bfff640 1 --2- 172.21.15.164:0/1881151653 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9f9c141470 0x7f9f9c1438d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:05.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.541+0000 7f9f93fff640 1 -- 172.21.15.164:0/1881151653 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9f9c141470 msgr2=0x7f9f9c1438d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:05.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.541+0000 7f9f93fff640 1 --2- 172.21.15.164:0/1881151653 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9f9c141470 0x7f9f9c1438d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:05.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.541+0000 7f9f93fff640 1 -- 172.21.15.164:0/1881151653 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9f9c0b4a70 msgr2=0x7f9f9c149930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:05.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.541+0000 7f9f93fff640 1 --2- 172.21.15.164:0/1881151653 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9f9c0b4a70 0x7f9f9c149930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:05.548 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.541+0000 7f9f93fff640 1 -- 172.21.15.164:0/1881151653 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f94009b30 con 0x7f9f9c0b56f0 2024-03-23T14:58:05.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.541+0000 7f9f93fff640 1 --2- 172.21.15.164:0/1881151653 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9f9c0b56f0 0x7f9f9c143390 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f9f8400c5f0 tx=0x7f9f8400cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:58:05.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.541+0000 7f9f98ff9640 1 -- 172.21.15.164:0/1881151653 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f84011800 con 0x7f9f9c0b56f0 2024-03-23T14:58:05.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.541+0000 7f9f98ff9640 1 -- 172.21.15.164:0/1881151653 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f84011960 con 0x7f9f9c0b56f0 2024-03-23T14:58:05.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.541+0000 7f9fa1ba2640 1 -- 172.21.15.164:0/1881151653 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f9c1e05b0 con 0x7f9f9c0b56f0 2024-03-23T14:58:05.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.541+0000 7f9f98ff9640 1 -- 172.21.15.164:0/1881151653 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f84011cf0 con 0x7f9f9c0b56f0 2024-03-23T14:58:05.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.541+0000 7f9fa1ba2640 1 -- 172.21.15.164:0/1881151653 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f9c1405e0 con 0x7f9f9c0b56f0 2024-03-23T14:58:05.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.541+0000 7f9fa1ba2640 1 -- 172.21.15.164:0/1881151653 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9f9c0b9170 con 0x7f9f9c0b56f0 2024-03-23T14:58:05.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.541+0000 7f9f98ff9640 1 -- 172.21.15.164:0/1881151653 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7f9f840233e0 con 0x7f9f9c0b56f0 2024-03-23T14:58:05.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.541+0000 7f9f98ff9640 1 --2- 172.21.15.164:0/1881151653 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f9f7c0672a0 0x7f9f7c069760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:05.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.541+0000 7f9f98ff9640 1 -- 172.21.15.164:0/1881151653 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4943+0+0 (secure 0 0 0) 0x7f9f84085210 con 0x7f9f9c0b56f0 2024-03-23T14:58:05.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.541+0000 7f9f9b7fe640 1 --2- 172.21.15.164:0/1881151653 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f9f7c0672a0 0x7f9f7c069760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:05.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.545+0000 7f9f98ff9640 1 --2- 172.21.15.164:0/1881151653 >> [v2:172.21.15.173:6800/3676483185,v1:172.21.15.173:6801/3676483185] conn(0x7f9f7c06d160 0x7f9f7c06f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:05.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.545+0000 7f9f98ff9640 1 -- 172.21.15.164:0/1881151653 --> [v2:172.21.15.173:6800/3676483185,v1:172.21.15.173:6801/3676483185] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9f7c06fc30 con 0x7f9f7c06d160 2024-03-23T14:58:05.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.545+0000 7f9f98ff9640 1 -- 172.21.15.164:0/1881151653 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f9f84085660 con 0x7f9f9c0b56f0 2024-03-23T14:58:05.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.545+0000 7f9f9b7fe640 1 --2- 172.21.15.164:0/1881151653 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f9f7c0672a0 0x7f9f7c069760 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c006fd0 tx=0x7f9f8c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:58:05.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.545+0000 7f9f9bfff640 1 --2- 172.21.15.164:0/1881151653 >> [v2:172.21.15.173:6800/3676483185,v1:172.21.15.173:6801/3676483185] conn(0x7f9f7c06d160 0x7f9f7c06f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:05.552 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.545+0000 7f9f9bfff640 1 --2- 172.21.15.164:0/1881151653 >> [v2:172.21.15.173:6800/3676483185,v1:172.21.15.173:6801/3676483185] conn(0x7f9f7c06d160 0x7f9f7c06f580 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:58:05.552 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.545+0000 7f9f98ff9640 1 -- 172.21.15.164:0/1881151653 <== osd.5 v2:172.21.15.173:6800/3676483185 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9f7c06fc30 con 0x7f9f7c06d160 2024-03-23T14:58:05.563 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.557+0000 7f9fa1ba2640 1 -- 172.21.15.164:0/1881151653 --> [v2:172.21.15.173:6800/3676483185,v1:172.21.15.173:6801/3676483185] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f9f9c0b5ad0 con 0x7f9f7c06d160 2024-03-23T14:58:05.563 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.561+0000 7f9f98ff9640 1 -- 172.21.15.164:0/1881151653 <== osd.5 v2:172.21.15.173:6800/3676483185 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f9f94009510 con 0x7f9f7c06d160 2024-03-23T14:58:05.564 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-03-23T14:58:05.564 INFO:teuthology.orchestra.run.smithi164.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-23T14:58:05.564 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-03-23T14:58:05.565 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.561+0000 7f9fa1ba2640 1 -- 172.21.15.164:0/1881151653 >> [v2:172.21.15.173:6800/3676483185,v1:172.21.15.173:6801/3676483185] conn(0x7f9f7c06d160 msgr2=0x7f9f7c06f580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:05.565 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.561+0000 7f9fa1ba2640 1 --2- 172.21.15.164:0/1881151653 >> [v2:172.21.15.173:6800/3676483185,v1:172.21.15.173:6801/3676483185] conn(0x7f9f7c06d160 0x7f9f7c06f580 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:05.565 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.561+0000 7f9fa1ba2640 1 -- 172.21.15.164:0/1881151653 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f9f7c0672a0 msgr2=0x7f9f7c069760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:05.565 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.561+0000 7f9fa1ba2640 1 --2- 172.21.15.164:0/1881151653 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f9f7c0672a0 0x7f9f7c069760 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c006fd0 tx=0x7f9f8c008040 comp rx=0 tx=0).stop 2024-03-23T14:58:05.566 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.561+0000 7f9fa1ba2640 1 -- 172.21.15.164:0/1881151653 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9f9c0b56f0 msgr2=0x7f9f9c143390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:05.566 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.561+0000 7f9fa1ba2640 1 --2- 172.21.15.164:0/1881151653 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9f9c0b56f0 0x7f9f9c143390 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f9f8400c5f0 tx=0x7f9f8400cab0 comp rx=0 tx=0).stop 2024-03-23T14:58:05.566 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.561+0000 7f9f9bfff640 1 -- 172.21.15.164:0/1881151653 reap_dead start 2024-03-23T14:58:05.566 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.561+0000 7f9fa1ba2640 1 -- 172.21.15.164:0/1881151653 shutdown_connections 2024-03-23T14:58:05.566 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.561+0000 7f9fa1ba2640 1 -- 172.21.15.164:0/1881151653 >> 172.21.15.164:0/1881151653 conn(0x7f9f9c02c3c0 msgr2=0x7f9f9c0bb670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:58:05.566 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.561+0000 7f9fa1ba2640 1 -- 172.21.15.164:0/1881151653 shutdown_connections 2024-03-23T14:58:05.566 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.561+0000 7f9fa1ba2640 1 -- 172.21.15.164:0/1881151653 wait complete. 2024-03-23T14:58:05.579 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-03-23T14:58:05.684 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.681+0000 7fc5e0c53640 1 Processor -- start 2024-03-23T14:58:05.684 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.681+0000 7fc5e0c53640 1 -- start start 2024-03-23T14:58:05.685 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.681+0000 7fc5e0c53640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fc5dc0b55c0 0x7fc5dc14a940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:05.685 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.681+0000 7fc5e0c53640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fc5dc14ae80 0x7fc5dc141010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:05.685 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.681+0000 7fc5e0c53640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5dc0ba040 con 0x7fc5dc141550 2024-03-23T14:58:05.686 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.681+0000 7fc5e0c53640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fc5dc0b6810 con 0x7fc5dc14ae80 2024-03-23T14:58:05.686 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.681+0000 7fc5e0c53640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 -- 0x7fc5dc0b65a0 con 0x7fc5dc0b55c0 2024-03-23T14:58:05.686 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.681+0000 7fc5d1d9b640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fc5dc14ae80 0x7fc5dc141010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:05.686 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.681+0000 7fc5dad9d640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fc5dc141550 0x7fc5dc0b7640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:37872/0 (socket says 172.21.15.164:37872) 2024-03-23T14:58:05.686 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.681+0000 7fc5dad9d640 1 -- 172.21.15.164:0/490942682 learned_addr learned my addr 172.21.15.164:0/490942682 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-03-23T14:58:05.686 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.681+0000 7fc5d1d9b640 1 -- 172.21.15.164:0/490942682 >> v2:172.21.15.173:3300/0 conn(0x7fc5dc0b55c0 msgr2=0x7fc5dc14a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:58:05.686 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.681+0000 7fc5d1d9b640 1 --2- 172.21.15.164:0/490942682 >> v2:172.21.15.173:3300/0 conn(0x7fc5dc0b55c0 0x7fc5dc14a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:05.686 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.681+0000 7fc5d1d9b640 1 -- 172.21.15.164:0/490942682 >> v1:172.21.15.164:6789/0 conn(0x7fc5dc141550 legacy=0x7fc5dc0b7640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:05.686 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.681+0000 7fc5d9d9b640 1 -- 172.21.15.164:0/490942682 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1237712208 0 0) 0x7fc5dc0ba040 con 0x7fc5dc141550 2024-03-23T14:58:05.687 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.681+0000 7fc5d1d9b640 1 -- 172.21.15.164:0/490942682 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5dc14b310 con 0x7fc5dc14ae80 2024-03-23T14:58:05.688 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.681+0000 7fc5d1d9b640 1 --2- 172.21.15.164:0/490942682 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc5dc14ae80 0x7fc5dc141010 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fc5dc0bdd60 tx=0x7fc5c401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94319fc03b1a12b3 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:58:05.688 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.681+0000 7fc5d9d9b640 1 -- 172.21.15.164:0/490942682 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c40ab600 con 0x7fc5dc14ae80 2024-03-23T14:58:05.688 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.681+0000 7fc5d9d9b640 1 -- 172.21.15.164:0/490942682 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5c40a7050 con 0x7fc5dc14ae80 2024-03-23T14:58:05.688 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.681+0000 7fc5d9d9b640 1 -- 172.21.15.164:0/490942682 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c40ab980 con 0x7fc5dc14ae80 2024-03-23T14:58:05.688 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.685+0000 7fc5e0c53640 1 -- 172.21.15.164:0/490942682 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc5dc14ae80 msgr2=0x7fc5dc141010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:05.688 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.685+0000 7fc5e0c53640 1 --2- 172.21.15.164:0/490942682 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc5dc14ae80 0x7fc5dc141010 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fc5dc0bdd60 tx=0x7fc5c401c480 comp rx=0 tx=0).stop 2024-03-23T14:58:05.688 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.685+0000 7fc5e0c53640 1 -- 172.21.15.164:0/490942682 shutdown_connections 2024-03-23T14:58:05.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.685+0000 7fc5e0c53640 1 --2- 172.21.15.164:0/490942682 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc5dc14ae80 0x7fc5dc141010 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:05.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.685+0000 7fc5e0c53640 1 --2- 172.21.15.164:0/490942682 >> v2:172.21.15.173:3300/0 conn(0x7fc5dc0b55c0 0x7fc5dc14a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:05.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.685+0000 7fc5e0c53640 1 -- 172.21.15.164:0/490942682 >> 172.21.15.164:0/490942682 conn(0x7fc5dc072cd0 msgr2=0x7fc5dc0730e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:58:05.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.685+0000 7fc5e0c53640 1 -- 172.21.15.164:0/490942682 shutdown_connections 2024-03-23T14:58:05.689 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.685+0000 7fc5e0c53640 1 -- 172.21.15.164:0/490942682 wait complete. 2024-03-23T14:58:05.691 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.685+0000 7fc5e0c53640 1 Processor -- start 2024-03-23T14:58:05.692 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.685+0000 7fc5e0c53640 1 -- start start 2024-03-23T14:58:05.692 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.685+0000 7fc5e0c53640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc5dc0b55c0 0x7fc5dc0bb790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:05.692 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.685+0000 7fc5e0c53640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc5dc141550 0x7fc5dc1e2730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:05.692 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.685+0000 7fc5e0c53640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc5dc14ae80 0x7fc5dc136ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:05.692 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.685+0000 7fc5e0c53640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc5dc0bcab0 con 0x7fc5dc0b55c0 2024-03-23T14:58:05.692 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.685+0000 7fc5e0c53640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fc5dc0bbe00 con 0x7fc5dc141550 2024-03-23T14:58:05.692 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.685+0000 7fc5e0c53640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fc5dc13a640 con 0x7fc5dc14ae80 2024-03-23T14:58:05.692 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.685+0000 7fc5da59c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc5dc0b55c0 0x7fc5dc0bb790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:05.692 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.685+0000 7fc5da59c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc5dc0b55c0 0x7fc5dc0bb790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:49010/0 (socket says 172.21.15.164:49010) 2024-03-23T14:58:05.693 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.685+0000 7fc5da59c640 1 -- 172.21.15.164:0/4033558005 learned_addr learned my addr 172.21.15.164:0/4033558005 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:58:05.693 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.685+0000 7fc5dad9d640 1 --2- 172.21.15.164:0/4033558005 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc5dc14ae80 0x7fc5dc136ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:05.693 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.685+0000 7fc5d1d9b640 1 --2- 172.21.15.164:0/4033558005 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc5dc141550 0x7fc5dc1e2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:05.693 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.685+0000 7fc5da59c640 1 -- 172.21.15.164:0/4033558005 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc5dc14ae80 msgr2=0x7fc5dc136ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:05.693 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.685+0000 7fc5da59c640 1 --2- 172.21.15.164:0/4033558005 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc5dc14ae80 0x7fc5dc136ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:05.693 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.685+0000 7fc5da59c640 1 -- 172.21.15.164:0/4033558005 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc5dc141550 msgr2=0x7fc5dc1e2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:05.693 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.685+0000 7fc5da59c640 1 --2- 172.21.15.164:0/4033558005 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc5dc141550 0x7fc5dc1e2730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:05.693 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.685+0000 7fc5da59c640 1 -- 172.21.15.164:0/4033558005 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5c4009420 con 0x7fc5dc0b55c0 2024-03-23T14:58:05.693 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.685+0000 7fc5dad9d640 1 --2- 172.21.15.164:0/4033558005 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc5dc14ae80 0x7fc5dc136ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T14:58:05.694 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.685+0000 7fc5d1d9b640 1 --2- 172.21.15.164:0/4033558005 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc5dc141550 0x7fc5dc1e2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T14:58:05.694 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.685+0000 7fc5da59c640 1 --2- 172.21.15.164:0/4033558005 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc5dc0b55c0 0x7fc5dc0bb790 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc00b580 tx=0x7fc5cc00ba40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:58:05.694 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.685+0000 7fc5d37fe640 1 -- 172.21.15.164:0/4033558005 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5cc010870 con 0x7fc5dc0b55c0 2024-03-23T14:58:05.694 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.685+0000 7fc5d37fe640 1 -- 172.21.15.164:0/4033558005 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5cc0109f0 con 0x7fc5dc0b55c0 2024-03-23T14:58:05.694 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.685+0000 7fc5e0c53640 1 -- 172.21.15.164:0/4033558005 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5dc13d5c0 con 0x7fc5dc0b55c0 2024-03-23T14:58:05.694 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.685+0000 7fc5d37fe640 1 -- 172.21.15.164:0/4033558005 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5cc010da0 con 0x7fc5dc0b55c0 2024-03-23T14:58:05.694 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.685+0000 7fc5e0c53640 1 -- 172.21.15.164:0/4033558005 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5dc0babc0 con 0x7fc5dc0b55c0 2024-03-23T14:58:05.694 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.685+0000 7fc5e0c53640 1 -- 172.21.15.164:0/4033558005 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5dc13d7b0 con 0x7fc5dc0b55c0 2024-03-23T14:58:05.695 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.689+0000 7fc5d37fe640 1 -- 172.21.15.164:0/4033558005 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7fc5cc00f3c0 con 0x7fc5dc0b55c0 2024-03-23T14:58:05.695 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.689+0000 7fc5d37fe640 1 --2- 172.21.15.164:0/4033558005 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fc5bc0672a0 0x7fc5bc069760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:05.695 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.689+0000 7fc5d37fe640 1 -- 172.21.15.164:0/4033558005 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4943+0+0 (secure 0 0 0) 0x7fc5cc084cd0 con 0x7fc5dc0b55c0 2024-03-23T14:58:05.696 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.689+0000 7fc5d1d9b640 1 --2- 172.21.15.164:0/4033558005 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fc5bc0672a0 0x7fc5bc069760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:05.696 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.689+0000 7fc5d1d9b640 1 --2- 172.21.15.164:0/4033558005 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fc5bc0672a0 0x7fc5bc069760 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc5c40ad000 tx=0x7fc5c40a6000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:58:05.696 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.693+0000 7fc5d37fe640 1 -- 172.21.15.164:0/4033558005 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7fc5cc051b90 con 0x7fc5dc0b55c0 2024-03-23T14:58:05.853 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.849+0000 7fc5e0c53640 1 -- 172.21.15.164:0/4033558005 --> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7fc5dc0690a0 con 0x7fc5bc0672a0 2024-03-23T14:58:05.854 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.849+0000 7fc5d37fe640 1 -- 172.21.15.164:0/4033558005 <== mgr.4107 v2:172.21.15.164:6832/2139087985 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fc5dc0690a0 con 0x7fc5bc0672a0 2024-03-23T14:58:05.854 INFO:teuthology.orchestra.run.smithi164.stderr:instructed osd(s) 5 to deep-scrub 2024-03-23T14:58:05.856 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.853+0000 7fc5e0c53640 1 -- 172.21.15.164:0/4033558005 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fc5bc0672a0 msgr2=0x7fc5bc069760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:05.856 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.853+0000 7fc5e0c53640 1 --2- 172.21.15.164:0/4033558005 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fc5bc0672a0 0x7fc5bc069760 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc5c40ad000 tx=0x7fc5c40a6000 comp rx=0 tx=0).stop 2024-03-23T14:58:05.856 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.853+0000 7fc5e0c53640 1 -- 172.21.15.164:0/4033558005 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc5dc0b55c0 msgr2=0x7fc5dc0bb790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:05.856 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.853+0000 7fc5e0c53640 1 --2- 172.21.15.164:0/4033558005 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc5dc0b55c0 0x7fc5dc0bb790 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc00b580 tx=0x7fc5cc00ba40 comp rx=0 tx=0).stop 2024-03-23T14:58:05.856 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.853+0000 7fc5e0c53640 1 -- 172.21.15.164:0/4033558005 shutdown_connections 2024-03-23T14:58:05.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.853+0000 7fc5e0c53640 1 --2- 172.21.15.164:0/4033558005 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc5dc14ae80 0x7fc5dc136ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:05.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.853+0000 7fc5e0c53640 1 --2- 172.21.15.164:0/4033558005 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fc5bc0672a0 0x7fc5bc069760 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:05.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.853+0000 7fc5e0c53640 1 --2- 172.21.15.164:0/4033558005 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc5dc141550 0x7fc5dc1e2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:05.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.853+0000 7fc5e0c53640 1 --2- 172.21.15.164:0/4033558005 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc5dc0b55c0 0x7fc5dc0bb790 secure :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc00b580 tx=0x7fc5cc00ba40 comp rx=0 tx=0).stop 2024-03-23T14:58:05.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.853+0000 7fc5e0c53640 1 -- 172.21.15.164:0/4033558005 >> 172.21.15.164:0/4033558005 conn(0x7fc5dc072cd0 msgr2=0x7fc5dc146800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:58:05.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.853+0000 7fc5e0c53640 1 -- 172.21.15.164:0/4033558005 shutdown_connections 2024-03-23T14:58:05.857 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.853+0000 7fc5e0c53640 1 -- 172.21.15.164:0/4033558005 wait complete. 2024-03-23T14:58:05.872 INFO:tasks.ceph:Scrubbing osd.6 2024-03-23T14:58:05.872 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-03-23T14:58:05.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.985+0000 7fa59d197640 1 Processor -- start 2024-03-23T14:58:05.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.985+0000 7fa59d197640 1 -- start start 2024-03-23T14:58:05.989 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.985+0000 7fa59d197640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa59809def0 0x7fa5980b45e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:05.990 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.985+0000 7fa59d197640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fa5980add50 0x7fa59806bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:05.990 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.985+0000 7fa59d197640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa59806d650 con 0x7fa5980b4b20 2024-03-23T14:58:05.990 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.985+0000 7fa59d197640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 -- 0x7fa5980af4a0 con 0x7fa5980add50 2024-03-23T14:58:05.990 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.985+0000 7fa59d197640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fa59809e2d0 con 0x7fa59809def0 2024-03-23T14:58:05.990 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.985+0000 7fa59659c640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fa5980add50 0x7fa59806bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:05.990 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.985+0000 7fa596d9d640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa59809def0 0x7fa5980b45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:05.990 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.985+0000 7fa596d9d640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa59809def0 0x7fa5980b45e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.164:60664/0 (socket says 172.21.15.164:60664) 2024-03-23T14:58:05.990 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.985+0000 7fa596d9d640 1 -- 172.21.15.164:0/3364355588 learned_addr learned my addr 172.21.15.164:0/3364355588 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:58:05.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.985+0000 7fa596d9d640 1 -- 172.21.15.164:0/3364355588 >> v2:172.21.15.173:3300/0 conn(0x7fa5980add50 msgr2=0x7fa59806bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:58:05.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.985+0000 7fa596d9d640 1 --2- 172.21.15.164:0/3364355588 >> v2:172.21.15.173:3300/0 conn(0x7fa5980add50 0x7fa59806bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:05.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.985+0000 7fa596d9d640 1 -- 172.21.15.164:0/3364355588 >> v1:172.21.15.173:6789/0 conn(0x7fa5980b4b20 legacy=0x7fa59806c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:05.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.985+0000 7fa596d9d640 1 -- 172.21.15.164:0/3364355588 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5980aeca0 con 0x7fa59809def0 2024-03-23T14:58:05.991 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.985+0000 7fa59659c640 1 --2- 172.21.15.164:0/3364355588 >> v2:172.21.15.173:3300/0 conn(0x7fa5980add50 0x7fa59806bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-23T14:58:05.996 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.993+0000 7fa596d9d640 1 --2- 172.21.15.164:0/3364355588 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa59809def0 0x7fa5980b45e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa58c009870 tx=0x7fa58c01c440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7995f1655fe07ccd server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:58:05.996 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.993+0000 7fa595d9b640 1 -- 172.21.15.164:0/3364355588 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa58c0ac600 con 0x7fa59809def0 2024-03-23T14:58:05.996 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.993+0000 7fa595d9b640 1 -- 172.21.15.164:0/3364355588 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa58c0a8050 con 0x7fa59809def0 2024-03-23T14:58:05.996 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.993+0000 7fa595d9b640 1 -- 172.21.15.164:0/3364355588 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa58c0ac980 con 0x7fa59809def0 2024-03-23T14:58:05.997 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.993+0000 7fa59d197640 1 -- 172.21.15.164:0/3364355588 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa59809def0 msgr2=0x7fa5980b45e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:05.997 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.993+0000 7fa59d197640 1 --2- 172.21.15.164:0/3364355588 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa59809def0 0x7fa5980b45e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa58c009870 tx=0x7fa58c01c440 comp rx=0 tx=0).stop 2024-03-23T14:58:05.997 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.993+0000 7fa59d197640 1 -- 172.21.15.164:0/3364355588 shutdown_connections 2024-03-23T14:58:05.997 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.993+0000 7fa59d197640 1 --2- 172.21.15.164:0/3364355588 >> v2:172.21.15.173:3300/0 conn(0x7fa5980add50 0x7fa59806bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:05.997 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.993+0000 7fa59d197640 1 --2- 172.21.15.164:0/3364355588 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa59809def0 0x7fa5980b45e0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:05.998 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.993+0000 7fa59d197640 1 -- 172.21.15.164:0/3364355588 >> 172.21.15.164:0/3364355588 conn(0x7fa5980b8ca0 msgr2=0x7fa5980b90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:58:05.998 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.993+0000 7fa59d197640 1 -- 172.21.15.164:0/3364355588 shutdown_connections 2024-03-23T14:58:05.998 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.993+0000 7fa59d197640 1 -- 172.21.15.164:0/3364355588 wait complete. 2024-03-23T14:58:05.998 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.993+0000 7fa59d197640 1 Processor -- start 2024-03-23T14:58:05.998 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.993+0000 7fa59d197640 1 -- start start 2024-03-23T14:58:06.000 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.993+0000 7fa59d197640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa5980add50 0x7fa598142720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:06.001 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.993+0000 7fa59d197640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa5980b4b20 0x7fa5981433b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:06.001 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.993+0000 7fa59d197640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa59813de60 0x7fa5981438f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:06.001 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.993+0000 7fa59d197640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa59813ca00 con 0x7fa59813de60 2024-03-23T14:58:06.001 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.993+0000 7fa59d197640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa59813d050 con 0x7fa5980b4b20 2024-03-23T14:58:06.001 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.993+0000 7fa59d197640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fa59813d330 con 0x7fa5980add50 2024-03-23T14:58:06.001 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.993+0000 7fa59759e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa59813de60 0x7fa5981438f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:06.001 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.993+0000 7fa59759e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa59813de60 0x7fa5981438f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:49026/0 (socket says 172.21.15.164:49026) 2024-03-23T14:58:06.001 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.993+0000 7fa59759e640 1 -- 172.21.15.164:0/1536139367 learned_addr learned my addr 172.21.15.164:0/1536139367 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:58:06.001 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.997+0000 7fa59659c640 1 --2- 172.21.15.164:0/1536139367 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa5980b4b20 0x7fa5981433b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:06.002 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.997+0000 7fa596d9d640 1 --2- 172.21.15.164:0/1536139367 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa5980add50 0x7fa598142720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:06.002 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.997+0000 7fa59759e640 1 -- 172.21.15.164:0/1536139367 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa5980add50 msgr2=0x7fa598142720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:06.002 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.997+0000 7fa59759e640 1 --2- 172.21.15.164:0/1536139367 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa5980add50 0x7fa598142720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:06.002 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.997+0000 7fa59759e640 1 -- 172.21.15.164:0/1536139367 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa5980b4b20 msgr2=0x7fa5981433b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:06.002 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.997+0000 7fa59759e640 1 --2- 172.21.15.164:0/1536139367 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa5980b4b20 0x7fa5981433b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:06.002 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.997+0000 7fa59759e640 1 -- 172.21.15.164:0/1536139367 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa588009420 con 0x7fa59813de60 2024-03-23T14:58:06.002 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.997+0000 7fa59759e640 1 --2- 172.21.15.164:0/1536139367 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa59813de60 0x7fa5981438f0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fa59000c730 tx=0x7fa59000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:58:06.002 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.997+0000 7fa57f7fe640 1 -- 172.21.15.164:0/1536139367 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa590003b80 con 0x7fa59813de60 2024-03-23T14:58:06.002 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.997+0000 7fa57f7fe640 1 -- 172.21.15.164:0/1536139367 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa590003ce0 con 0x7fa59813de60 2024-03-23T14:58:06.003 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.997+0000 7fa59d197640 1 -- 172.21.15.164:0/1536139367 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa58c0094d0 con 0x7fa59813de60 2024-03-23T14:58:06.003 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.997+0000 7fa57f7fe640 1 -- 172.21.15.164:0/1536139367 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa59000aa30 con 0x7fa59813de60 2024-03-23T14:58:06.003 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.997+0000 7fa59d197640 1 -- 172.21.15.164:0/1536139367 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa59813c720 con 0x7fa59813de60 2024-03-23T14:58:06.003 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.997+0000 7fa59d197640 1 -- 172.21.15.164:0/1536139367 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa5980b34d0 con 0x7fa59813de60 2024-03-23T14:58:06.004 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.997+0000 7fa57f7fe640 1 -- 172.21.15.164:0/1536139367 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7fa59000ab90 con 0x7fa59813de60 2024-03-23T14:58:06.004 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.997+0000 7fa57f7fe640 1 --2- 172.21.15.164:0/1536139367 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fa5840672a0 0x7fa584069760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:06.004 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.997+0000 7fa57f7fe640 1 -- 172.21.15.164:0/1536139367 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4943+0+0 (secure 0 0 0) 0x7fa590085550 con 0x7fa59813de60 2024-03-23T14:58:06.004 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.997+0000 7fa596d9d640 1 --2- 172.21.15.164:0/1536139367 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fa5840672a0 0x7fa584069760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:06.004 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.997+0000 7fa57f7fe640 1 --2- 172.21.15.164:0/1536139367 >> [v2:172.21.15.173:6808/3631684087,v1:172.21.15.173:6809/3631684087] conn(0x7fa58406d160 0x7fa58406f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:06.005 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.997+0000 7fa57f7fe640 1 -- 172.21.15.164:0/1536139367 --> [v2:172.21.15.173:6808/3631684087,v1:172.21.15.173:6809/3631684087] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa58406fc30 con 0x7fa58406d160 2024-03-23T14:58:06.005 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.997+0000 7fa57f7fe640 1 -- 172.21.15.164:0/1536139367 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa5900859a0 con 0x7fa59813de60 2024-03-23T14:58:06.005 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.997+0000 7fa59659c640 1 --2- 172.21.15.164:0/1536139367 >> [v2:172.21.15.173:6808/3631684087,v1:172.21.15.173:6809/3631684087] conn(0x7fa58406d160 0x7fa58406f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:06.005 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.997+0000 7fa596d9d640 1 --2- 172.21.15.164:0/1536139367 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fa5840672a0 0x7fa584069760 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fa58c0ae000 tx=0x7fa58c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:58:06.005 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:05.997+0000 7fa59659c640 1 --2- 172.21.15.164:0/1536139367 >> [v2:172.21.15.173:6808/3631684087,v1:172.21.15.173:6809/3631684087] conn(0x7fa58406d160 0x7fa58406f580 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:58:06.005 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.001+0000 7fa57f7fe640 1 -- 172.21.15.164:0/1536139367 <== osd.6 v2:172.21.15.173:6808/3631684087 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa58406fc30 con 0x7fa58406d160 2024-03-23T14:58:06.017 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.013+0000 7fa59d197640 1 -- 172.21.15.164:0/1536139367 --> [v2:172.21.15.173:6808/3631684087,v1:172.21.15.173:6809/3631684087] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fa59813ce90 con 0x7fa58406d160 2024-03-23T14:58:06.018 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.013+0000 7fa57f7fe640 1 -- 172.21.15.164:0/1536139367 <== osd.6 v2:172.21.15.173:6808/3631684087 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fa59813ce90 con 0x7fa58406d160 2024-03-23T14:58:06.018 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-03-23T14:58:06.018 INFO:teuthology.orchestra.run.smithi164.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-23T14:58:06.019 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-03-23T14:58:06.019 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.013+0000 7fa59d197640 1 -- 172.21.15.164:0/1536139367 >> [v2:172.21.15.173:6808/3631684087,v1:172.21.15.173:6809/3631684087] conn(0x7fa58406d160 msgr2=0x7fa58406f580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:06.020 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.013+0000 7fa59d197640 1 --2- 172.21.15.164:0/1536139367 >> [v2:172.21.15.173:6808/3631684087,v1:172.21.15.173:6809/3631684087] conn(0x7fa58406d160 0x7fa58406f580 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:06.020 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.013+0000 7fa59d197640 1 -- 172.21.15.164:0/1536139367 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fa5840672a0 msgr2=0x7fa584069760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:06.020 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.013+0000 7fa59d197640 1 --2- 172.21.15.164:0/1536139367 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fa5840672a0 0x7fa584069760 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fa58c0ae000 tx=0x7fa58c008040 comp rx=0 tx=0).stop 2024-03-23T14:58:06.020 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.013+0000 7fa59d197640 1 -- 172.21.15.164:0/1536139367 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa59813de60 msgr2=0x7fa5981438f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:06.020 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.013+0000 7fa59d197640 1 --2- 172.21.15.164:0/1536139367 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa59813de60 0x7fa5981438f0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fa59000c730 tx=0x7fa59000cbf0 comp rx=0 tx=0).stop 2024-03-23T14:58:06.020 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.013+0000 7fa59759e640 1 -- 172.21.15.164:0/1536139367 reap_dead start 2024-03-23T14:58:06.020 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.013+0000 7fa59d197640 1 -- 172.21.15.164:0/1536139367 shutdown_connections 2024-03-23T14:58:06.020 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.013+0000 7fa59d197640 1 -- 172.21.15.164:0/1536139367 >> 172.21.15.164:0/1536139367 conn(0x7fa5980b8ca0 msgr2=0x7fa59813e7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:58:06.020 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.013+0000 7fa59d197640 1 -- 172.21.15.164:0/1536139367 shutdown_connections 2024-03-23T14:58:06.021 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.013+0000 7fa59d197640 1 -- 172.21.15.164:0/1536139367 wait complete. 2024-03-23T14:58:06.033 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-03-23T14:58:06.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.141+0000 7f8297e40640 1 Processor -- start 2024-03-23T14:58:06.144 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.141+0000 7f8297e40640 1 -- start start 2024-03-23T14:58:06.147 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.141+0000 7f8297e40640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f82900b0230 0x7f82900b34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:06.147 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.141+0000 7f8297e40640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82900b51d0 con 0x7f829014db70 2024-03-23T14:58:06.147 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.141+0000 7f8297e40640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82900b0610 con 0x7f8290148cf0 2024-03-23T14:58:06.147 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.141+0000 7f8297e40640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f82900b1500 con 0x7f82900b0230 2024-03-23T14:58:06.147 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.141+0000 7f82963dd640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f829014db70 0x7f829014ff60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:37884/0 (socket says 172.21.15.164:37884) 2024-03-23T14:58:06.147 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.141+0000 7f82963dd640 1 -- 172.21.15.164:0/93312833 learned_addr learned my addr 172.21.15.164:0/93312833 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-03-23T14:58:06.147 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.141+0000 7f8295bdc640 1 --2- 172.21.15.164:0/93312833 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f82900b0230 0x7f82900b34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:06.148 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.141+0000 7f8294bda640 1 -- 172.21.15.164:0/93312833 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2474020552 0 0) 0x7f82900b51d0 con 0x7f829014db70 2024-03-23T14:58:06.148 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.141+0000 7f8294bda640 1 -- 172.21.15.164:0/93312833 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f827c003530 con 0x7f829014db70 2024-03-23T14:58:06.148 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.141+0000 7f8294bda640 1 -- 172.21.15.164:0/93312833 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1601219349 0 0) 0x7f827c003530 con 0x7f829014db70 2024-03-23T14:58:06.148 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.141+0000 7f8294bda640 1 -- 172.21.15.164:0/93312833 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f82900b0230 msgr2=0x7f82900b34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:58:06.148 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.141+0000 7f8294bda640 1 --2- 172.21.15.164:0/93312833 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f82900b0230 0x7f82900b34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:06.148 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.141+0000 7f8294bda640 1 -- 172.21.15.164:0/93312833 >> v1:172.21.15.173:6789/0 conn(0x7f8290148cf0 legacy=0x7f8290149150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:06.148 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.141+0000 7f8294bda640 1 -- 172.21.15.164:0/93312833 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f829006d250 con 0x7f829014db70 2024-03-23T14:58:06.148 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.141+0000 7f8294bda640 1 -- 172.21.15.164:0/93312833 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2526390548 0 0) 0x7f8288001a00 con 0x7f829014db70 2024-03-23T14:58:06.148 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.141+0000 7f8294bda640 1 -- 172.21.15.164:0/93312833 >> v1:172.21.15.164:6789/0 conn(0x7f829014db70 legacy=0x7f829014ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:06.148 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.141+0000 7f8294bda640 1 --2- 172.21.15.164:0/93312833 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f827c0039b0 0x7f827c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:06.149 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.141+0000 7f8294bda640 1 --2- 172.21.15.164:0/93312833 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f827c0047d0 0x7f827c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:06.149 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.141+0000 7f8294bda640 1 --2- 172.21.15.164:0/93312833 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f82900b0230 0x7f827c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:06.149 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.141+0000 7f8294bda640 1 -- 172.21.15.164:0/93312833 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f82900b1500 con 0x7f827c0039b0 2024-03-23T14:58:06.149 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.141+0000 7f8294bda640 1 -- 172.21.15.164:0/93312833 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f827c00bac0 con 0x7f827c0047d0 2024-03-23T14:58:06.149 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.141+0000 7f8294bda640 1 -- 172.21.15.164:0/93312833 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f827c00bc80 con 0x7f82900b0230 2024-03-23T14:58:06.149 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.141+0000 7f8295bdc640 1 --2- 172.21.15.164:0/93312833 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f827c0039b0 0x7f827c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:06.149 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.141+0000 7f82963dd640 1 --2- 172.21.15.164:0/93312833 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f82900b0230 0x7f827c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:06.149 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.141+0000 7f82953db640 1 --2- 172.21.15.164:0/93312833 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f827c0047d0 0x7f827c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:06.149 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.141+0000 7f8295bdc640 1 -- 172.21.15.164:0/93312833 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f82900b0230 msgr2=0x7f827c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:58:06.149 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.141+0000 7f8295bdc640 1 --2- 172.21.15.164:0/93312833 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f82900b0230 0x7f827c0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:06.150 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.141+0000 7f8295bdc640 1 -- 172.21.15.164:0/93312833 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f827c0047d0 msgr2=0x7f827c004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:58:06.150 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.141+0000 7f8295bdc640 1 --2- 172.21.15.164:0/93312833 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f827c0047d0 0x7f827c004c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:06.150 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.141+0000 7f8295bdc640 1 -- 172.21.15.164:0/93312833 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f827c003530 con 0x7f827c0039b0 2024-03-23T14:58:06.150 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.141+0000 7f82963dd640 1 --2- 172.21.15.164:0/93312833 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f82900b0230 0x7f827c0051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T14:58:06.153 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.145+0000 7f8295bdc640 1 --2- 172.21.15.164:0/93312833 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f827c0039b0 0x7f827c003dd0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f828c009ae0 tx=0x7f828c01ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36d6d7144a2a64fa server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:58:06.153 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.145+0000 7f8294bda640 1 -- 172.21.15.164:0/93312833 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f828c0a5610 con 0x7f827c0039b0 2024-03-23T14:58:06.153 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.145+0000 7f8294bda640 1 -- 172.21.15.164:0/93312833 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f828c0a5770 con 0x7f827c0039b0 2024-03-23T14:58:06.154 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.145+0000 7f8294bda640 1 -- 172.21.15.164:0/93312833 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f828c0a5b20 con 0x7f827c0039b0 2024-03-23T14:58:06.154 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.145+0000 7f8297e40640 1 -- 172.21.15.164:0/93312833 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f827c0039b0 msgr2=0x7f827c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:06.154 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.145+0000 7f8297e40640 1 --2- 172.21.15.164:0/93312833 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f827c0039b0 0x7f827c003dd0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f828c009ae0 tx=0x7f828c01ca40 comp rx=0 tx=0).stop 2024-03-23T14:58:06.154 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.145+0000 7f82963dd640 1 -- 172.21.15.164:0/93312833 reap_dead start 2024-03-23T14:58:06.154 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.145+0000 7f8297e40640 1 -- 172.21.15.164:0/93312833 shutdown_connections 2024-03-23T14:58:06.154 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.145+0000 7f8297e40640 1 -- 172.21.15.164:0/93312833 >> 172.21.15.164:0/93312833 conn(0x7f82900b8ca0 msgr2=0x7f82900b90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:58:06.154 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.145+0000 7f8297e40640 1 -- 172.21.15.164:0/93312833 shutdown_connections 2024-03-23T14:58:06.154 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.145+0000 7f8297e40640 1 -- 172.21.15.164:0/93312833 wait complete. 2024-03-23T14:58:06.154 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.145+0000 7f8297e40640 1 Processor -- start 2024-03-23T14:58:06.155 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.145+0000 7f8297e40640 1 -- start start 2024-03-23T14:58:06.155 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.145+0000 7f8297e40640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f827c0039b0 0x7f8290147760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:06.155 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.145+0000 7f8297e40640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8290142dd0 0x7f82901df500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:06.155 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.145+0000 7f8297e40640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f829006f1b0 0x7f829013e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:06.155 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.145+0000 7f8297e40640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f82900725b0 con 0x7f827c0039b0 2024-03-23T14:58:06.155 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.145+0000 7f8297e40640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f829009da40 con 0x7f8290142dd0 2024-03-23T14:58:06.155 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.145+0000 7f8297e40640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f82900704c0 con 0x7f829006f1b0 2024-03-23T14:58:06.155 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.145+0000 7f8295bdc640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f827c0039b0 0x7f8290147760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:06.155 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.145+0000 7f8295bdc640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f827c0039b0 0x7f8290147760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:49038/0 (socket says 172.21.15.164:49038) 2024-03-23T14:58:06.156 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.145+0000 7f8295bdc640 1 -- 172.21.15.164:0/1573430541 learned_addr learned my addr 172.21.15.164:0/1573430541 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:58:06.156 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.145+0000 7f82963dd640 1 --2- 172.21.15.164:0/1573430541 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f829006f1b0 0x7f829013e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:06.156 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.145+0000 7f82953db640 1 --2- 172.21.15.164:0/1573430541 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8290142dd0 0x7f82901df500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:06.156 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.145+0000 7f8295bdc640 1 -- 172.21.15.164:0/1573430541 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f829006f1b0 msgr2=0x7f829013e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:06.156 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.145+0000 7f8295bdc640 1 --2- 172.21.15.164:0/1573430541 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f829006f1b0 0x7f829013e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:06.156 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.145+0000 7f8295bdc640 1 -- 172.21.15.164:0/1573430541 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8290142dd0 msgr2=0x7f82901df500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:06.156 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.145+0000 7f8295bdc640 1 --2- 172.21.15.164:0/1573430541 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8290142dd0 0x7f82901df500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:06.156 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.145+0000 7f8295bdc640 1 -- 172.21.15.164:0/1573430541 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82900b0f70 con 0x7f827c0039b0 2024-03-23T14:58:06.156 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.145+0000 7f8295bdc640 1 --2- 172.21.15.164:0/1573430541 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f827c0039b0 0x7f8290147760 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f828c01cf20 tx=0x7f827c005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:58:06.156 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.149+0000 7f82867fc640 1 -- 172.21.15.164:0/1573430541 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f828c0a5610 con 0x7f827c0039b0 2024-03-23T14:58:06.157 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.149+0000 7f82867fc640 1 -- 172.21.15.164:0/1573430541 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f828c004020 con 0x7f827c0039b0 2024-03-23T14:58:06.157 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.149+0000 7f8297e40640 1 -- 172.21.15.164:0/1573430541 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f829009d7a0 con 0x7f827c0039b0 2024-03-23T14:58:06.157 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.149+0000 7f82867fc640 1 -- 172.21.15.164:0/1573430541 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f828c0043b0 con 0x7f827c0039b0 2024-03-23T14:58:06.157 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.149+0000 7f8297e40640 1 -- 172.21.15.164:0/1573430541 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82901480d0 con 0x7f827c0039b0 2024-03-23T14:58:06.157 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.149+0000 7f8297e40640 1 -- 172.21.15.164:0/1573430541 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82900afba0 con 0x7f827c0039b0 2024-03-23T14:58:06.157 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.149+0000 7f82867fc640 1 -- 172.21.15.164:0/1573430541 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7f828c0b4040 con 0x7f827c0039b0 2024-03-23T14:58:06.157 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.149+0000 7f82867fc640 1 --2- 172.21.15.164:0/1573430541 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f82740672a0 0x7f8274069760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:06.157 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.149+0000 7f82867fc640 1 -- 172.21.15.164:0/1573430541 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4943+0+0 (secure 0 0 0) 0x7f828c11ee90 con 0x7f827c0039b0 2024-03-23T14:58:06.157 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.149+0000 7f82953db640 1 --2- 172.21.15.164:0/1573430541 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f82740672a0 0x7f8274069760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:06.158 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.149+0000 7f82953db640 1 --2- 172.21.15.164:0/1573430541 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f82740672a0 0x7f8274069760 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f82901431d0 tx=0x7f8280009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:58:06.158 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.153+0000 7f82867fc640 1 -- 172.21.15.164:0/1573430541 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7f828c0ebfa0 con 0x7f827c0039b0 2024-03-23T14:58:06.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.313+0000 7f8297e40640 1 -- 172.21.15.164:0/1573430541 --> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7f82900b0df0 con 0x7f82740672a0 2024-03-23T14:58:06.319 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.313+0000 7f82867fc640 1 -- 172.21.15.164:0/1573430541 <== mgr.4107 v2:172.21.15.164:6832/2139087985 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f828000d040 con 0x7f82740672a0 2024-03-23T14:58:06.319 INFO:teuthology.orchestra.run.smithi164.stderr:instructed osd(s) 6 to deep-scrub 2024-03-23T14:58:06.321 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.317+0000 7f8297e40640 1 -- 172.21.15.164:0/1573430541 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f82740672a0 msgr2=0x7f8274069760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:06.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.317+0000 7f8297e40640 1 --2- 172.21.15.164:0/1573430541 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f82740672a0 0x7f8274069760 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f82901431d0 tx=0x7f8280009040 comp rx=0 tx=0).stop 2024-03-23T14:58:06.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.317+0000 7f8297e40640 1 -- 172.21.15.164:0/1573430541 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f827c0039b0 msgr2=0x7f8290147760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:06.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.317+0000 7f8297e40640 1 --2- 172.21.15.164:0/1573430541 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f827c0039b0 0x7f8290147760 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f828c01cf20 tx=0x7f827c005800 comp rx=0 tx=0).stop 2024-03-23T14:58:06.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.317+0000 7f8297e40640 1 -- 172.21.15.164:0/1573430541 shutdown_connections 2024-03-23T14:58:06.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.317+0000 7f8297e40640 1 --2- 172.21.15.164:0/1573430541 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f829006f1b0 0x7f829013e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:06.322 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.317+0000 7f8297e40640 1 --2- 172.21.15.164:0/1573430541 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f82740672a0 0x7f8274069760 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:06.323 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.317+0000 7f8297e40640 1 --2- 172.21.15.164:0/1573430541 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8290142dd0 0x7f82901df500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:06.323 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.317+0000 7f8297e40640 1 --2- 172.21.15.164:0/1573430541 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f827c0039b0 0x7f8290147760 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:06.323 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.317+0000 7f8297e40640 1 -- 172.21.15.164:0/1573430541 >> 172.21.15.164:0/1573430541 conn(0x7f82900b8ca0 msgr2=0x7f8290149130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:58:06.323 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.317+0000 7f8297e40640 1 -- 172.21.15.164:0/1573430541 shutdown_connections 2024-03-23T14:58:06.323 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.317+0000 7f8297e40640 1 -- 172.21.15.164:0/1573430541 wait complete. 2024-03-23T14:58:06.338 INFO:tasks.ceph:Scrubbing osd.7 2024-03-23T14:58:06.338 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-03-23T14:58:06.459 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.453+0000 7fd04fef7640 1 Processor -- start 2024-03-23T14:58:06.460 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.453+0000 7fd04fef7640 1 -- start start 2024-03-23T14:58:06.460 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.457+0000 7fd04fef7640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd0480b9e60 0x7fd048147400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:06.461 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.457+0000 7fd04fef7640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fd0480b5ab0 0x7fd048141180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:06.461 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.457+0000 7fd04fef7640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0480b3e20 con 0x7fd0481416c0 2024-03-23T14:58:06.461 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.457+0000 7fd04fef7640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 -- 0x7fd0480b6070 con 0x7fd0480b5ab0 2024-03-23T14:58:06.461 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.457+0000 7fd04fef7640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fd0480b3b80 con 0x7fd0480b9e60 2024-03-23T14:58:06.461 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.457+0000 7fd04dc93640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd0480b9e60 0x7fd048147400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:06.462 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.457+0000 7fd04d492640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fd0480b5ab0 0x7fd048141180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:06.462 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.457+0000 7fd04dc93640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd0480b9e60 0x7fd048147400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.164:60708/0 (socket says 172.21.15.164:60708) 2024-03-23T14:58:06.462 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.457+0000 7fd04dc93640 1 -- 172.21.15.164:0/2008696151 learned_addr learned my addr 172.21.15.164:0/2008696151 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:58:06.462 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.457+0000 7fd04e494640 1 --1- 172.21.15.164:0/2008696151 >> v1:172.21.15.173:6789/0 conn(0x7fd0481416c0 0x7fd0480b75f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.173:6789/0 says I am v1:172.21.15.164:43672/0 (socket says 172.21.15.164:43672) 2024-03-23T14:58:06.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.457+0000 7fd04d492640 1 -- 172.21.15.164:0/2008696151 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd0480b9e60 msgr2=0x7fd048147400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:58:06.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.457+0000 7fd04d492640 1 --2- 172.21.15.164:0/2008696151 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd0480b9e60 0x7fd048147400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:06.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.457+0000 7fd04d492640 1 -- 172.21.15.164:0/2008696151 >> v1:172.21.15.173:6789/0 conn(0x7fd0481416c0 legacy=0x7fd0480b75f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:06.463 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.457+0000 7fd04d492640 1 -- 172.21.15.164:0/2008696151 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0480ba240 con 0x7fd0480b5ab0 2024-03-23T14:58:06.464 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.461+0000 7fd04d492640 1 --2- 172.21.15.164:0/2008696151 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd0480b5ab0 0x7fd048141180 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd0380097c0 tx=0x7fd03801c3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c305eb2d67115a8a server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:58:06.465 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.461+0000 7fd04cc91640 1 -- 172.21.15.164:0/2008696151 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0380ab600 con 0x7fd0480b5ab0 2024-03-23T14:58:06.465 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.461+0000 7fd04cc91640 1 -- 172.21.15.164:0/2008696151 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0380a7050 con 0x7fd0480b5ab0 2024-03-23T14:58:06.465 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.461+0000 7fd04cc91640 1 -- 172.21.15.164:0/2008696151 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0380ab980 con 0x7fd0480b5ab0 2024-03-23T14:58:06.466 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.461+0000 7fd04fef7640 1 -- 172.21.15.164:0/2008696151 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd0480b5ab0 msgr2=0x7fd048141180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:06.466 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.461+0000 7fd04fef7640 1 --2- 172.21.15.164:0/2008696151 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd0480b5ab0 0x7fd048141180 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd0380097c0 tx=0x7fd03801c3e0 comp rx=0 tx=0).stop 2024-03-23T14:58:06.467 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.461+0000 7fd04fef7640 1 -- 172.21.15.164:0/2008696151 shutdown_connections 2024-03-23T14:58:06.467 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.461+0000 7fd04fef7640 1 --2- 172.21.15.164:0/2008696151 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd0480b5ab0 0x7fd048141180 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:06.467 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.461+0000 7fd04fef7640 1 --2- 172.21.15.164:0/2008696151 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd0480b9e60 0x7fd048147400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:06.467 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.461+0000 7fd04fef7640 1 -- 172.21.15.164:0/2008696151 >> 172.21.15.164:0/2008696151 conn(0x7fd048072cd0 msgr2=0x7fd0480730e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:58:06.467 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.461+0000 7fd04fef7640 1 -- 172.21.15.164:0/2008696151 shutdown_connections 2024-03-23T14:58:06.467 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.461+0000 7fd04fef7640 1 -- 172.21.15.164:0/2008696151 wait complete. 2024-03-23T14:58:06.467 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.461+0000 7fd04fef7640 1 Processor -- start 2024-03-23T14:58:06.468 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.461+0000 7fd04fef7640 1 -- start start 2024-03-23T14:58:06.470 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.461+0000 7fd04fef7640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd0480b5ab0 0x7fd048136550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:06.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.461+0000 7fd04fef7640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd0480b9e60 0x7fd048137ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:06.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.461+0000 7fd04fef7640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd0481416c0 0x7fd0481381e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:06.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.461+0000 7fd04fef7640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd04813a6f0 con 0x7fd0481416c0 2024-03-23T14:58:06.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.461+0000 7fd04fef7640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd04813a4a0 con 0x7fd0480b5ab0 2024-03-23T14:58:06.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.461+0000 7fd04fef7640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fd048139cd0 con 0x7fd0480b9e60 2024-03-23T14:58:06.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.465+0000 7fd04e494640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd0481416c0 0x7fd0481381e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:06.471 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.465+0000 7fd04e494640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd0481416c0 0x7fd0481381e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:49050/0 (socket says 172.21.15.164:49050) 2024-03-23T14:58:06.472 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.465+0000 7fd04e494640 1 -- 172.21.15.164:0/1840538384 learned_addr learned my addr 172.21.15.164:0/1840538384 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:58:06.472 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.465+0000 7fd04d492640 1 --2- 172.21.15.164:0/1840538384 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd0480b9e60 0x7fd048137ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:06.472 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.465+0000 7fd04dc93640 1 --2- 172.21.15.164:0/1840538384 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd0480b5ab0 0x7fd048136550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:06.472 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.465+0000 7fd04e494640 1 -- 172.21.15.164:0/1840538384 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd0480b9e60 msgr2=0x7fd048137ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:06.472 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.465+0000 7fd04e494640 1 --2- 172.21.15.164:0/1840538384 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd0480b9e60 0x7fd048137ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:06.472 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.465+0000 7fd04e494640 1 -- 172.21.15.164:0/1840538384 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd0480b5ab0 msgr2=0x7fd048136550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:06.472 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.465+0000 7fd04e494640 1 --2- 172.21.15.164:0/1840538384 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd0480b5ab0 0x7fd048136550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:06.472 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.465+0000 7fd04e494640 1 -- 172.21.15.164:0/1840538384 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd038009420 con 0x7fd0481416c0 2024-03-23T14:58:06.473 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.465+0000 7fd04d492640 1 --2- 172.21.15.164:0/1840538384 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd0480b9e60 0x7fd048137ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T14:58:06.473 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.465+0000 7fd04dc93640 1 --2- 172.21.15.164:0/1840538384 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd0480b5ab0 0x7fd048136550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T14:58:06.473 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.465+0000 7fd04e494640 1 --2- 172.21.15.164:0/1840538384 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd0481416c0 0x7fd0481381e0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fd04400c3f0 tx=0x7fd04400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:58:06.473 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.465+0000 7fd03e7fc640 1 -- 172.21.15.164:0/1840538384 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0440074e0 con 0x7fd0481416c0 2024-03-23T14:58:06.473 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.465+0000 7fd03e7fc640 1 -- 172.21.15.164:0/1840538384 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd044007640 con 0x7fd0481416c0 2024-03-23T14:58:06.473 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.465+0000 7fd04fef7640 1 -- 172.21.15.164:0/1840538384 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0481496c0 con 0x7fd0481416c0 2024-03-23T14:58:06.473 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.465+0000 7fd03e7fc640 1 -- 172.21.15.164:0/1840538384 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd044004a10 con 0x7fd0481416c0 2024-03-23T14:58:06.473 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.465+0000 7fd04fef7640 1 -- 172.21.15.164:0/1840538384 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd048136df0 con 0x7fd0481416c0 2024-03-23T14:58:06.473 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.465+0000 7fd04fef7640 1 -- 172.21.15.164:0/1840538384 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd0480b4d00 con 0x7fd0481416c0 2024-03-23T14:58:06.474 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.465+0000 7fd03e7fc640 1 -- 172.21.15.164:0/1840538384 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7fd044004b70 con 0x7fd0481416c0 2024-03-23T14:58:06.474 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.465+0000 7fd03e7fc640 1 --2- 172.21.15.164:0/1840538384 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fd02c0672a0 0x7fd02c069760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:06.474 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.465+0000 7fd03e7fc640 1 -- 172.21.15.164:0/1840538384 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4943+0+0 (secure 0 0 0) 0x7fd044085030 con 0x7fd0481416c0 2024-03-23T14:58:06.474 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.465+0000 7fd04dc93640 1 --2- 172.21.15.164:0/1840538384 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fd02c0672a0 0x7fd02c069760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:06.474 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.465+0000 7fd03e7fc640 1 --2- 172.21.15.164:0/1840538384 >> [v2:172.21.15.173:6816/3261114948,v1:172.21.15.173:6817/3261114948] conn(0x7fd02c06d160 0x7fd02c06f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:06.474 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.465+0000 7fd03e7fc640 1 -- 172.21.15.164:0/1840538384 --> [v2:172.21.15.173:6816/3261114948,v1:172.21.15.173:6817/3261114948] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd02c06fc30 con 0x7fd02c06d160 2024-03-23T14:58:06.474 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.465+0000 7fd03e7fc640 1 -- 172.21.15.164:0/1840538384 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fd044085480 con 0x7fd0481416c0 2024-03-23T14:58:06.474 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.465+0000 7fd04dc93640 1 --2- 172.21.15.164:0/1840538384 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fd02c0672a0 0x7fd02c069760 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fd048137a40 tx=0x7fd04000b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:58:06.475 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.465+0000 7fd04d492640 1 --2- 172.21.15.164:0/1840538384 >> [v2:172.21.15.173:6816/3261114948,v1:172.21.15.173:6817/3261114948] conn(0x7fd02c06d160 0x7fd02c06f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:06.475 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.469+0000 7fd04d492640 1 --2- 172.21.15.164:0/1840538384 >> [v2:172.21.15.173:6816/3261114948,v1:172.21.15.173:6817/3261114948] conn(0x7fd02c06d160 0x7fd02c06f580 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:58:06.475 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.469+0000 7fd03e7fc640 1 -- 172.21.15.164:0/1840538384 <== osd.7 v2:172.21.15.173:6816/3261114948 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd02c06fc30 con 0x7fd02c06d160 2024-03-23T14:58:06.485 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.481+0000 7fd04fef7640 1 -- 172.21.15.164:0/1840538384 --> [v2:172.21.15.173:6816/3261114948,v1:172.21.15.173:6817/3261114948] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fd048139b30 con 0x7fd02c06d160 2024-03-23T14:58:06.486 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.481+0000 7fd03e7fc640 1 -- 172.21.15.164:0/1840538384 <== osd.7 v2:172.21.15.173:6816/3261114948 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fd048139b30 con 0x7fd02c06d160 2024-03-23T14:58:06.487 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-03-23T14:58:06.487 INFO:teuthology.orchestra.run.smithi164.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-23T14:58:06.487 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-03-23T14:58:06.488 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.481+0000 7fd04fef7640 1 -- 172.21.15.164:0/1840538384 >> [v2:172.21.15.173:6816/3261114948,v1:172.21.15.173:6817/3261114948] conn(0x7fd02c06d160 msgr2=0x7fd02c06f580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:06.488 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.481+0000 7fd04fef7640 1 --2- 172.21.15.164:0/1840538384 >> [v2:172.21.15.173:6816/3261114948,v1:172.21.15.173:6817/3261114948] conn(0x7fd02c06d160 0x7fd02c06f580 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:06.488 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.481+0000 7fd04fef7640 1 -- 172.21.15.164:0/1840538384 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fd02c0672a0 msgr2=0x7fd02c069760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:06.488 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.481+0000 7fd04fef7640 1 --2- 172.21.15.164:0/1840538384 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fd02c0672a0 0x7fd02c069760 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fd048137a40 tx=0x7fd04000b040 comp rx=0 tx=0).stop 2024-03-23T14:58:06.488 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.481+0000 7fd04fef7640 1 -- 172.21.15.164:0/1840538384 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd0481416c0 msgr2=0x7fd0481381e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:06.489 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.481+0000 7fd04fef7640 1 --2- 172.21.15.164:0/1840538384 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd0481416c0 0x7fd0481381e0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fd04400c3f0 tx=0x7fd04400c8b0 comp rx=0 tx=0).stop 2024-03-23T14:58:06.489 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.481+0000 7fd04e494640 1 -- 172.21.15.164:0/1840538384 reap_dead start 2024-03-23T14:58:06.489 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.481+0000 7fd04fef7640 1 -- 172.21.15.164:0/1840538384 shutdown_connections 2024-03-23T14:58:06.489 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.481+0000 7fd04fef7640 1 -- 172.21.15.164:0/1840538384 >> 172.21.15.164:0/1840538384 conn(0x7fd048072cd0 msgr2=0x7fd048133f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:58:06.489 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.481+0000 7fd04fef7640 1 -- 172.21.15.164:0/1840538384 shutdown_connections 2024-03-23T14:58:06.489 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.481+0000 7fd04fef7640 1 -- 172.21.15.164:0/1840538384 wait complete. 2024-03-23T14:58:06.503 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-03-23T14:58:06.608 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.605+0000 7fe5014c5640 1 Processor -- start 2024-03-23T14:58:06.609 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.605+0000 7fe5014c5640 1 -- start start 2024-03-23T14:58:06.610 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.605+0000 7fe5014c5640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fe4fc0b5790 0x7fe4fc141320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:06.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.605+0000 7fe5014c5640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe4fc0b5b70 0x7fe4fc151cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:06.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.605+0000 7fe5014c5640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4fc06ca90 con 0x7fe4fc0b82b0 2024-03-23T14:58:06.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.605+0000 7fe5014c5640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fe4fc0b7110 con 0x7fe4fc0b5790 2024-03-23T14:58:06.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.605+0000 7fe5014c5640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fe4fc06d220 con 0x7fe4fc0b5b70 2024-03-23T14:58:06.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.605+0000 7fe4faffd640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fe4fc0b5790 0x7fe4fc141320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:06.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.605+0000 7fe4faffd640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fe4fc0b5790 0x7fe4fc141320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:49056/0 (socket says 172.21.15.164:49056) 2024-03-23T14:58:06.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.605+0000 7fe4faffd640 1 -- 172.21.15.164:0/2554685863 learned_addr learned my addr 172.21.15.164:0/2554685863 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:58:06.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.605+0000 7fe4fb7fe640 1 --2- 172.21.15.164:0/2554685863 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe4fc0b5b70 0x7fe4fc151cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:06.611 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.605+0000 7fe4faffd640 1 -- 172.21.15.164:0/2554685863 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe4fc0b5b70 msgr2=0x7fe4fc151cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:58:06.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.605+0000 7fe4faffd640 1 --2- 172.21.15.164:0/2554685863 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe4fc0b5b70 0x7fe4fc151cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:06.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.605+0000 7fe4faffd640 1 -- 172.21.15.164:0/2554685863 >> v1:172.21.15.173:6789/0 conn(0x7fe4fc0b82b0 legacy=0x7fe4fc14acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:58:06.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.605+0000 7fe4faffd640 1 -- 172.21.15.164:0/2554685863 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4fc06c7b0 con 0x7fe4fc0b5790 2024-03-23T14:58:06.612 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.605+0000 7fe4fb7fe640 1 --2- 172.21.15.164:0/2554685863 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe4fc0b5b70 0x7fe4fc151cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T14:58:06.613 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.609+0000 7fe4faffd640 1 --2- 172.21.15.164:0/2554685863 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe4fc0b5790 0x7fe4fc141320 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec009870 tx=0x7fe4ec01c340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bd2dae310f61ede server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:58:06.613 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.609+0000 7fe4f9ffb640 1 -- 172.21.15.164:0/2554685863 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4ec0ab600 con 0x7fe4fc0b5790 2024-03-23T14:58:06.613 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.609+0000 7fe4f9ffb640 1 -- 172.21.15.164:0/2554685863 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4ec0a7050 con 0x7fe4fc0b5790 2024-03-23T14:58:06.613 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.609+0000 7fe4f9ffb640 1 -- 172.21.15.164:0/2554685863 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4ec0ab980 con 0x7fe4fc0b5790 2024-03-23T14:58:06.614 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.609+0000 7fe5014c5640 1 -- 172.21.15.164:0/2554685863 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe4fc0b5790 msgr2=0x7fe4fc141320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:06.614 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.609+0000 7fe5014c5640 1 --2- 172.21.15.164:0/2554685863 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe4fc0b5790 0x7fe4fc141320 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec009870 tx=0x7fe4ec01c340 comp rx=0 tx=0).stop 2024-03-23T14:58:06.614 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.609+0000 7fe5014c5640 1 -- 172.21.15.164:0/2554685863 shutdown_connections 2024-03-23T14:58:06.614 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.609+0000 7fe5014c5640 1 --2- 172.21.15.164:0/2554685863 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe4fc0b5b70 0x7fe4fc151cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:06.614 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.609+0000 7fe5014c5640 1 --2- 172.21.15.164:0/2554685863 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe4fc0b5790 0x7fe4fc141320 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:06.615 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.609+0000 7fe5014c5640 1 -- 172.21.15.164:0/2554685863 >> 172.21.15.164:0/2554685863 conn(0x7fe4fc0b7630 msgr2=0x7fe4fc0b7a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:58:06.615 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.609+0000 7fe5014c5640 1 -- 172.21.15.164:0/2554685863 shutdown_connections 2024-03-23T14:58:06.615 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.609+0000 7fe5014c5640 1 -- 172.21.15.164:0/2554685863 wait complete. 2024-03-23T14:58:06.617 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.609+0000 7fe5014c5640 1 Processor -- start 2024-03-23T14:58:06.617 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.609+0000 7fe5014c5640 1 -- start start 2024-03-23T14:58:06.617 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.609+0000 7fe5014c5640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe4fc0b5790 0x7fe4fc149f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:06.618 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.609+0000 7fe5014c5640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe4fc0b5b70 0x7fe4fc0b2d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:06.618 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.609+0000 7fe5014c5640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe4fc0b3f90 0x7fe4fc0b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:06.618 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.609+0000 7fe5014c5640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe4fc06c5d0 con 0x7fe4fc0b5b70 2024-03-23T14:58:06.618 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.609+0000 7fe5014c5640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fe4fc0b2960 con 0x7fe4fc0b3f90 2024-03-23T14:58:06.618 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.609+0000 7fe5014c5640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fe4fc0b4450 con 0x7fe4fc0b5790 2024-03-23T14:58:06.618 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.613+0000 7fe4fa7fc640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe4fc0b5b70 0x7fe4fc0b2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:06.618 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.613+0000 7fe4fa7fc640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe4fc0b5b70 0x7fe4fc0b2d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:49068/0 (socket says 172.21.15.164:49068) 2024-03-23T14:58:06.618 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.613+0000 7fe4fa7fc640 1 -- 172.21.15.164:0/2331264410 learned_addr learned my addr 172.21.15.164:0/2331264410 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:58:06.618 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.613+0000 7fe4faffd640 1 --2- 172.21.15.164:0/2331264410 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe4fc0b5790 0x7fe4fc149f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:06.619 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.613+0000 7fe4fb7fe640 1 --2- 172.21.15.164:0/2331264410 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe4fc0b3f90 0x7fe4fc0b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:06.619 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.613+0000 7fe4fa7fc640 1 -- 172.21.15.164:0/2331264410 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe4fc0b5790 msgr2=0x7fe4fc149f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:06.619 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.613+0000 7fe4fa7fc640 1 --2- 172.21.15.164:0/2331264410 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe4fc0b5790 0x7fe4fc149f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:06.619 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.613+0000 7fe4fa7fc640 1 -- 172.21.15.164:0/2331264410 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe4fc0b3f90 msgr2=0x7fe4fc0b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:06.619 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.613+0000 7fe4fa7fc640 1 --2- 172.21.15.164:0/2331264410 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe4fc0b3f90 0x7fe4fc0b32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:06.619 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.613+0000 7fe4fa7fc640 1 -- 172.21.15.164:0/2331264410 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4ec0094d0 con 0x7fe4fc0b5b70 2024-03-23T14:58:06.619 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.613+0000 7fe4faffd640 1 --2- 172.21.15.164:0/2331264410 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe4fc0b5790 0x7fe4fc149f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T14:58:06.619 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.613+0000 7fe4fb7fe640 1 --2- 172.21.15.164:0/2331264410 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe4fc0b3f90 0x7fe4fc0b32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T14:58:06.620 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.613+0000 7fe4fa7fc640 1 --2- 172.21.15.164:0/2331264410 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe4fc0b5b70 0x7fe4fc0b2d80 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fe4f000b3e0 tx=0x7fe4f000b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:58:06.620 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.613+0000 7fe4e37fe640 1 -- 172.21.15.164:0/2331264410 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4f0011830 con 0x7fe4fc0b5b70 2024-03-23T14:58:06.620 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.613+0000 7fe4e37fe640 1 -- 172.21.15.164:0/2331264410 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4f0011990 con 0x7fe4fc0b5b70 2024-03-23T14:58:06.620 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.613+0000 7fe5014c5640 1 -- 172.21.15.164:0/2331264410 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4fc0b4730 con 0x7fe4fc0b5b70 2024-03-23T14:58:06.620 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.613+0000 7fe4e37fe640 1 -- 172.21.15.164:0/2331264410 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4f0011d80 con 0x7fe4fc0b5b70 2024-03-23T14:58:06.620 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.613+0000 7fe5014c5640 1 -- 172.21.15.164:0/2331264410 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4fc0b24e0 con 0x7fe4fc0b5b70 2024-03-23T14:58:06.620 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.613+0000 7fe5014c5640 1 -- 172.21.15.164:0/2331264410 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4fc0b2aa0 con 0x7fe4fc0b5b70 2024-03-23T14:58:06.620 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.613+0000 7fe4e37fe640 1 -- 172.21.15.164:0/2331264410 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7fe4f001a400 con 0x7fe4fc0b5b70 2024-03-23T14:58:06.621 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.613+0000 7fe4e37fe640 1 --2- 172.21.15.164:0/2331264410 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fe4e80672d0 0x7fe4e8069790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:06.621 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.613+0000 7fe4faffd640 1 --2- 172.21.15.164:0/2331264410 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fe4e80672d0 0x7fe4e8069790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:06.621 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.613+0000 7fe4e37fe640 1 -- 172.21.15.164:0/2331264410 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4943+0+0 (secure 0 0 0) 0x7fe4f0084a70 con 0x7fe4fc0b5b70 2024-03-23T14:58:06.621 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.613+0000 7fe4faffd640 1 --2- 172.21.15.164:0/2331264410 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fe4e80672d0 0x7fe4e8069790 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec0ad000 tx=0x7fe4ec008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:58:06.622 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.617+0000 7fe4e37fe640 1 -- 172.21.15.164:0/2331264410 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7fe4fc0b2aa0 con 0x7fe4fc0b5b70 2024-03-23T14:58:06.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.773+0000 7fe5014c5640 1 -- 172.21.15.164:0/2331264410 --> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7fe4fc0b2aa0 con 0x7fe4e80672d0 2024-03-23T14:58:06.779 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.777+0000 7fe4e37fe640 1 -- 172.21.15.164:0/2331264410 <== mgr.4107 v2:172.21.15.164:6832/2139087985 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fe4fc0b2aa0 con 0x7fe4e80672d0 2024-03-23T14:58:06.780 INFO:teuthology.orchestra.run.smithi164.stderr:instructed osd(s) 7 to deep-scrub 2024-03-23T14:58:06.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.777+0000 7fe5014c5640 1 -- 172.21.15.164:0/2331264410 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fe4e80672d0 msgr2=0x7fe4e8069790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:06.782 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.777+0000 7fe5014c5640 1 --2- 172.21.15.164:0/2331264410 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fe4e80672d0 0x7fe4e8069790 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec0ad000 tx=0x7fe4ec008040 comp rx=0 tx=0).stop 2024-03-23T14:58:06.783 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.777+0000 7fe5014c5640 1 -- 172.21.15.164:0/2331264410 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe4fc0b5b70 msgr2=0x7fe4fc0b2d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:06.783 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.777+0000 7fe5014c5640 1 --2- 172.21.15.164:0/2331264410 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe4fc0b5b70 0x7fe4fc0b2d80 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fe4f000b3e0 tx=0x7fe4f000b8a0 comp rx=0 tx=0).stop 2024-03-23T14:58:06.783 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.777+0000 7fe5014c5640 1 -- 172.21.15.164:0/2331264410 shutdown_connections 2024-03-23T14:58:06.783 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.777+0000 7fe5014c5640 1 --2- 172.21.15.164:0/2331264410 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fe4e80672d0 0x7fe4e8069790 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:06.783 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.777+0000 7fe5014c5640 1 --2- 172.21.15.164:0/2331264410 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe4fc0b3f90 0x7fe4fc0b32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:06.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.777+0000 7fe5014c5640 1 --2- 172.21.15.164:0/2331264410 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe4fc0b5b70 0x7fe4fc0b2d80 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:06.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.777+0000 7fe5014c5640 1 --2- 172.21.15.164:0/2331264410 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe4fc0b5790 0x7fe4fc149f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:06.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.777+0000 7fe5014c5640 1 -- 172.21.15.164:0/2331264410 >> 172.21.15.164:0/2331264410 conn(0x7fe4fc0b7630 msgr2=0x7fe4fc147480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:58:06.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.777+0000 7fe5014c5640 1 -- 172.21.15.164:0/2331264410 shutdown_connections 2024-03-23T14:58:06.784 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.777+0000 7fe5014c5640 1 -- 172.21.15.164:0/2331264410 wait complete. 2024-03-23T14:58:06.798 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-23T14:58:06.915 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.909+0000 7fa2953a0640 1 Processor -- start 2024-03-23T14:58:06.915 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.909+0000 7fa2953a0640 1 -- start start 2024-03-23T14:58:06.916 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.913+0000 7fa2953a0640 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fa29009def0 0x7fa2900b45e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:06.916 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.913+0000 7fa2953a0640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fa2900add50 0x7fa29006bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:06.917 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.913+0000 7fa2953a0640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa29006d650 con 0x7fa2900b4b20 2024-03-23T14:58:06.917 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.913+0000 7fa2953a0640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fa2900af4a0 con 0x7fa2900add50 2024-03-23T14:58:06.917 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.913+0000 7fa2953a0640 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 -- 0x7fa29009e2d0 con 0x7fa29009def0 2024-03-23T14:58:06.917 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.913+0000 7fa28e7fc640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fa2900add50 0x7fa29006bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:06.917 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.913+0000 7fa28e7fc640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fa2900add50 0x7fa29006bf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:49076/0 (socket says 172.21.15.164:49076) 2024-03-23T14:58:06.917 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.913+0000 7fa28e7fc640 1 -- 172.21.15.164:0/2686435038 learned_addr learned my addr 172.21.15.164:0/2686435038 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:58:06.917 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.913+0000 7fa28effd640 1 --2- 172.21.15.164:0/2686435038 >> v2:172.21.15.173:3300/0 conn(0x7fa29009def0 0x7fa2900b45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:06.917 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.913+0000 7fa28e7fc640 1 -- 172.21.15.164:0/2686435038 >> v2:172.21.15.173:3300/0 conn(0x7fa29009def0 msgr2=0x7fa2900b45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:58:06.917 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.913+0000 7fa28e7fc640 1 --2- 172.21.15.164:0/2686435038 >> v2:172.21.15.173:3300/0 conn(0x7fa29009def0 0x7fa2900b45e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:06.917 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.913+0000 7fa28e7fc640 1 -- 172.21.15.164:0/2686435038 >> v1:172.21.15.173:6789/0 conn(0x7fa2900b4b20 legacy=0x7fa29006c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:58:06.918 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.913+0000 7fa28e7fc640 1 -- 172.21.15.164:0/2686435038 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2900aeca0 con 0x7fa2900add50 2024-03-23T14:58:06.918 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.913+0000 7fa28effd640 1 --2- 172.21.15.164:0/2686435038 >> v2:172.21.15.173:3300/0 conn(0x7fa29009def0 0x7fa2900b45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T14:58:06.919 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.913+0000 7fa28e7fc640 1 --2- 172.21.15.164:0/2686435038 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa2900add50 0x7fa29006bf70 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fa29006da30 tx=0x7fa28401c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=594d062aba6d2ba1 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:58:06.919 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.913+0000 7fa28dffb640 1 -- 172.21.15.164:0/2686435038 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2840ab600 con 0x7fa2900add50 2024-03-23T14:58:06.919 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.913+0000 7fa28dffb640 1 -- 172.21.15.164:0/2686435038 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2840a7050 con 0x7fa2900add50 2024-03-23T14:58:06.919 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.913+0000 7fa28dffb640 1 -- 172.21.15.164:0/2686435038 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2840ab980 con 0x7fa2900add50 2024-03-23T14:58:06.919 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.913+0000 7fa2953a0640 1 -- 172.21.15.164:0/2686435038 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa2900add50 msgr2=0x7fa29006bf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:06.920 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.913+0000 7fa2953a0640 1 --2- 172.21.15.164:0/2686435038 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa2900add50 0x7fa29006bf70 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fa29006da30 tx=0x7fa28401c3c0 comp rx=0 tx=0).stop 2024-03-23T14:58:06.920 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.913+0000 7fa2953a0640 1 -- 172.21.15.164:0/2686435038 shutdown_connections 2024-03-23T14:58:06.920 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.913+0000 7fa2953a0640 1 --2- 172.21.15.164:0/2686435038 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa2900add50 0x7fa29006bf70 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:06.920 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.913+0000 7fa2953a0640 1 --2- 172.21.15.164:0/2686435038 >> v2:172.21.15.173:3300/0 conn(0x7fa29009def0 0x7fa2900b45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:06.920 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.913+0000 7fa2953a0640 1 -- 172.21.15.164:0/2686435038 >> 172.21.15.164:0/2686435038 conn(0x7fa2900b8ca0 msgr2=0x7fa2900b90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:58:06.920 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.913+0000 7fa2953a0640 1 -- 172.21.15.164:0/2686435038 shutdown_connections 2024-03-23T14:58:06.920 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.913+0000 7fa2953a0640 1 -- 172.21.15.164:0/2686435038 wait complete. 2024-03-23T14:58:06.922 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.917+0000 7fa2953a0640 1 Processor -- start 2024-03-23T14:58:06.922 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.917+0000 7fa2953a0640 1 -- start start 2024-03-23T14:58:06.922 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.917+0000 7fa2953a0640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa29009def0 0x7fa2900b1a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:06.923 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.917+0000 7fa2953a0640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa2900add50 0x7fa290142730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:06.923 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.917+0000 7fa2953a0640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa2900b4b20 0x7fa2901e13f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:06.923 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.917+0000 7fa2953a0640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa29009cde0 con 0x7fa29009def0 2024-03-23T14:58:06.923 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.917+0000 7fa2953a0640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa290142c70 con 0x7fa2900b4b20 2024-03-23T14:58:06.923 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.917+0000 7fa2953a0640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fa290142e80 con 0x7fa2900add50 2024-03-23T14:58:06.923 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.917+0000 7fa28effd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa29009def0 0x7fa2900b1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:06.923 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.917+0000 7fa28effd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa29009def0 0x7fa2900b1a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:49086/0 (socket says 172.21.15.164:49086) 2024-03-23T14:58:06.923 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.917+0000 7fa28effd640 1 -- 172.21.15.164:0/3190972632 learned_addr learned my addr 172.21.15.164:0/3190972632 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:58:06.923 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.917+0000 7fa28e7fc640 1 --2- 172.21.15.164:0/3190972632 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa2900add50 0x7fa290142730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:06.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.917+0000 7fa28f7fe640 1 --2- 172.21.15.164:0/3190972632 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa2900b4b20 0x7fa2901e13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:06.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.917+0000 7fa28effd640 1 -- 172.21.15.164:0/3190972632 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa2900add50 msgr2=0x7fa290142730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:06.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.917+0000 7fa28effd640 1 --2- 172.21.15.164:0/3190972632 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa2900add50 0x7fa290142730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:06.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.917+0000 7fa28effd640 1 -- 172.21.15.164:0/3190972632 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa2900b4b20 msgr2=0x7fa2901e13f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:06.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.917+0000 7fa28effd640 1 --2- 172.21.15.164:0/3190972632 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa2900b4b20 0x7fa2901e13f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:06.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.917+0000 7fa28effd640 1 -- 172.21.15.164:0/3190972632 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2840094d0 con 0x7fa29009def0 2024-03-23T14:58:06.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.917+0000 7fa28effd640 1 --2- 172.21.15.164:0/3190972632 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa29009def0 0x7fa2900b1a80 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fa28800cf60 tx=0x7fa28800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:58:06.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.917+0000 7fa2777fe640 1 -- 172.21.15.164:0/3190972632 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa28800d840 con 0x7fa29009def0 2024-03-23T14:58:06.924 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.917+0000 7fa2777fe640 1 -- 172.21.15.164:0/3190972632 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa28800d9a0 con 0x7fa29009def0 2024-03-23T14:58:06.925 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.917+0000 7fa2953a0640 1 -- 172.21.15.164:0/3190972632 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa290143160 con 0x7fa29009def0 2024-03-23T14:58:06.925 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.917+0000 7fa2777fe640 1 -- 172.21.15.164:0/3190972632 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa28800dd30 con 0x7fa29009def0 2024-03-23T14:58:06.925 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.917+0000 7fa2953a0640 1 -- 172.21.15.164:0/3190972632 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa29009bd50 con 0x7fa29009def0 2024-03-23T14:58:06.925 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.917+0000 7fa2953a0640 1 -- 172.21.15.164:0/3190972632 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2900690a0 con 0x7fa29009def0 2024-03-23T14:58:06.926 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.917+0000 7fa2777fe640 1 -- 172.21.15.164:0/3190972632 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7fa288015440 con 0x7fa29009def0 2024-03-23T14:58:06.926 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.917+0000 7fa2777fe640 1 --2- 172.21.15.164:0/3190972632 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fa27c0672a0 0x7fa27c069760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:06.926 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.917+0000 7fa2777fe640 1 -- 172.21.15.164:0/3190972632 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4943+0+0 (secure 0 0 0) 0x7fa2880856e0 con 0x7fa29009def0 2024-03-23T14:58:06.926 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.917+0000 7fa28e7fc640 1 --2- 172.21.15.164:0/3190972632 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fa27c0672a0 0x7fa27c069760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:06.926 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.921+0000 7fa28e7fc640 1 --2- 172.21.15.164:0/3190972632 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fa27c0672a0 0x7fa27c069760 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fa2840ad000 tx=0x7fa284008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:58:06.927 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:06.925+0000 7fa2777fe640 1 -- 172.21.15.164:0/3190972632 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7fa2880526d0 con 0x7fa29009def0 2024-03-23T14:58:07.069 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:07.065+0000 7fa2953a0640 1 -- 172.21.15.164:0/3190972632 --> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa29013cc30 con 0x7fa27c0672a0 2024-03-23T14:58:07.071 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:07.065+0000 7fa2777fe640 1 -- 172.21.15.164:0/3190972632 <== mgr.4107 v2:172.21.15.164:6832/2139087985 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13479 (secure 0 0 0) 0x7fa29013cc30 con 0x7fa27c0672a0 2024-03-23T14:58:07.071 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-23T14:58:07.072 INFO:teuthology.orchestra.run.smithi164.stderr:dumped all 2024-03-23T14:58:07.074 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:07.069+0000 7fa2953a0640 1 -- 172.21.15.164:0/3190972632 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fa27c0672a0 msgr2=0x7fa27c069760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:07.074 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:07.069+0000 7fa2953a0640 1 --2- 172.21.15.164:0/3190972632 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fa27c0672a0 0x7fa27c069760 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fa2840ad000 tx=0x7fa284008040 comp rx=0 tx=0).stop 2024-03-23T14:58:07.074 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:07.069+0000 7fa2953a0640 1 -- 172.21.15.164:0/3190972632 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa29009def0 msgr2=0x7fa2900b1a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:07.074 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:07.069+0000 7fa2953a0640 1 --2- 172.21.15.164:0/3190972632 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa29009def0 0x7fa2900b1a80 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fa28800cf60 tx=0x7fa28800cf90 comp rx=0 tx=0).stop 2024-03-23T14:58:07.074 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:07.069+0000 7fa2953a0640 1 -- 172.21.15.164:0/3190972632 shutdown_connections 2024-03-23T14:58:07.074 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:07.069+0000 7fa2953a0640 1 --2- 172.21.15.164:0/3190972632 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fa27c0672a0 0x7fa27c069760 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:07.074 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:07.069+0000 7fa2953a0640 1 --2- 172.21.15.164:0/3190972632 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa2900b4b20 0x7fa2901e13f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:07.075 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:07.069+0000 7fa2953a0640 1 --2- 172.21.15.164:0/3190972632 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa2900add50 0x7fa290142730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:07.075 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:07.069+0000 7fa2953a0640 1 --2- 172.21.15.164:0/3190972632 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa29009def0 0x7fa2900b1a80 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:07.075 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:07.069+0000 7fa2953a0640 1 -- 172.21.15.164:0/3190972632 >> 172.21.15.164:0/3190972632 conn(0x7fa2900b8ca0 msgr2=0x7fa29013c520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:58:07.075 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:07.069+0000 7fa2953a0640 1 -- 172.21.15.164:0/3190972632 shutdown_connections 2024-03-23T14:58:07.075 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:07.069+0000 7fa2953a0640 1 -- 172.21.15.164:0/3190972632 wait complete. 2024-03-23T14:58:07.089 INFO:teuthology.orchestra.run.smithi164.stdout:{"pg_ready":true,"pg_map":{"version":64,"stamp":"2024-03-23T14:58:05.126392+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":47,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":220216,"kb_used_data":4040,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754504,"statfs":{"total":773094113280,"available":772868612096,"internally_reserved":0,"allocated":4136960,"data_stored":2314024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-10891,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-55,"num_write_kb":-28,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001762"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":193,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-23T14:58:00.962141+0000","last_change":"2024-03-23T14:56:48.722063+0000","last_active":"2024-03-23T14:58:00.962141+0000","last_peered":"2024-03-23T14:58:00.962141+0000","last_clean":"2024-03-23T14:58:00.962141+0000","last_became_active":"2024-03-23T14:56:48.720079+0000","last_became_peered":"2024-03-23T14:56:48.720079+0000","last_unstale":"2024-03-23T14:58:00.962141+0000","last_undegraded":"2024-03-23T14:58:00.962141+0000","last_fullsized":"2024-03-23T14:58:00.962141+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-23T14:56:47.478078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-23T14:56:47.478078+0000","last_clean_scrub_stamp":"2024-03-23T14:56:47.478078+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:55:51.231901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574867,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27960,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343880,"statfs":{"total":96636764160,"available":96608133120,"internally_reserved":0,"allocated":962560,"data_stored":732029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574867,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27380,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":368640,"data_stored":141661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574867,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27384,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":368640,"data_stored":141661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574867,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27384,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":368640,"data_stored":141661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27384,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":368640,"data_stored":141661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574867,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27380,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":368640,"data_stored":141661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574868,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27380,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":368640,"data_stored":141661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574866,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27964,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343876,"statfs":{"total":96636764160,"available":96608129024,"internally_reserved":0,"allocated":962560,"data_stored":732029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-23T14:58:07.091 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-03-23T14:56:47.478078+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=23, tm_hour=14, tm_min=56, tm_sec=47, tm_wday=5, tm_yday=83, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=23, tm_hour=14, tm_min=58, tm_sec=2, tm_wday=5, tm_yday=83, tm_isdst=0) 2024-03-23T14:58:07.091 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-03-23T14:58:27.093 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-23T14:58:27.225 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.221+0000 7fdf4231e640 1 Processor -- start 2024-03-23T14:58:27.225 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.221+0000 7fdf4231e640 1 -- start start 2024-03-23T14:58:27.227 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.221+0000 7fdf4231e640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fdf3c06e730 0x7fdf3c0b3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:27.227 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.221+0000 7fdf4231e640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdf3c06d770 0x7fdf3c0b37c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:27.227 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.221+0000 7fdf4231e640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf3c0b0ca0 con 0x7fdf3c0725b0 2024-03-23T14:58:27.228 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.221+0000 7fdf4231e640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fdf3c072990 con 0x7fdf3c06e730 2024-03-23T14:58:27.228 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.221+0000 7fdf4231e640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fdf3c0b95f0 con 0x7fdf3c06d770 2024-03-23T14:58:27.228 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.221+0000 7fdf3b7fe640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fdf3c06e730 0x7fdf3c0b3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:27.228 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.221+0000 7fdf3b7fe640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fdf3c06e730 0x7fdf3c0b3280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:42234/0 (socket says 172.21.15.164:42234) 2024-03-23T14:58:27.228 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.221+0000 7fdf3b7fe640 1 -- 172.21.15.164:0/3145285034 learned_addr learned my addr 172.21.15.164:0/3145285034 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:58:27.228 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.221+0000 7fdf408bb640 1 --2- 172.21.15.164:0/3145285034 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdf3c06d770 0x7fdf3c0b37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:27.228 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.221+0000 7fdf3b7fe640 1 -- 172.21.15.164:0/3145285034 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdf3c06d770 msgr2=0x7fdf3c0b37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:58:27.228 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.221+0000 7fdf3b7fe640 1 --2- 172.21.15.164:0/3145285034 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdf3c06d770 0x7fdf3c0b37c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:27.228 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.221+0000 7fdf3b7fe640 1 -- 172.21.15.164:0/3145285034 >> v1:172.21.15.173:6789/0 conn(0x7fdf3c0725b0 legacy=0x7fdf3c148890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:58:27.228 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.221+0000 7fdf3b7fe640 1 -- 172.21.15.164:0/3145285034 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf3c06cf00 con 0x7fdf3c06e730 2024-03-23T14:58:27.230 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.225+0000 7fdf3b7fe640 1 --2- 172.21.15.164:0/3145285034 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf3c06e730 0x7fdf3c0b3280 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c009e50 tx=0x7fdf2c01c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bca981714650ffd server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:58:27.230 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.225+0000 7fdf3affd640 1 -- 172.21.15.164:0/3145285034 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf2c0a56f0 con 0x7fdf3c06e730 2024-03-23T14:58:27.230 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.225+0000 7fdf3affd640 1 -- 172.21.15.164:0/3145285034 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf2c0a5850 con 0x7fdf3c06e730 2024-03-23T14:58:27.230 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.225+0000 7fdf3affd640 1 -- 172.21.15.164:0/3145285034 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf2c0a5be0 con 0x7fdf3c06e730 2024-03-23T14:58:27.230 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.225+0000 7fdf4231e640 1 -- 172.21.15.164:0/3145285034 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf3c06e730 msgr2=0x7fdf3c0b3280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:27.230 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.225+0000 7fdf4231e640 1 --2- 172.21.15.164:0/3145285034 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf3c06e730 0x7fdf3c0b3280 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c009e50 tx=0x7fdf2c01c960 comp rx=0 tx=0).stop 2024-03-23T14:58:27.231 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.225+0000 7fdf4231e640 1 -- 172.21.15.164:0/3145285034 shutdown_connections 2024-03-23T14:58:27.231 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.225+0000 7fdf4231e640 1 --2- 172.21.15.164:0/3145285034 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdf3c06d770 0x7fdf3c0b37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:27.231 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.225+0000 7fdf4231e640 1 --2- 172.21.15.164:0/3145285034 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf3c06e730 0x7fdf3c0b3280 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:27.231 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.225+0000 7fdf4231e640 1 -- 172.21.15.164:0/3145285034 >> 172.21.15.164:0/3145285034 conn(0x7fdf3c0b7970 msgr2=0x7fdf3c0b7d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:58:27.231 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.225+0000 7fdf4231e640 1 -- 172.21.15.164:0/3145285034 shutdown_connections 2024-03-23T14:58:27.231 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.225+0000 7fdf4231e640 1 -- 172.21.15.164:0/3145285034 wait complete. 2024-03-23T14:58:27.231 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.225+0000 7fdf4231e640 1 Processor -- start 2024-03-23T14:58:27.231 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.225+0000 7fdf4231e640 1 -- start start 2024-03-23T14:58:27.234 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.225+0000 7fdf4231e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf3c06d770 0x7fdf3c140080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:27.234 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.225+0000 7fdf4231e640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdf3c06e730 0x7fdf3c1e0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:27.234 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.225+0000 7fdf4231e640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdf3c0725b0 0x7fdf3c1e10d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:27.234 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.225+0000 7fdf4231e640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fdf3c141140 con 0x7fdf3c06d770 2024-03-23T14:58:27.234 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.225+0000 7fdf4231e640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fdf3c140860 con 0x7fdf3c0725b0 2024-03-23T14:58:27.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.225+0000 7fdf4231e640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7fdf3c1405c0 con 0x7fdf3c06e730 2024-03-23T14:58:27.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.225+0000 7fdf3bfff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf3c06d770 0x7fdf3c140080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:27.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.225+0000 7fdf3bfff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf3c06d770 0x7fdf3c140080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:42236/0 (socket says 172.21.15.164:42236) 2024-03-23T14:58:27.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.225+0000 7fdf3bfff640 1 -- 172.21.15.164:0/1398167173 learned_addr learned my addr 172.21.15.164:0/1398167173 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:58:27.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.225+0000 7fdf3b7fe640 1 --2- 172.21.15.164:0/1398167173 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdf3c06e730 0x7fdf3c1e0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:27.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.225+0000 7fdf408bb640 1 --2- 172.21.15.164:0/1398167173 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdf3c0725b0 0x7fdf3c1e10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:27.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.225+0000 7fdf3bfff640 1 -- 172.21.15.164:0/1398167173 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdf3c06e730 msgr2=0x7fdf3c1e0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:27.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.225+0000 7fdf3bfff640 1 --2- 172.21.15.164:0/1398167173 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdf3c06e730 0x7fdf3c1e0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:27.235 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.225+0000 7fdf3bfff640 1 -- 172.21.15.164:0/1398167173 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdf3c0725b0 msgr2=0x7fdf3c1e10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:27.236 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.225+0000 7fdf3bfff640 1 --2- 172.21.15.164:0/1398167173 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdf3c0725b0 0x7fdf3c1e10d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:27.236 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.225+0000 7fdf3bfff640 1 -- 172.21.15.164:0/1398167173 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf2c009b30 con 0x7fdf3c06d770 2024-03-23T14:58:27.236 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.225+0000 7fdf408bb640 1 --2- 172.21.15.164:0/1398167173 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdf3c0725b0 0x7fdf3c1e10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T14:58:27.236 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.225+0000 7fdf3b7fe640 1 --2- 172.21.15.164:0/1398167173 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdf3c06e730 0x7fdf3c1e0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-23T14:58:27.236 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.225+0000 7fdf3bfff640 1 --2- 172.21.15.164:0/1398167173 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf3c06d770 0x7fdf3c140080 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fdf2400c450 tx=0x7fdf2400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:58:27.236 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.229+0000 7fdf38ff9640 1 -- 172.21.15.164:0/1398167173 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf24012830 con 0x7fdf3c06d770 2024-03-23T14:58:27.236 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.229+0000 7fdf38ff9640 1 -- 172.21.15.164:0/1398167173 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf240129b0 con 0x7fdf3c06d770 2024-03-23T14:58:27.236 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.229+0000 7fdf4231e640 1 -- 172.21.15.164:0/1398167173 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf3c1429d0 con 0x7fdf3c06d770 2024-03-23T14:58:27.236 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.229+0000 7fdf38ff9640 1 -- 172.21.15.164:0/1398167173 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf24012d60 con 0x7fdf3c06d770 2024-03-23T14:58:27.237 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.229+0000 7fdf4231e640 1 -- 172.21.15.164:0/1398167173 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf3c13c5b0 con 0x7fdf3c06d770 2024-03-23T14:58:27.237 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.229+0000 7fdf4231e640 1 -- 172.21.15.164:0/1398167173 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf3c13e2f0 con 0x7fdf3c06d770 2024-03-23T14:58:27.237 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.229+0000 7fdf38ff9640 1 -- 172.21.15.164:0/1398167173 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7fdf2401c460 con 0x7fdf3c06d770 2024-03-23T14:58:27.237 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.229+0000 7fdf38ff9640 1 --2- 172.21.15.164:0/1398167173 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fdf1c0672a0 0x7fdf1c069760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:27.237 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.229+0000 7fdf3b7fe640 1 --2- 172.21.15.164:0/1398167173 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fdf1c0672a0 0x7fdf1c069760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:27.237 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.229+0000 7fdf38ff9640 1 -- 172.21.15.164:0/1398167173 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4943+0+0 (secure 0 0 0) 0x7fdf24084b70 con 0x7fdf3c06d770 2024-03-23T14:58:27.237 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.229+0000 7fdf3b7fe640 1 --2- 172.21.15.164:0/1398167173 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fdf1c0672a0 0x7fdf1c069760 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c0ae000 tx=0x7fdf2c0a7070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:58:27.238 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.233+0000 7fdf38ff9640 1 -- 172.21.15.164:0/1398167173 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7fdf3c13e2f0 con 0x7fdf3c06d770 2024-03-23T14:58:27.395 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.389+0000 7fdf4231e640 1 -- 172.21.15.164:0/1398167173 --> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fdf3c13e2f0 con 0x7fdf1c0672a0 2024-03-23T14:58:27.397 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.393+0000 7fdf38ff9640 1 -- 172.21.15.164:0/1398167173 <== mgr.4107 v2:172.21.15.164:6832/2139087985 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13354 (secure 0 0 0) 0x7fdf3c13e2f0 con 0x7fdf1c0672a0 2024-03-23T14:58:27.398 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-23T14:58:27.398 INFO:teuthology.orchestra.run.smithi164.stderr:dumped all 2024-03-23T14:58:27.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.393+0000 7fdf4231e640 1 -- 172.21.15.164:0/1398167173 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fdf1c0672a0 msgr2=0x7fdf1c069760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:27.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.393+0000 7fdf4231e640 1 --2- 172.21.15.164:0/1398167173 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fdf1c0672a0 0x7fdf1c069760 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c0ae000 tx=0x7fdf2c0a7070 comp rx=0 tx=0).stop 2024-03-23T14:58:27.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.393+0000 7fdf4231e640 1 -- 172.21.15.164:0/1398167173 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf3c06d770 msgr2=0x7fdf3c140080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:27.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.393+0000 7fdf4231e640 1 --2- 172.21.15.164:0/1398167173 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf3c06d770 0x7fdf3c140080 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fdf2400c450 tx=0x7fdf2400c910 comp rx=0 tx=0).stop 2024-03-23T14:58:27.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.393+0000 7fdf4231e640 1 -- 172.21.15.164:0/1398167173 shutdown_connections 2024-03-23T14:58:27.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.397+0000 7fdf4231e640 1 --2- 172.21.15.164:0/1398167173 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7fdf1c0672a0 0x7fdf1c069760 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:27.400 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.397+0000 7fdf4231e640 1 --2- 172.21.15.164:0/1398167173 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdf3c0725b0 0x7fdf3c1e10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:27.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.397+0000 7fdf4231e640 1 --2- 172.21.15.164:0/1398167173 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdf3c06e730 0x7fdf3c1e0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:27.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.397+0000 7fdf4231e640 1 --2- 172.21.15.164:0/1398167173 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf3c06d770 0x7fdf3c140080 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:27.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.397+0000 7fdf4231e640 1 -- 172.21.15.164:0/1398167173 >> 172.21.15.164:0/1398167173 conn(0x7fdf3c0b7970 msgr2=0x7fdf3c13ef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:58:27.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.397+0000 7fdf4231e640 1 -- 172.21.15.164:0/1398167173 shutdown_connections 2024-03-23T14:58:27.401 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.397+0000 7fdf4231e640 1 -- 172.21.15.164:0/1398167173 wait complete. 2024-03-23T14:58:27.416 INFO:teuthology.orchestra.run.smithi164.stdout:{"pg_ready":true,"pg_map":{"version":75,"stamp":"2024-03-23T14:58:27.131952+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":220204,"kb_used_data":4040,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754516,"statfs":{"total":773094113280,"available":772868624384,"internally_reserved":0,"allocated":4136960,"data_stored":2314024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003123"},"pg_stats":[{"pgid":"1.0","version":"32'119","reported_seq":204,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-03-23T14:58:07.439837+0000","last_change":"2024-03-23T14:58:07.440001+0000","last_active":"2024-03-23T14:58:07.439837+0000","last_peered":"2024-03-23T14:58:07.439837+0000","last_clean":"2024-03-23T14:58:07.439837+0000","last_became_active":"2024-03-23T14:56:48.720079+0000","last_became_peered":"2024-03-23T14:56:48.720079+0000","last_unstale":"2024-03-23T14:58:07.439837+0000","last_undegraded":"2024-03-23T14:58:07.439837+0000","last_fullsized":"2024-03-23T14:58:07.439837+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'119","last_scrub_stamp":"2024-03-23T14:58:07.439603+0000","last_deep_scrub":"32'119","last_deep_scrub_stamp":"2024-03-23T14:58:07.439603+0000","last_clean_scrub_stamp":"2024-03-23T14:58:07.439603+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-24T15:11:56.756751+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574871,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27956,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343884,"statfs":{"total":96636764160,"available":96608137216,"internally_reserved":0,"allocated":962560,"data_stored":732029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574871,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27380,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":368640,"data_stored":141661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574871,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27384,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":368640,"data_stored":141661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574871,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27384,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":368640,"data_stored":141661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574872,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27384,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":368640,"data_stored":141661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574871,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27380,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":368640,"data_stored":141661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574872,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27380,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":368640,"data_stored":141661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574870,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27956,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343884,"statfs":{"total":96636764160,"available":96608137216,"internally_reserved":0,"allocated":962560,"data_stored":732029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-23T14:58:27.417 DEBUG:teuthology.orchestra.run.smithi164:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-03-23T14:58:27.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.537+0000 7f90fe333640 1 Processor -- start 2024-03-23T14:58:27.540 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.537+0000 7f90fe333640 1 -- start start 2024-03-23T14:58:27.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.537+0000 7f90fe333640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f90f80b0230 0x7f90f80b34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:27.541 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.537+0000 7f90fe333640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f90f8148cf0 0x7f90f8149180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:27.542 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.537+0000 7f90fe333640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90f80b51d0 con 0x7f90f80b1c70 2024-03-23T14:58:27.542 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.537+0000 7f90fe333640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f90f80b0610 con 0x7f90f80b0230 2024-03-23T14:58:27.542 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.537+0000 7f90fe333640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f90f80b1500 con 0x7f90f8148cf0 2024-03-23T14:58:27.542 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.537+0000 7f90fc8d0640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f90f80b1c70 0x7f90f80b20a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:50212/0 (socket says 172.21.15.164:50212) 2024-03-23T14:58:27.542 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.537+0000 7f90f7fff640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f90f80b0230 0x7f90f80b34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:27.542 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.537+0000 7f90fc8d0640 1 -- 172.21.15.164:0/2001882221 learned_addr learned my addr 172.21.15.164:0/2001882221 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-03-23T14:58:27.542 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.537+0000 7f90f77fe640 1 --2- 172.21.15.164:0/2001882221 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f90f8148cf0 0x7f90f8149180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:27.543 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.537+0000 7f90f7fff640 1 -- 172.21.15.164:0/2001882221 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f90f8148cf0 msgr2=0x7f90f8149180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:58:27.543 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.537+0000 7f90f7fff640 1 --2- 172.21.15.164:0/2001882221 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f90f8148cf0 0x7f90f8149180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:27.543 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.537+0000 7f90f7fff640 1 -- 172.21.15.164:0/2001882221 >> v1:172.21.15.164:6789/0 conn(0x7f90f80b1c70 legacy=0x7f90f80b20a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:27.543 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.537+0000 7f90f6ffd640 1 -- 172.21.15.164:0/2001882221 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2310442352 0 0) 0x7f90f80b51d0 con 0x7f90f80b1c70 2024-03-23T14:58:27.543 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.537+0000 7f90f7fff640 1 -- 172.21.15.164:0/2001882221 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90f806d250 con 0x7f90f80b0230 2024-03-23T14:58:27.545 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.541+0000 7f90f7fff640 1 --2- 172.21.15.164:0/2001882221 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f90f80b0230 0x7f90f80b34d0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f90e80097c0 tx=0x7f90e801c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=324834867bc0b941 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:58:27.545 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.541+0000 7f90f6ffd640 1 -- 172.21.15.164:0/2001882221 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e80ac640 con 0x7f90f80b0230 2024-03-23T14:58:27.545 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.541+0000 7f90f6ffd640 1 -- 172.21.15.164:0/2001882221 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90e80ac7a0 con 0x7f90f80b0230 2024-03-23T14:58:27.545 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.541+0000 7f90f6ffd640 1 -- 172.21.15.164:0/2001882221 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e80acb30 con 0x7f90f80b0230 2024-03-23T14:58:27.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.541+0000 7f90fe333640 1 -- 172.21.15.164:0/2001882221 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f90f80b0230 msgr2=0x7f90f80b34d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:27.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.541+0000 7f90fe333640 1 --2- 172.21.15.164:0/2001882221 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f90f80b0230 0x7f90f80b34d0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f90e80097c0 tx=0x7f90e801c3e0 comp rx=0 tx=0).stop 2024-03-23T14:58:27.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.541+0000 7f90fe333640 1 -- 172.21.15.164:0/2001882221 shutdown_connections 2024-03-23T14:58:27.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.541+0000 7f90fe333640 1 --2- 172.21.15.164:0/2001882221 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f90f8148cf0 0x7f90f8149180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:27.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.541+0000 7f90fe333640 1 --2- 172.21.15.164:0/2001882221 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f90f80b0230 0x7f90f80b34d0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:27.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.541+0000 7f90fe333640 1 -- 172.21.15.164:0/2001882221 >> 172.21.15.164:0/2001882221 conn(0x7f90f80b8ca0 msgr2=0x7f90f80b90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:58:27.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.541+0000 7f90fe333640 1 -- 172.21.15.164:0/2001882221 shutdown_connections 2024-03-23T14:58:27.546 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.541+0000 7f90fe333640 1 -- 172.21.15.164:0/2001882221 wait complete. 2024-03-23T14:58:27.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.541+0000 7f90fe333640 1 Processor -- start 2024-03-23T14:58:27.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.541+0000 7f90fe333640 1 -- start start 2024-03-23T14:58:27.549 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.541+0000 7f90fe333640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f90f80b0230 0x7f90f8071cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:27.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.541+0000 7f90fe333640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f90f80b1c70 0x7f90f81dd9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:27.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.541+0000 7f90fe333640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f90f8148cf0 0x7f90f80705b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:27.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.541+0000 7f90fe333640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f90f8071810 con 0x7f90f8148cf0 2024-03-23T14:58:27.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.541+0000 7f90fe333640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f90f806f7b0 con 0x7f90f80b0230 2024-03-23T14:58:27.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.541+0000 7f90fe333640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f90f8070fb0 con 0x7f90f80b1c70 2024-03-23T14:58:27.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.541+0000 7f90fc8d0640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f90f8148cf0 0x7f90f80705b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:27.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.541+0000 7f90fc8d0640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f90f8148cf0 0x7f90f80705b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:42252/0 (socket says 172.21.15.164:42252) 2024-03-23T14:58:27.550 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.541+0000 7f90fc8d0640 1 -- 172.21.15.164:0/2279729516 learned_addr learned my addr 172.21.15.164:0/2279729516 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:58:27.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.541+0000 7f90f77fe640 1 --2- 172.21.15.164:0/2279729516 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f90f80b1c70 0x7f90f81dd9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:27.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.541+0000 7f90f7fff640 1 --2- 172.21.15.164:0/2279729516 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f90f80b0230 0x7f90f8071cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:27.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.541+0000 7f90fc8d0640 1 -- 172.21.15.164:0/2279729516 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f90f80b1c70 msgr2=0x7f90f81dd9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:27.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.541+0000 7f90fc8d0640 1 --2- 172.21.15.164:0/2279729516 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f90f80b1c70 0x7f90f81dd9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:27.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.541+0000 7f90fc8d0640 1 -- 172.21.15.164:0/2279729516 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f90f80b0230 msgr2=0x7f90f8071cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:27.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.541+0000 7f90fc8d0640 1 --2- 172.21.15.164:0/2279729516 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f90f80b0230 0x7f90f8071cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:27.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.541+0000 7f90fc8d0640 1 -- 172.21.15.164:0/2279729516 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90e8009420 con 0x7f90f8148cf0 2024-03-23T14:58:27.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.541+0000 7f90f77fe640 1 --2- 172.21.15.164:0/2279729516 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f90f80b1c70 0x7f90f81dd9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T14:58:27.551 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.541+0000 7f90f7fff640 1 --2- 172.21.15.164:0/2279729516 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f90f80b0230 0x7f90f8071cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-23T14:58:27.552 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.545+0000 7f90fc8d0640 1 --2- 172.21.15.164:0/2279729516 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f90f8148cf0 0x7f90f80705b0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f90f000c7a0 tx=0x7f90f000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:58:27.552 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.545+0000 7f90f4ff9640 1 -- 172.21.15.164:0/2279729516 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90f000a640 con 0x7f90f8148cf0 2024-03-23T14:58:27.552 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.545+0000 7f90f4ff9640 1 -- 172.21.15.164:0/2279729516 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90f000a7a0 con 0x7f90f8148cf0 2024-03-23T14:58:27.552 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.545+0000 7f90fe333640 1 -- 172.21.15.164:0/2279729516 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90f81d80c0 con 0x7f90f8148cf0 2024-03-23T14:58:27.552 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.545+0000 7f90f4ff9640 1 -- 172.21.15.164:0/2279729516 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90f000abd0 con 0x7f90f8148cf0 2024-03-23T14:58:27.552 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.545+0000 7f90fe333640 1 -- 172.21.15.164:0/2279729516 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90f8070af0 con 0x7f90f8148cf0 2024-03-23T14:58:27.552 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.545+0000 7f90fe333640 1 -- 172.21.15.164:0/2279729516 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90f8070e10 con 0x7f90f8148cf0 2024-03-23T14:58:27.552 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.545+0000 7f90f4ff9640 1 -- 172.21.15.164:0/2279729516 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7f90f000ad30 con 0x7f90f8148cf0 2024-03-23T14:58:27.552 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.545+0000 7f90f4ff9640 1 --2- 172.21.15.164:0/2279729516 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f90e40672a0 0x7f90e4069760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:27.553 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.545+0000 7f90f4ff9640 1 -- 172.21.15.164:0/2279729516 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4943+0+0 (secure 0 0 0) 0x7f90f0084630 con 0x7f90f8148cf0 2024-03-23T14:58:27.553 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.545+0000 7f90f7fff640 1 --2- 172.21.15.164:0/2279729516 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f90e40672a0 0x7f90e4069760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:27.554 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.545+0000 7f90f7fff640 1 --2- 172.21.15.164:0/2279729516 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f90e40672a0 0x7f90e4069760 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f90e80ae000 tx=0x7f90e80a7000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:58:27.554 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.549+0000 7f90f4ff9640 1 -- 172.21.15.164:0/2279729516 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7f90f8070e10 con 0x7f90f8148cf0 2024-03-23T14:58:27.711 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.705+0000 7f90fe333640 1 -- 172.21.15.164:0/2279729516 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7f90f80719f0 con 0x7f90f8148cf0 2024-03-23T14:58:27.715 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.709+0000 7f90f4ff9640 1 -- 172.21.15.164:0/2279729516 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v12)=0 v12) ==== 123+0+0 (secure 0 0 0) 0x7f90f80719f0 con 0x7f90f8148cf0 2024-03-23T14:58:27.716 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.713+0000 7f90f4ff9640 1 -- 172.21.15.164:0/2279729516 <== mon.0 v2:172.21.15.164:3300/0 8 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f90f0001840 con 0x7f90f8148cf0 2024-03-23T14:58:27.718 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.713+0000 7f90fe333640 1 -- 172.21.15.164:0/2279729516 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f90e40672a0 msgr2=0x7f90e4069760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:27.719 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.713+0000 7f90fe333640 1 --2- 172.21.15.164:0/2279729516 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f90e40672a0 0x7f90e4069760 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f90e80ae000 tx=0x7f90e80a7000 comp rx=0 tx=0).stop 2024-03-23T14:58:27.719 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.713+0000 7f90fe333640 1 -- 172.21.15.164:0/2279729516 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f90f8148cf0 msgr2=0x7f90f80705b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:27.719 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.713+0000 7f90fe333640 1 --2- 172.21.15.164:0/2279729516 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f90f8148cf0 0x7f90f80705b0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f90f000c7a0 tx=0x7f90f000cc60 comp rx=0 tx=0).stop 2024-03-23T14:58:27.719 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.713+0000 7f90fe333640 1 -- 172.21.15.164:0/2279729516 shutdown_connections 2024-03-23T14:58:27.719 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.713+0000 7f90fe333640 1 --2- 172.21.15.164:0/2279729516 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f90f8148cf0 0x7f90f80705b0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:27.719 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.713+0000 7f90fe333640 1 --2- 172.21.15.164:0/2279729516 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f90f80b1c70 0x7f90f81dd9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:27.719 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.713+0000 7f90fe333640 1 --2- 172.21.15.164:0/2279729516 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f90e40672a0 0x7f90e4069760 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:27.720 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.713+0000 7f90fe333640 1 --2- 172.21.15.164:0/2279729516 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f90f80b0230 0x7f90f8071cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:27.720 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.713+0000 7f90fe333640 1 -- 172.21.15.164:0/2279729516 >> 172.21.15.164:0/2279729516 conn(0x7f90f80b8ca0 msgr2=0x7f90f813f200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:58:27.720 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.717+0000 7f90fe333640 1 -- 172.21.15.164:0/2279729516 shutdown_connections 2024-03-23T14:58:27.720 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.717+0000 7f90fe333640 1 -- 172.21.15.164:0/2279729516 wait complete. 2024-03-23T14:58:27.738 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-23T14:58:27.739 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-23T14:58:27.859 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.853+0000 7f838c04a640 1 Processor -- start 2024-03-23T14:58:27.859 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.853+0000 7f838c04a640 1 -- start start 2024-03-23T14:58:27.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.857+0000 7f838c04a640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f838409def0 0x7f83840b45e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:27.860 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.857+0000 7f838c04a640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f838406d650 con 0x7f83840b4b20 2024-03-23T14:58:27.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.857+0000 7f838c04a640 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f838406cd90 con 0x7f83840add50 2024-03-23T14:58:27.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.857+0000 7f838c04a640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f83840af4a0 con 0x7f838409def0 2024-03-23T14:58:27.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.857+0000 7f8389de6640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f838409def0 0x7f83840b45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:27.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.857+0000 7f838a5e7640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f83840b4b20 0x7f838406c680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:50214/0 (socket says 172.21.15.164:50214) 2024-03-23T14:58:27.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.857+0000 7f838a5e7640 1 -- 172.21.15.164:0/1420531570 learned_addr learned my addr 172.21.15.164:0/1420531570 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-03-23T14:58:27.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.857+0000 7f8389de6640 1 -- 172.21.15.164:0/1420531570 >> v1:172.21.15.173:6789/0 conn(0x7f83840add50 legacy=0x7f838406bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-23T14:58:27.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.857+0000 7f8389de6640 1 -- 172.21.15.164:0/1420531570 >> v1:172.21.15.164:6789/0 conn(0x7f83840b4b20 legacy=0x7f838406c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:27.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.857+0000 7f8389de6640 1 -- 172.21.15.164:0/1420531570 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f838409e2d0 con 0x7f838409def0 2024-03-23T14:58:27.861 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.857+0000 7f8388de4640 1 -- 172.21.15.164:0/1420531570 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 645280622 0 0) 0x7f838406d650 con 0x7f83840b4b20 2024-03-23T14:58:27.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.857+0000 7f8389de6640 1 --2- 172.21.15.164:0/1420531570 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f838409def0 0x7f83840b45e0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f83800097c0 tx=0x7f838001c340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=593176e91826fc70 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:58:27.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.857+0000 7f8388de4640 1 -- 172.21.15.164:0/1420531570 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83800ab600 con 0x7f838409def0 2024-03-23T14:58:27.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.857+0000 7f8388de4640 1 -- 172.21.15.164:0/1420531570 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f83800a7050 con 0x7f838409def0 2024-03-23T14:58:27.863 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.857+0000 7f8388de4640 1 -- 172.21.15.164:0/1420531570 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83800ab9a0 con 0x7f838409def0 2024-03-23T14:58:27.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.857+0000 7f838c04a640 1 -- 172.21.15.164:0/1420531570 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f838409def0 msgr2=0x7f83840b45e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:27.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.857+0000 7f838c04a640 1 --2- 172.21.15.164:0/1420531570 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f838409def0 0x7f83840b45e0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f83800097c0 tx=0x7f838001c340 comp rx=0 tx=0).stop 2024-03-23T14:58:27.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.857+0000 7f838c04a640 1 -- 172.21.15.164:0/1420531570 shutdown_connections 2024-03-23T14:58:27.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.857+0000 7f838c04a640 1 --2- 172.21.15.164:0/1420531570 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f838409def0 0x7f83840b45e0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:27.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.857+0000 7f838c04a640 1 -- 172.21.15.164:0/1420531570 >> 172.21.15.164:0/1420531570 conn(0x7f83840b8ca0 msgr2=0x7f83840b90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:58:27.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.861+0000 7f838c04a640 1 -- 172.21.15.164:0/1420531570 shutdown_connections 2024-03-23T14:58:27.864 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.861+0000 7f838c04a640 1 -- 172.21.15.164:0/1420531570 wait complete. 2024-03-23T14:58:27.866 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.861+0000 7f838c04a640 1 Processor -- start 2024-03-23T14:58:27.866 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.861+0000 7f838c04a640 1 -- start start 2024-03-23T14:58:27.866 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.861+0000 7f838c04a640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f838409def0 0x7f83841e35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:27.867 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.861+0000 7f838c04a640 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f83840add50 0x7f838409aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:27.867 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.861+0000 7f838c04a640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f83840b4b20 0x7f83840b1e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:27.867 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.861+0000 7f838c04a640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f838414d2c0 con 0x7f838409def0 2024-03-23T14:58:27.867 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.861+0000 7f838c04a640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f83841e3150 con 0x7f83840b4b20 2024-03-23T14:58:27.867 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.861+0000 7f838c04a640 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 -- 0x7f83840b2e40 con 0x7f83840add50 2024-03-23T14:58:27.867 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.861+0000 7f8389de6640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f838409def0 0x7f83841e35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:27.867 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.861+0000 7f8389de6640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f838409def0 0x7f83841e35e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:42270/0 (socket says 172.21.15.164:42270) 2024-03-23T14:58:27.867 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.861+0000 7f8389de6640 1 -- 172.21.15.164:0/2848439174 learned_addr learned my addr 172.21.15.164:0/2848439174 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-03-23T14:58:27.867 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.861+0000 7f838a5e7640 1 --2- 172.21.15.164:0/2848439174 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f83840b4b20 0x7f83840b1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:27.868 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.861+0000 7f8389de6640 1 -- 172.21.15.164:0/2848439174 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f83840add50 msgr2=0x7f838409aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:27.868 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.861+0000 7f83895e5640 1 --2- 172.21.15.164:0/2848439174 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f83840add50 0x7f838409aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:27.868 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.861+0000 7f8389de6640 1 --2- 172.21.15.164:0/2848439174 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f83840add50 0x7f838409aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:27.868 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.861+0000 7f8389de6640 1 -- 172.21.15.164:0/2848439174 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f83840b4b20 msgr2=0x7f83840b1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:27.868 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.861+0000 7f8389de6640 1 --2- 172.21.15.164:0/2848439174 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f83840b4b20 0x7f83840b1e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:27.868 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.861+0000 7f8389de6640 1 -- 172.21.15.164:0/2848439174 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8380009420 con 0x7f838409def0 2024-03-23T14:58:27.868 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.861+0000 7f8389de6640 1 --2- 172.21.15.164:0/2848439174 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f838409def0 0x7f83841e35e0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f83800097c0 tx=0x7f838001ab80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:58:27.868 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.861+0000 7f837a7fc640 1 -- 172.21.15.164:0/2848439174 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8380004230 con 0x7f838409def0 2024-03-23T14:58:27.868 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.861+0000 7f837a7fc640 1 -- 172.21.15.164:0/2848439174 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f83800ace90 con 0x7f838409def0 2024-03-23T14:58:27.868 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.861+0000 7f838c04a640 1 -- 172.21.15.164:0/2848439174 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f838414d780 con 0x7f838409def0 2024-03-23T14:58:27.869 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.861+0000 7f837a7fc640 1 -- 172.21.15.164:0/2848439174 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8380003620 con 0x7f838409def0 2024-03-23T14:58:27.869 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.861+0000 7f838c04a640 1 -- 172.21.15.164:0/2848439174 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83840b8740 con 0x7f838409def0 2024-03-23T14:58:27.869 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.861+0000 7f838c04a640 1 -- 172.21.15.164:0/2848439174 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83840b2ca0 con 0x7f838409def0 2024-03-23T14:58:27.870 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.865+0000 7f837a7fc640 1 -- 172.21.15.164:0/2848439174 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 86727+0+0 (secure 0 0 0) 0x7f83800b2040 con 0x7f838409def0 2024-03-23T14:58:27.870 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.865+0000 7f837a7fc640 1 --2- 172.21.15.164:0/2848439174 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f83680672a0 0x7f8368069760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-23T14:58:27.870 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.865+0000 7f83895e5640 1 --2- 172.21.15.164:0/2848439174 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f83680672a0 0x7f8368069760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-23T14:58:27.870 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.865+0000 7f837a7fc640 1 -- 172.21.15.164:0/2848439174 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4943+0+0 (secure 0 0 0) 0x7f838011f010 con 0x7f838409def0 2024-03-23T14:58:27.870 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.865+0000 7f83895e5640 1 --2- 172.21.15.164:0/2848439174 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f83680672a0 0x7f8368069760 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f83841e3b40 tx=0x7f8374009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-23T14:58:27.872 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:27.869+0000 7f837a7fc640 1 -- 172.21.15.164:0/2848439174 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+176632 (secure 0 0 0) 0x7f83840b2ca0 con 0x7f838409def0 2024-03-23T14:58:28.064 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:28.061+0000 7f838c04a640 1 -- 172.21.15.164:0/2848439174 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f83840b2ca0 con 0x7f838409def0 2024-03-23T14:58:28.064 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:28.061+0000 7f837a7fc640 1 -- 172.21.15.164:0/2848439174 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) ==== 94+0+2225 (secure 0 0 0) 0x7f83840b2ca0 con 0x7f838409def0 2024-03-23T14:58:28.065 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-03-23T14:58:28.065 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":15,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4619,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/1401585102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":1401585102},{"type":"v1","addr":"172.21.15.173:6833","nonce":1401585102}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":5253,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6835/4126916384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":4126916384},{"type":"v1","addr":"172.21.15.164:6835","nonce":4126916384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":15}],"filesystems":[]} 2024-03-23T14:58:28.065 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 15 2024-03-23T14:58:28.067 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:28.061+0000 7f838c04a640 1 -- 172.21.15.164:0/2848439174 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f83680672a0 msgr2=0x7f8368069760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:28.067 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:28.061+0000 7f838c04a640 1 --2- 172.21.15.164:0/2848439174 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f83680672a0 0x7f8368069760 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f83841e3b40 tx=0x7f8374009040 comp rx=0 tx=0).stop 2024-03-23T14:58:28.067 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:28.061+0000 7f838c04a640 1 -- 172.21.15.164:0/2848439174 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f838409def0 msgr2=0x7f83841e35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-23T14:58:28.067 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:28.061+0000 7f838c04a640 1 --2- 172.21.15.164:0/2848439174 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f838409def0 0x7f83841e35e0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f83800097c0 tx=0x7f838001ab80 comp rx=0 tx=0).stop 2024-03-23T14:58:28.067 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:28.061+0000 7f838c04a640 1 -- 172.21.15.164:0/2848439174 shutdown_connections 2024-03-23T14:58:28.067 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:28.061+0000 7f838c04a640 1 --2- 172.21.15.164:0/2848439174 >> [v2:172.21.15.164:6832/2139087985,v1:172.21.15.164:6833/2139087985] conn(0x7f83680672a0 0x7f8368069760 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:28.067 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:28.061+0000 7f838c04a640 1 --2- 172.21.15.164:0/2848439174 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f83840b4b20 0x7f83840b1e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:28.067 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:28.061+0000 7f838c04a640 1 --2- 172.21.15.164:0/2848439174 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f83840add50 0x7f838409aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:28.067 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:28.061+0000 7f838c04a640 1 --2- 172.21.15.164:0/2848439174 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f838409def0 0x7f83841e35e0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-23T14:58:28.068 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:28.061+0000 7f838c04a640 1 -- 172.21.15.164:0/2848439174 >> 172.21.15.164:0/2848439174 conn(0x7f83840b8ca0 msgr2=0x7f8384144970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-23T14:58:28.068 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:28.061+0000 7f838c04a640 1 -- 172.21.15.164:0/2848439174 shutdown_connections 2024-03-23T14:58:28.068 INFO:teuthology.orchestra.run.smithi164.stderr:2024-03-23T14:58:28.061+0000 7f838c04a640 1 -- 172.21.15.164:0/2848439174 wait complete. 2024-03-23T14:58:28.084 DEBUG:tasks.cephfs.filesystem:already dead... 2024-03-23T14:58:28.084 INFO:teuthology.misc:Shutting down mds daemons... 2024-03-23T14:58:28.084 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-03-23T14:58:28.084 INFO:teuthology.orchestra.run:waiting for 300 2024-03-23T14:58:28.095 INFO:tasks.ceph.mds.a:Stopped 2024-03-23T14:58:28.095 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-03-23T14:58:28.095 INFO:teuthology.orchestra.run:waiting for 300 2024-03-23T14:58:28.110 INFO:tasks.ceph.mds.b:Stopped 2024-03-23T14:58:28.110 INFO:teuthology.misc:Shutting down osd daemons... 2024-03-23T14:58:28.110 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-03-23T14:58:28.111 INFO:teuthology.orchestra.run:waiting for 300 2024-03-23T14:58:28.910 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-23T14:58:28.914 DEBUG:teuthology.orchestra.run.smithi173:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-23T14:58:33.260 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~0s 2024-03-23T14:58:34.212 INFO:tasks.ceph.osd.0:Stopped 2024-03-23T14:58:34.213 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-03-23T14:58:34.213 INFO:teuthology.orchestra.run:waiting for 300 2024-03-23T14:58:34.266 INFO:tasks.ceph.osd.1:Stopped 2024-03-23T14:58:34.266 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-03-23T14:58:34.266 INFO:teuthology.orchestra.run:waiting for 300 2024-03-23T14:58:34.352 INFO:tasks.ceph.osd.2:Stopped 2024-03-23T14:58:34.352 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-03-23T14:58:34.352 INFO:teuthology.orchestra.run:waiting for 300 2024-03-23T14:58:39.168 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~0s 2024-03-23T14:58:39.168 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 has been restored 2024-03-23T14:58:40.454 INFO:tasks.ceph.osd.3:Stopped 2024-03-23T14:58:40.454 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-03-23T14:58:40.454 INFO:teuthology.orchestra.run:waiting for 300 2024-03-23T14:58:40.510 INFO:tasks.ceph.osd.4:Stopped 2024-03-23T14:58:40.510 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-03-23T14:58:40.510 INFO:teuthology.orchestra.run:waiting for 300 2024-03-23T14:58:40.583 INFO:tasks.ceph.osd.5:Stopped 2024-03-23T14:58:40.584 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-03-23T14:58:40.584 INFO:teuthology.orchestra.run:waiting for 300 2024-03-23T14:58:44.773 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~0s 2024-03-23T14:58:44.774 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 has been restored 2024-03-23T14:58:46.685 INFO:tasks.ceph.osd.6:Stopped 2024-03-23T14:58:46.685 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-03-23T14:58:46.686 INFO:teuthology.orchestra.run:waiting for 300 2024-03-23T14:58:46.746 INFO:tasks.ceph.osd.7:Stopped 2024-03-23T14:58:46.746 INFO:teuthology.misc:Shutting down mgr daemons... 2024-03-23T14:58:46.746 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-03-23T14:58:46.747 INFO:teuthology.orchestra.run:waiting for 300 2024-03-23T14:58:46.776 INFO:tasks.ceph.mgr.y:Stopped 2024-03-23T14:58:46.776 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-03-23T14:58:46.776 INFO:teuthology.orchestra.run:waiting for 300 2024-03-23T14:58:46.804 INFO:tasks.ceph.mgr.x:Stopped 2024-03-23T14:58:46.805 INFO:teuthology.misc:Shutting down mon daemons... 2024-03-23T14:58:46.805 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-03-23T14:58:46.805 INFO:teuthology.orchestra.run:waiting for 300 2024-03-23T14:58:46.832 INFO:tasks.ceph.mon.a:Stopped 2024-03-23T14:58:46.832 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-03-23T14:58:46.832 INFO:teuthology.orchestra.run:waiting for 300 2024-03-23T14:58:46.860 INFO:tasks.ceph.mon.b:Stopped 2024-03-23T14:58:46.861 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-03-23T14:58:46.861 INFO:teuthology.orchestra.run:waiting for 300 2024-03-23T14:58:46.919 INFO:tasks.ceph.mon.c:Stopped 2024-03-23T14:58:46.919 INFO:tasks.ceph:Checking cluster log for badness... 2024-03-23T14:58:46.919 DEBUG:teuthology.orchestra.run.smithi164:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | head -n 1 2024-03-23T14:58:46.936 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi164.front.sepia.ceph.com 2024-03-23T14:58:46.937 DEBUG:teuthology.orchestra.run.smithi164:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-03-23T14:58:47.143 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi164.front.sepia.ceph.com 2024-03-23T14:58:47.143 DEBUG:teuthology.orchestra.run.smithi164:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-03-23T14:58:47.358 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi164.front.sepia.ceph.com 2024-03-23T14:58:47.358 DEBUG:teuthology.orchestra.run.smithi164:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-03-23T14:58:47.550 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi164.front.sepia.ceph.com 2024-03-23T14:58:47.550 DEBUG:teuthology.orchestra.run.smithi164:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-03-23T14:58:47.725 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi173.front.sepia.ceph.com 2024-03-23T14:58:47.726 DEBUG:teuthology.orchestra.run.smithi173:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-03-23T14:58:47.942 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi173.front.sepia.ceph.com 2024-03-23T14:58:47.943 DEBUG:teuthology.orchestra.run.smithi173:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-03-23T14:58:48.133 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi173.front.sepia.ceph.com 2024-03-23T14:58:48.134 DEBUG:teuthology.orchestra.run.smithi173:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-03-23T14:58:48.317 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi173.front.sepia.ceph.com 2024-03-23T14:58:48.318 DEBUG:teuthology.orchestra.run.smithi173:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-03-23T14:58:48.509 INFO:tasks.ceph:Archiving mon data... 2024-03-23T14:58:48.510 DEBUG:teuthology.misc:Transferring archived files from smithi164:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/pdonnell-2024-03-23_09:56:53-fs-wip-batrick-testing-20240323.003144-squid-distro-default-smithi/7618407/data/mon.a.tgz 2024-03-23T14:58:48.510 DEBUG:teuthology.orchestra.run.smithi164:> mktemp 2024-03-23T14:58:48.516 INFO:teuthology.orchestra.run.smithi164.stdout:/tmp/tmp.G7LB69MVsv 2024-03-23T14:58:48.516 DEBUG:teuthology.orchestra.run.smithi164:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.G7LB69MVsv 2024-03-23T14:58:48.630 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod 0666 /tmp/tmp.G7LB69MVsv 2024-03-23T14:58:48.692 DEBUG:teuthology.orchestra.remote:smithi164:/tmp/tmp.G7LB69MVsv is 274KB 2024-03-23T14:58:48.753 DEBUG:teuthology.orchestra.run.smithi164:> rm -fr /tmp/tmp.G7LB69MVsv 2024-03-23T14:58:48.758 DEBUG:teuthology.misc:Transferring archived files from smithi173:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/pdonnell-2024-03-23_09:56:53-fs-wip-batrick-testing-20240323.003144-squid-distro-default-smithi/7618407/data/mon.b.tgz 2024-03-23T14:58:48.758 DEBUG:teuthology.orchestra.run.smithi173:> mktemp 2024-03-23T14:58:48.764 INFO:teuthology.orchestra.run.smithi173.stdout:/tmp/tmp.S4chpqxcZL 2024-03-23T14:58:48.764 DEBUG:teuthology.orchestra.run.smithi173:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.S4chpqxcZL 2024-03-23T14:58:48.887 DEBUG:teuthology.orchestra.run.smithi173:> sudo chmod 0666 /tmp/tmp.S4chpqxcZL 2024-03-23T14:58:48.949 DEBUG:teuthology.orchestra.remote:smithi173:/tmp/tmp.S4chpqxcZL is 365KB 2024-03-23T14:58:48.967 DEBUG:teuthology.orchestra.run.smithi173:> rm -fr /tmp/tmp.S4chpqxcZL 2024-03-23T14:58:48.971 DEBUG:teuthology.misc:Transferring archived files from smithi173:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/pdonnell-2024-03-23_09:56:53-fs-wip-batrick-testing-20240323.003144-squid-distro-default-smithi/7618407/data/mon.c.tgz 2024-03-23T14:58:48.972 DEBUG:teuthology.orchestra.run.smithi173:> mktemp 2024-03-23T14:58:49.018 INFO:teuthology.orchestra.run.smithi173.stdout:/tmp/tmp.PFHY7nnE8T 2024-03-23T14:58:49.018 DEBUG:teuthology.orchestra.run.smithi173:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.PFHY7nnE8T 2024-03-23T14:58:49.144 DEBUG:teuthology.orchestra.run.smithi173:> sudo chmod 0666 /tmp/tmp.PFHY7nnE8T 2024-03-23T14:58:49.205 DEBUG:teuthology.orchestra.remote:smithi173:/tmp/tmp.PFHY7nnE8T is 365KB 2024-03-23T14:58:49.222 DEBUG:teuthology.orchestra.run.smithi173:> rm -fr /tmp/tmp.PFHY7nnE8T 2024-03-23T14:58:49.227 INFO:tasks.ceph:Cleaning ceph cluster... 2024-03-23T14:58:49.227 DEBUG:teuthology.orchestra.run.smithi164:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-03-23T14:58:49.229 DEBUG:teuthology.orchestra.run.smithi173:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-03-23T14:58:49.289 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-03-23T14:58:49.331 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-03-23T14:58:49.331 INFO:tasks.ceph:Archiving crash dumps... 2024-03-23T14:58:49.333 DEBUG:teuthology.misc:Transferring archived files from smithi164:/var/lib/ceph/crash to /home/teuthworker/archive/pdonnell-2024-03-23_09:56:53-fs-wip-batrick-testing-20240323.003144-squid-distro-default-smithi/7618407/remote/smithi164/crash 2024-03-23T14:58:49.334 DEBUG:teuthology.orchestra.run.smithi164:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-03-23T14:58:49.348 DEBUG:teuthology.misc:Transferring archived files from smithi173:/var/lib/ceph/crash to /home/teuthworker/archive/pdonnell-2024-03-23_09:56:53-fs-wip-batrick-testing-20240323.003144-squid-distro-default-smithi/7618407/remote/smithi173/crash 2024-03-23T14:58:49.349 DEBUG:teuthology.orchestra.run.smithi173:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-03-23T14:58:49.386 INFO:tasks.ceph:Shutting down logrotate 2024-03-23T14:58:49.386 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-23T14:58:49.393 DEBUG:teuthology.orchestra.run.smithi173:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-23T14:58:49.464 DEBUG:teuthology.orchestra.run.smithi164:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-03-23T14:58:49.495 DEBUG:teuthology.orchestra.run.smithi173:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-03-23T14:58:49.518 INFO:tasks.ceph:Compressing logs... 2024-03-23T14:58:49.518 DEBUG:teuthology.orchestra.run.smithi164:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-23T14:58:49.541 DEBUG:teuthology.orchestra.run.smithi173:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-23T14:58:49.568 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35202.log 2024-03-23T14:58:49.569 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36103.log 2024-03-23T14:58:49.569 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.35202.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34810.log 2024-03-23T14:58:49.570 INFO:teuthology.orchestra.run.smithi164.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.35202.log.gz 2024-03-23T14:58:49.571 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.36103.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36334.log 2024-03-23T14:58:49.571 INFO:teuthology.orchestra.run.smithi164.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.36103.log.gz 2024-03-23T14:58:49.571 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33900.log 2024-03-23T14:58:49.571 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.34810.log: /var/log/ceph/ceph-client.admin.36334.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37705.log 2024-03-23T14:58:49.573 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.33900.log: gzip -5 89.2% --verbose -- replaced with /var/log/ceph/ceph-client.admin.36334.log.gz -- 2024-03-23T14:58:49.573 INFO:teuthology.orchestra.run.smithi164.stderr: /var/log/ceph/ceph-mgr.y.log 2024-03-23T14:58:49.573 INFO:teuthology.orchestra.run.smithi164.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.33900.log.gz 2024-03-23T14:58:49.573 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5/var/log/ceph/ceph-client.admin.37705.log: --verbose -- /var/log/ceph/ceph-client.admin.33374.log 2024-03-23T14:58:49.573 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-mgr.y.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.37705.log.gz 2024-03-23T14:58:49.573 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37386.log 2024-03-23T14:58:49.573 INFO:teuthology.orchestra.run.smithi164.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.34810.log.gz 2024-03-23T14:58:49.573 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.33374.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36711.log 2024-03-23T14:58:49.574 INFO:teuthology.orchestra.run.smithi164.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.33374.log.gz 2024-03-23T14:58:49.574 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.37386.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35046.log 2024-03-23T14:58:49.577 INFO:teuthology.orchestra.run.smithi164.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.37386.log.gz 2024-03-23T14:58:49.577 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.36711.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36025.log 2024-03-23T14:58:49.577 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.35046.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35943.log 2024-03-23T14:58:49.577 INFO:teuthology.orchestra.run.smithi164.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.36711.log.gz 2024-03-23T14:58:49.578 INFO:teuthology.orchestra.run.smithi164.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.35046.log.gz 2024-03-23T14:58:49.578 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.36025.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36051.log 2024-03-23T14:58:49.578 INFO:teuthology.orchestra.run.smithi164.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.36025.log.gz 2024-03-23T14:58:49.578 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.35943.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34026.log 2024-03-23T14:58:49.578 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.36051.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36516.log 2024-03-23T14:58:49.578 INFO:teuthology.orchestra.run.smithi164.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.35943.log.gz 2024-03-23T14:58:49.578 INFO:teuthology.orchestra.run.smithi164.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.36051.log.gz 2024-03-23T14:58:49.578 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.34026.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36077.log 2024-03-23T14:58:49.578 INFO:teuthology.orchestra.run.smithi164.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.34026.log.gz 2024-03-23T14:58:49.579 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.36516.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33504.log 2024-03-23T14:58:49.579 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.36077.log: 88.1%gzip -- replaced with /var/log/ceph/ceph-client.admin.36516.log.gz -5 2024-03-23T14:58:49.579 INFO:teuthology.orchestra.run.smithi164.stderr: --verbose -- /var/log/ceph/ceph-client.admin.36845.log 2024-03-23T14:58:49.579 INFO:teuthology.orchestra.run.smithi164.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.36077.log.gz 2024-03-23T14:58:49.579 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.33504.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36767.log 2024-03-23T14:58:49.579 INFO:teuthology.orchestra.run.smithi164.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.33504.log.gz 2024-03-23T14:58:49.579 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.36845.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34472.log 2024-03-23T14:58:49.584 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.36767.log: gzip 88.5% -5 -- replaced with /var/log/ceph/ceph-client.admin.36845.log.gz --verbose 2024-03-23T14:58:49.584 INFO:teuthology.orchestra.run.smithi164.stderr: -- /var/log/ceph/ceph-client.admin.33640.log 2024-03-23T14:58:49.584 INFO:teuthology.orchestra.run.smithi164.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.36767.log.gz 2024-03-23T14:58:49.584 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.34472.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37757.log 2024-03-23T14:58:49.584 INFO:teuthology.orchestra.run.smithi164.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.34472.log.gz 2024-03-23T14:58:49.584 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.33640.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36568.log 2024-03-23T14:58:49.584 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.37757.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-03-23T14:58:49.585 INFO:teuthology.orchestra.run.smithi164.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.33640.log.gz 2024-03-23T14:58:49.585 INFO:teuthology.orchestra.run.smithi164.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.37757.log.gz 2024-03-23T14:58:49.585 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.36568.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38095.log 2024-03-23T14:58:49.585 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-osd.0.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.36568.log.gz 2024-03-23T14:58:49.585 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38173.log 2024-03-23T14:58:49.585 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.38095.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36360.log 2024-03-23T14:58:49.585 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.38173.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35860.log 2024-03-23T14:58:49.585 INFO:teuthology.orchestra.run.smithi164.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.38095.log.gz 2024-03-23T14:58:49.585 INFO:teuthology.orchestra.run.smithi164.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.38173.log.gz 2024-03-23T14:58:49.586 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.36360.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37090.log 2024-03-23T14:58:49.586 INFO:teuthology.orchestra.run.smithi164.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.36360.log.gz 2024-03-23T14:58:49.586 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35729.log 2024-03-23T14:58:49.586 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.35860.log: /var/log/ceph/ceph-client.admin.37090.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36685.log 2024-03-23T14:58:49.586 INFO:teuthology.orchestra.run.smithi164.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.35860.log.gz 2024-03-23T14:58:49.586 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.35729.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37256.log 2024-03-23T14:58:49.586 INFO:teuthology.orchestra.run.smithi164.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.37090.log.gz 2024-03-23T14:58:49.586 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.36685.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37667.log 2024-03-23T14:58:49.586 INFO:teuthology.orchestra.run.smithi164.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.35729.log.gz 2024-03-23T14:58:49.587 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.37256.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.36685.log.gz 2024-03-23T14:58:49.587 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34784.log 2024-03-23T14:58:49.587 INFO:teuthology.orchestra.run.smithi164.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.37256.log.gz 2024-03-23T14:58:49.587 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.37667.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36308.log 2024-03-23T14:58:49.603 INFO:teuthology.orchestra.run.smithi173.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27717.log 2024-03-23T14:58:49.603 INFO:teuthology.orchestra.run.smithi173.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27693.log 2024-03-23T14:58:49.603 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/ceph-client.admin.27717.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-03-23T14:58:49.603 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/ceph-client.admin.27693.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.27717.log.gz 2024-03-23T14:58:49.603 INFO:teuthology.orchestra.run.smithi173.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.27693.log.gz 2024-03-23T14:58:49.604 INFO:teuthology.orchestra.run.smithi173.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-03-23T14:58:49.604 INFO:teuthology.orchestra.run.smithi173.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27549.log 2024-03-23T14:58:49.604 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/ceph-osd.7.log: /var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27597.log 2024-03-23T14:58:49.604 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/ceph-client.admin.27549.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-03-23T14:58:49.604 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/ceph-client.admin.27597.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-03-23T14:58:49.604 INFO:teuthology.orchestra.run.smithi173.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.27597.log.gz 2024-03-23T14:58:49.604 INFO:teuthology.orchestra.run.smithi173.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.27549.log.gz 2024-03-23T14:58:49.605 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27645.log 2024-03-23T14:58:49.605 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-03-23T14:58:49.605 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/ceph-client.admin.27645.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27669.log 87.3% 2024-03-23T14:58:49.605 INFO:teuthology.orchestra.run.smithi173.stderr: -- replaced with /var/log/ceph/ceph-client.admin.27645.log.gz 2024-03-23T14:58:49.605 INFO:teuthology.orchestra.run.smithi173.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27573.log 2024-03-23T14:58:49.605 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/ceph.log: /var/log/ceph/ceph-client.admin.27669.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-03-23T14:58:49.605 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/ceph-client.admin.27573.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.27669.log.gz 2024-03-23T14:58:49.605 INFO:teuthology.orchestra.run.smithi173.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-03-23T14:58:49.606 INFO:teuthology.orchestra.run.smithi173.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.27573.log.gz 2024-03-23T14:58:49.606 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27621.log 2024-03-23T14:58:49.606 INFO:teuthology.orchestra.run.smithi173.stderr: 91.8% -- replaced with /var/log/ceph/ceph.log.gz/var/log/ceph/ceph.audit.log: 2024-03-23T14:58:49.606 INFO:teuthology.orchestra.run.smithi173.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-03-23T14:58:49.606 INFO:teuthology.orchestra.run.smithi173.stderr:gzip/var/log/ceph/ceph-client.admin.27621.log: -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-03-23T14:58:49.606 INFO:teuthology.orchestra.run.smithi173.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.27621.log.gz 92.5% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-03-23T14:58:49.606 INFO:teuthology.orchestra.run.smithi173.stderr: 2024-03-23T14:58:49.606 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-03-23T14:58:49.607 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/ceph-mon.b.log: 93.6% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-03-23T14:58:49.614 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/ceph-osd.4.log: 90.4% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-03-23T14:58:49.630 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.34784.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34219.log 2024-03-23T14:58:49.631 INFO:teuthology.orchestra.run.smithi164.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.37667.log.gz 2024-03-23T14:58:49.631 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.36308.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.34784.log.gz 2024-03-23T14:58:49.631 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35436.log 2024-03-23T14:58:49.631 INFO:teuthology.orchestra.run.smithi164.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.36308.log.gz 2024-03-23T14:58:49.631 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.34219.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37731.log 2024-03-23T14:58:49.631 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.35436.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36741.log 2024-03-23T14:58:49.631 INFO:teuthology.orchestra.run.smithi164.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.34219.log.gz 2024-03-23T14:58:49.632 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.37731.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36819.log 2024-03-23T14:58:49.632 INFO:teuthology.orchestra.run.smithi164.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.35436.log.gz 2024-03-23T14:58:49.632 INFO:teuthology.orchestra.run.smithi164.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.37731.log.gz 2024-03-23T14:58:49.632 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.36741.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35488.log 2024-03-23T14:58:49.632 INFO:teuthology.orchestra.run.smithi164.stderr: 89.1%gzip -- replaced with /var/log/ceph/ceph-client.admin.36741.log.gz 2024-03-23T14:58:49.632 INFO:teuthology.orchestra.run.smithi164.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.34420.log 2024-03-23T14:58:49.632 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.36819.log: /var/log/ceph/ceph-client.admin.35488.log: gzip -5 --verbose -- 88.0% /var/log/ceph/ceph-client.admin.33926.log 2024-03-23T14:58:49.632 INFO:teuthology.orchestra.run.smithi164.stderr: -- replaced with /var/log/ceph/ceph-client.admin.36819.log.gz 2024-03-23T14:58:49.632 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.34420.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.35488.log.gz 2024-03-23T14:58:49.633 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37146.log 2024-03-23T14:58:49.633 INFO:teuthology.orchestra.run.smithi164.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.34420.log.gz 2024-03-23T14:58:49.633 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.33926.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38199.log 2024-03-23T14:58:49.633 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.37146.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.33926.log.gz 2024-03-23T14:58:49.633 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37360.log 2024-03-23T14:58:49.633 INFO:teuthology.orchestra.run.smithi164.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.37146.log.gz 2024-03-23T14:58:49.633 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.38199.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36490.log 2024-03-23T14:58:49.633 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35306.log 2024-03-23T14:58:49.633 INFO:teuthology.orchestra.run.smithi164.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.38199.log.gz 2024-03-23T14:58:49.634 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.37360.log: /var/log/ceph/ceph-client.admin.36490.log: gzip 87.8% -- replaced with /var/log/ceph/ceph-client.admin.37360.log.gz -5 2024-03-23T14:58:49.634 INFO:teuthology.orchestra.run.smithi164.stderr: --verbose -- /var/log/ceph/ceph-client.admin.34550.log 2024-03-23T14:58:49.634 INFO:teuthology.orchestra.run.smithi164.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.36490.log.gz 2024-03-23T14:58:49.634 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.35306.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36412.log 2024-03-23T14:58:49.634 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.34550.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.35306.log.gz 2024-03-23T14:58:49.634 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36793.log 2024-03-23T14:58:49.634 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.36412.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.34550.log.gz 2024-03-23T14:58:49.634 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-03-23T14:58:49.635 INFO:teuthology.orchestra.run.smithi164.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.36412.log.gz 2024-03-23T14:58:49.635 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.36793.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37809.log 2024-03-23T14:58:49.635 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37172.log 2024-03-23T14:58:49.635 INFO:teuthology.orchestra.run.smithi164.stderr: 88.8%/var/log/ceph/ceph-osd.2.log: -- replaced with /var/log/ceph/ceph-client.admin.36793.log.gz 2024-03-23T14:58:49.635 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.37809.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35997.log 2024-03-23T14:58:49.635 INFO:teuthology.orchestra.run.smithi164.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37809.log.gz 2024-03-23T14:58:49.635 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.37172.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37464.log 2024-03-23T14:58:49.635 INFO:teuthology.orchestra.run.smithi164.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.37172.log.gz 2024-03-23T14:58:49.635 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35384.log 2024-03-23T14:58:49.636 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.35997.log: /var/log/ceph/ceph-client.admin.37464.log: gzip -5 89.2% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.35997.log.gz 2024-03-23T14:58:49.636 INFO:teuthology.orchestra.run.smithi164.stderr: /var/log/ceph/ceph-client.admin.34235.log 2024-03-23T14:58:49.636 INFO:teuthology.orchestra.run.smithi164.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.37464.log.gz 2024-03-23T14:58:49.636 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.35384.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-03-23T14:58:49.636 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.34235.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35592.log 2024-03-23T14:58:49.636 INFO:teuthology.orchestra.run.smithi164.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.35384.log.gz 2024-03-23T14:58:49.636 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-mon.a.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.34235.log.gzgzip 2024-03-23T14:58:49.636 INFO:teuthology.orchestra.run.smithi164.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.35150.log 2024-03-23T14:58:49.636 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.35592.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34232.log 2024-03-23T14:58:49.637 INFO:teuthology.orchestra.run.smithi164.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.35592.log.gz 2024-03-23T14:58:49.637 INFO:teuthology.orchestra.run.smithi164.stderr:gzip/var/log/ceph/ceph-client.admin.35150.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.33426.log 2024-03-23T14:58:49.637 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.34232.log: gzip 89.5% -- replaced with /var/log/ceph/ceph-client.admin.35150.log.gz 2024-03-23T14:58:49.637 INFO:teuthology.orchestra.run.smithi164.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.36897.log 2024-03-23T14:58:49.637 INFO:teuthology.orchestra.run.smithi164.stderr: 89.1%/var/log/ceph/ceph-client.admin.33426.log: -- replaced with /var/log/ceph/ceph-client.admin.34232.log.gz 2024-03-23T14:58:49.637 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33692.log 2024-03-23T14:58:49.637 INFO:teuthology.orchestra.run.smithi164.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.33426.log.gz 2024-03-23T14:58:49.637 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.36897.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36596.log 2024-03-23T14:58:49.637 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.33692.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.36897.log.gz 2024-03-23T14:58:49.638 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36438.log 2024-03-23T14:58:49.638 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.36596.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.33692.log.gz 2024-03-23T14:58:49.638 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36464.log 2024-03-23T14:58:49.638 INFO:teuthology.orchestra.run.smithi164.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.36596.log.gz 2024-03-23T14:58:49.638 INFO:teuthology.orchestra.run.smithi164.stderr:gzip/var/log/ceph/ceph-client.admin.36438.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.37991.log 2024-03-23T14:58:49.638 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.36464.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34706.log 2024-03-23T14:58:49.638 INFO:teuthology.orchestra.run.smithi164.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.36438.log.gz 2024-03-23T14:58:49.638 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.37991.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35280.log 2024-03-23T14:58:49.638 INFO:teuthology.orchestra.run.smithi164.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.36464.log.gz 2024-03-23T14:58:49.639 INFO:teuthology.orchestra.run.smithi164.stderr: 89.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27519.log 2024-03-23T14:58:49.639 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.34706.log: -- replaced with /var/log/ceph/ceph-client.admin.37991.log.gz 2024-03-23T14:58:49.639 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33848.log 2024-03-23T14:58:49.639 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.35280.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.34706.log.gz 2024-03-23T14:58:49.639 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.27519.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.35280.log.gz 2024-03-23T14:58:49.639 INFO:teuthology.orchestra.run.smithi164.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.27519.log.gz 2024-03-23T14:58:49.639 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36659.log 2024-03-23T14:58:49.639 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.33848.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33796.log 2024-03-23T14:58:49.640 INFO:teuthology.orchestra.run.smithi164.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.33848.log.gz/var/log/ceph/ceph-client.admin.36659.log: 2024-03-23T14:58:49.640 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36207.log 2024-03-23T14:58:49.640 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.33796.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.36659.log.gz 2024-03-23T14:58:49.640 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37861.log 2024-03-23T14:58:49.640 INFO:teuthology.orchestra.run.smithi164.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.33796.log.gz 2024-03-23T14:58:49.640 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-03-23T14:58:49.640 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.36207.log: /var/log/ceph/ceph-client.admin.37861.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35891.log 2024-03-23T14:58:49.640 INFO:teuthology.orchestra.run.smithi164.stderr: 88.7%/var/log/ceph/ceph-osd.3.log: gzip -- replaced with /var/log/ceph/ceph-client.admin.37861.log.gz -5 --verbose -- /var/log/ceph/ceph-client.admin.35462.log 2024-03-23T14:58:49.641 INFO:teuthology.orchestra.run.smithi164.stderr: 2024-03-23T14:58:49.641 INFO:teuthology.orchestra.run.smithi164.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.36207.log.gz 2024-03-23T14:58:49.641 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.35891.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27410.log 2024-03-23T14:58:49.641 INFO:teuthology.orchestra.run.smithi164.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.35891.log.gz 2024-03-23T14:58:49.641 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.35462.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36129.log 2024-03-23T14:58:49.641 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.27410.log: 89.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34237.log 2024-03-23T14:58:49.641 INFO:teuthology.orchestra.run.smithi164.stderr: -- replaced with /var/log/ceph/ceph-client.admin.35462.log.gz 2024-03-23T14:58:49.641 INFO:teuthology.orchestra.run.smithi164.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.27410.log.gz 2024-03-23T14:58:49.641 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.36129.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.36129.log.gz 2024-03-23T14:58:49.642 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34031.log 2024-03-23T14:58:49.642 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.34237.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33770.log 2024-03-23T14:58:49.642 INFO:teuthology.orchestra.run.smithi164.stderr: 88.2%/var/log/ceph/ceph-client.admin.34031.log: -- replaced with /var/log/ceph/ceph-client.admin.34237.log.gz 2024-03-23T14:58:49.642 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36155.log 2024-03-23T14:58:49.642 INFO:teuthology.orchestra.run.smithi164.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.34031.log.gz 2024-03-23T14:58:49.642 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.33770.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.33770.log.gz 2024-03-23T14:58:49.642 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34654.log 2024-03-23T14:58:49.642 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.36155.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33614.log 2024-03-23T14:58:49.643 INFO:teuthology.orchestra.run.smithi164.stderr: 89.3%/var/log/ceph/ceph-client.admin.34654.log: -- replaced with /var/log/ceph/ceph-client.admin.36155.log.gzgzip 2024-03-23T14:58:49.643 INFO:teuthology.orchestra.run.smithi164.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.34524.log 2024-03-23T14:58:49.643 INFO:teuthology.orchestra.run.smithi164.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.34654.log.gz 2024-03-23T14:58:49.643 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.33614.log: 91.6% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-03-23T14:58:49.643 INFO:teuthology.orchestra.run.smithi164.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.33614.log.gz 2024-03-23T14:58:49.643 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35540.log 2024-03-23T14:58:49.643 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.34524.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33558.log 2024-03-23T14:58:49.643 INFO:teuthology.orchestra.run.smithi164.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.34524.log.gz 2024-03-23T14:58:49.644 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36624.log 2024-03-23T14:58:49.644 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.35540.log: gzip -5/var/log/ceph/ceph-client.admin.33558.log: --verbose -- /var/log/ceph/ceph-osd.1.log 2024-03-23T14:58:49.644 INFO:teuthology.orchestra.run.smithi164.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.35540.log.gz/var/log/ceph/ceph-client.admin.36624.log: 2024-03-23T14:58:49.644 INFO:teuthology.orchestra.run.smithi164.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.33558.log.gz 2024-03-23T14:58:49.644 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35566.log 2024-03-23T14:58:49.644 INFO:teuthology.orchestra.run.smithi164.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.36624.log.gz 2024-03-23T14:58:49.644 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34028.log 2024-03-23T14:58:49.644 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33230.log 2024-03-23T14:58:49.645 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.35566.log: /var/log/ceph/ceph-client.admin.34028.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36233.log 2024-03-23T14:58:49.645 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.33230.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.35566.log.gz 2024-03-23T14:58:49.645 INFO:teuthology.orchestra.run.smithi164.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.34028.log.gz 2024-03-23T14:58:49.645 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34238.log 2024-03-23T14:58:49.645 INFO:teuthology.orchestra.run.smithi164.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.33230.log.gz 2024-03-23T14:58:49.645 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.36233.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38017.log 2024-03-23T14:58:49.645 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.34238.log: 88.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38225.log 2024-03-23T14:58:49.645 INFO:teuthology.orchestra.run.smithi164.stderr: -- replaced with /var/log/ceph/ceph-client.admin.36233.log.gz 2024-03-23T14:58:49.645 INFO:teuthology.orchestra.run.smithi164.stderr: 89.3%/var/log/ceph/ceph-client.admin.38017.log: -- replaced with /var/log/ceph/ceph-client.admin.34238.log.gz 2024-03-23T14:58:49.646 INFO:teuthology.orchestra.run.smithi164.stderr:gzip 88.0% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.38017.log.gz -- 2024-03-23T14:58:49.646 INFO:teuthology.orchestra.run.smithi164.stderr: /var/log/ceph/ceph-client.admin.34240.log 2024-03-23T14:58:49.646 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.38225.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37965.log 2024-03-23T14:58:49.646 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.34240.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.38225.log.gz 2024-03-23T14:58:49.646 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35176.log 2024-03-23T14:58:49.646 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37516.log 2024-03-23T14:58:49.646 INFO:teuthology.orchestra.run.smithi164.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.34240.log.gz 2024-03-23T14:58:49.646 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.37965.log: /var/log/ceph/ceph-client.admin.35176.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.37965.log.gz 2024-03-23T14:58:49.647 INFO:teuthology.orchestra.run.smithi164.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.35176.log.gz 2024-03-23T14:58:49.647 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34236.log 2024-03-23T14:58:49.647 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.37516.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33328.log 2024-03-23T14:58:49.647 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.34236.log: 88.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37225.log 2024-03-23T14:58:49.647 INFO:teuthology.orchestra.run.smithi164.stderr: -- replaced with /var/log/ceph/ceph-client.admin.37516.log.gz 2024-03-23T14:58:49.647 INFO:teuthology.orchestra.run.smithi164.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.34236.log.gz 2024-03-23T14:58:49.647 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.33328.log: 89.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35625.log 2024-03-23T14:58:49.647 INFO:teuthology.orchestra.run.smithi164.stderr: -- replaced with /var/log/ceph/ceph-client.admin.33328.log.gz 2024-03-23T14:58:49.648 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.37225.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38147.log 2024-03-23T14:58:49.648 INFO:teuthology.orchestra.run.smithi164.stderr: 88.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35514.log 2024-03-23T14:58:49.648 INFO:teuthology.orchestra.run.smithi164.stderr: -- replaced with /var/log/ceph/ceph-client.admin.37225.log.gz 2024-03-23T14:58:49.648 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.35625.log: /var/log/ceph/ceph-client.admin.38147.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.35625.log.gz 2024-03-23T14:58:49.648 INFO:teuthology.orchestra.run.smithi164.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.38147.log.gz 2024-03-23T14:58:49.648 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35332.log 2024-03-23T14:58:49.648 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.35514.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37120.log 2024-03-23T14:58:49.648 INFO:teuthology.orchestra.run.smithi164.stderr: 88.9%/var/log/ceph/ceph-client.admin.35332.log: -- replaced with /var/log/ceph/ceph-client.admin.35514.log.gz 2024-03-23T14:58:49.648 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27645.log 2024-03-23T14:58:49.649 INFO:teuthology.orchestra.run.smithi164.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.35332.log.gz 2024-03-23T14:58:49.649 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.37120.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.37120.log.gz 2024-03-23T14:58:49.649 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33666.log 2024-03-23T14:58:49.649 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.27645.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-03-23T14:58:49.649 INFO:teuthology.orchestra.run.smithi164.stderr: 87.9%/var/log/ceph/ceph-client.admin.33666.log: -- replaced with /var/log/ceph/ceph-client.admin.27645.log.gz 2024-03-23T14:58:49.649 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34023.log 2024-03-23T14:58:49.649 INFO:teuthology.orchestra.run.smithi164.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.33666.log.gz 2024-03-23T14:58:49.649 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph.log: 85.3% -- replaced with /var/log/ceph/ceph.log.gz 2024-03-23T14:58:49.649 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36973.log 2024-03-23T14:58:49.650 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.34023.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34732.log 2024-03-23T14:58:49.650 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.36973.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.34023.log.gz 2024-03-23T14:58:49.650 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33530.log 2024-03-23T14:58:49.650 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33874.log 2024-03-23T14:58:49.650 INFO:teuthology.orchestra.run.smithi164.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.36973.log.gz/var/log/ceph/ceph-client.admin.34732.log: 2024-03-23T14:58:49.650 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.33530.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.33530.log.gz 89.2% 2024-03-23T14:58:49.650 INFO:teuthology.orchestra.run.smithi164.stderr: -- replaced with /var/log/ceph/ceph-client.admin.34732.log.gz 2024-03-23T14:58:49.650 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33478.log 2024-03-23T14:58:49.650 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.33874.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38301.log 2024-03-23T14:58:49.651 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.33478.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.33874.log.gz 2024-03-23T14:58:49.651 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34602.log 2024-03-23T14:58:49.651 INFO:teuthology.orchestra.run.smithi164.stderr: 89.5%/var/log/ceph/ceph-client.admin.38301.log: -- replaced with /var/log/ceph/ceph-client.admin.33478.log.gz 2024-03-23T14:58:49.651 INFO:teuthology.orchestra.run.smithi164.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.38301.log.gz 2024-03-23T14:58:49.651 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37887.log 2024-03-23T14:58:49.651 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.34602.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34758.log 2024-03-23T14:58:49.651 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.37887.log: gzip -5 89.2% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.34602.log.gz /var/log/ceph/ceph-client.admin.33718.log 2024-03-23T14:58:49.651 INFO:teuthology.orchestra.run.smithi164.stderr: 2024-03-23T14:58:49.651 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.34758.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.37887.log.gz 2024-03-23T14:58:49.672 INFO:teuthology.orchestra.run.smithi173.stderr: 92.0% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-03-23T14:58:49.675 INFO:teuthology.orchestra.run.smithi164.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.34758.log.gz 2024-03-23T14:58:49.675 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35410.log 2024-03-23T14:58:49.675 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.33718.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37308.log 2024-03-23T14:58:49.675 INFO:teuthology.orchestra.run.smithi164.stderr: 88.9%/var/log/ceph/ceph-client.admin.35410.log: -- replaced with /var/log/ceph/ceph-client.admin.33718.log.gz 2024-03-23T14:58:49.675 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35651.log 2024-03-23T14:58:49.675 INFO:teuthology.orchestra.run.smithi164.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.35410.log.gz 2024-03-23T14:58:49.675 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.37308.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.37308.log.gz 2024-03-23T14:58:49.676 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38069.log 2024-03-23T14:58:49.676 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35072.log 2024-03-23T14:58:49.676 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.35651.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34239.log 2024-03-23T14:58:49.676 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.38069.log: /var/log/ceph/ceph-client.admin.35072.log: 89.2% 89.2% -- replaced with /var/log/ceph/ceph-client.admin.38069.log.gz -- replaced with /var/log/ceph/ceph-client.admin.35651.log.gz 2024-03-23T14:58:49.676 INFO:teuthology.orchestra.run.smithi164.stderr: 2024-03-23T14:58:49.676 INFO:teuthology.orchestra.run.smithi164.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.35072.log.gz 2024-03-23T14:58:49.676 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35358.log 2024-03-23T14:58:49.676 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33744.log 2024-03-23T14:58:49.676 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.34239.log: gzip -5/var/log/ceph/ceph-client.admin.35358.log: --verbose -- /var/log/ceph/ceph-client.admin.35782.log 2024-03-23T14:58:49.677 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.33744.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.35358.log.gz 87.5% -- replaced with /var/log/ceph/ceph-client.admin.34239.log.gz 2024-03-23T14:58:49.677 INFO:teuthology.orchestra.run.smithi164.stderr: 2024-03-23T14:58:49.677 INFO:teuthology.orchestra.run.smithi164.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.33744.log.gz 2024-03-23T14:58:49.677 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37199.log 2024-03-23T14:58:49.677 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34030.log 2024-03-23T14:58:49.677 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.35782.log: /var/log/ceph/ceph-client.admin.37199.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33452.log 2024-03-23T14:58:49.677 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.34030.log: 88.7% 89.4% -- replaced with /var/log/ceph/ceph-client.admin.35782.log.gz 2024-03-23T14:58:49.677 INFO:teuthology.orchestra.run.smithi164.stderr: -- replaced with /var/log/ceph/ceph-client.admin.37199.log.gz 2024-03-23T14:58:49.678 INFO:teuthology.orchestra.run.smithi164.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.34030.log.gz 2024-03-23T14:58:49.678 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38251.log 2024-03-23T14:58:49.678 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37490.log 2024-03-23T14:58:49.678 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.33452.log: /var/log/ceph/ceph-client.admin.38251.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35834.log 2024-03-23T14:58:49.678 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.37490.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.38251.log.gz 2024-03-23T14:58:49.678 INFO:teuthology.orchestra.run.smithi164.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.33452.log.gz 2024-03-23T14:58:49.678 INFO:teuthology.orchestra.run.smithi164.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.37490.log.gz 2024-03-23T14:58:49.678 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36386.log 2024-03-23T14:58:49.679 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33822.log 2024-03-23T14:58:49.679 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.35834.log: /var/log/ceph/ceph-client.admin.36386.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35917.log 2024-03-23T14:58:49.679 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.33822.log: 88.9% 87.7% -- replaced with /var/log/ceph/ceph-client.admin.35834.log.gz 2024-03-23T14:58:49.679 INFO:teuthology.orchestra.run.smithi164.stderr: -- replaced with /var/log/ceph/ceph-client.admin.36386.log.gz 2024-03-23T14:58:49.679 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37412.log 2024-03-23T14:58:49.679 INFO:teuthology.orchestra.run.smithi164.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.33822.log.gz 2024-03-23T14:58:49.679 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-03-23T14:58:49.679 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.35917.log: /var/log/ceph/ceph-client.admin.37412.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37613.log 2024-03-23T14:58:49.679 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph.audit.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.37412.log.gz 88.3% -- replaced with /var/log/ceph/ceph-client.admin.35917.log.gz 2024-03-23T14:58:49.680 INFO:teuthology.orchestra.run.smithi164.stderr: 2024-03-23T14:58:49.680 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35677.log 2024-03-23T14:58:49.680 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-03-23T14:58:49.680 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.37613.log: /var/log/ceph/ceph-client.admin.35677.log: 89.0% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-03-23T14:58:49.680 INFO:teuthology.orchestra.run.smithi164.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.37613.log.gz 2024-03-23T14:58:49.680 INFO:teuthology.orchestra.run.smithi164.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.35677.log.gz 2024-03-23T14:58:49.680 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36871.log 2024-03-23T14:58:49.680 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33978.log 2024-03-23T14:58:49.681 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-mds.a.log: /var/log/ceph/ceph-client.admin.36871.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34032.log 2024-03-23T14:58:49.681 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.33978.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.36871.log.gz 2024-03-23T14:58:49.681 INFO:teuthology.orchestra.run.smithi164.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.33978.log.gz 2024-03-23T14:58:49.681 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36923.log 2024-03-23T14:58:49.681 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38275.log 2024-03-23T14:58:49.681 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.34032.log: /var/log/ceph/ceph-client.admin.36923.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.34032.log.gz 2024-03-23T14:58:49.681 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36181.log 2024-03-23T14:58:49.681 INFO:teuthology.orchestra.run.smithi164.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.36923.log.gz 2024-03-23T14:58:49.681 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.38275.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37587.log 2024-03-23T14:58:49.682 INFO:teuthology.orchestra.run.smithi164.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.38275.log.gz 2024-03-23T14:58:49.682 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.36181.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33298.log 2024-03-23T14:58:49.682 INFO:teuthology.orchestra.run.smithi164.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.36181.log.gz 2024-03-23T14:58:49.682 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.37587.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35808.log 2024-03-23T14:58:49.682 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.33298.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.37587.log.gz 2024-03-23T14:58:49.682 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37913.log 2024-03-23T14:58:49.682 INFO:teuthology.orchestra.run.smithi164.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.33298.log.gz 2024-03-23T14:58:49.682 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.35808.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.35808.log.gz 2024-03-23T14:58:49.683 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35971.log 2024-03-23T14:58:49.683 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35124.log 2024-03-23T14:58:49.683 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.37913.log: /var/log/ceph/ceph-client.admin.35971.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.37913.log.gz 2024-03-23T14:58:49.683 INFO:teuthology.orchestra.run.smithi164.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.35971.log.gz 2024-03-23T14:58:49.683 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37939.log 2024-03-23T14:58:49.683 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37334.log 2024-03-23T14:58:49.683 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.35124.log: /var/log/ceph/ceph-client.admin.37939.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.35124.log.gz 2024-03-23T14:58:49.683 INFO:teuthology.orchestra.run.smithi164.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.37939.log.gz 2024-03-23T14:58:49.684 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33586.log 2024-03-23T14:58:49.684 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.37334.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.37334.log.gz 2024-03-23T14:58:49.684 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38043.log 2024-03-23T14:58:49.684 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38121.log 2024-03-23T14:58:49.684 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.33586.log: /var/log/ceph/ceph-client.admin.38043.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.33586.log.gz 2024-03-23T14:58:49.684 INFO:teuthology.orchestra.run.smithi164.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.38043.log.gz 2024-03-23T14:58:49.684 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37783.log 2024-03-23T14:58:49.684 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34025.log 2024-03-23T14:58:49.684 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.38121.log: /var/log/ceph/ceph-client.admin.37783.log: 89.4% 88.5% -- replaced with /var/log/ceph/ceph-client.admin.38121.log.gz 2024-03-23T14:58:49.685 INFO:teuthology.orchestra.run.smithi164.stderr: -- replaced with /var/log/ceph/ceph-client.admin.37783.log.gz 2024-03-23T14:58:49.685 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34680.log 2024-03-23T14:58:49.685 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33400.log 2024-03-23T14:58:49.685 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.34025.log: /var/log/ceph/ceph-client.admin.34680.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.34025.log.gz 2024-03-23T14:58:49.685 INFO:teuthology.orchestra.run.smithi164.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.34680.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35254.log 2024-03-23T14:58:49.685 INFO:teuthology.orchestra.run.smithi164.stderr: 2024-03-23T14:58:49.685 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.33400.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.33400.log.gz 2024-03-23T14:58:49.685 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35228.log 2024-03-23T14:58:49.685 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37835.log 2024-03-23T14:58:49.686 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.35254.log: /var/log/ceph/ceph-client.admin.35228.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.35254.log.gz 2024-03-23T14:58:49.686 INFO:teuthology.orchestra.run.smithi164.stderr: 89.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34446.log 2024-03-23T14:58:49.686 INFO:teuthology.orchestra.run.smithi164.stderr: -- replaced with /var/log/ceph/ceph-client.admin.35228.log.gz 2024-03-23T14:58:49.686 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.37835.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.37835.log.gz 2024-03-23T14:58:49.686 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34628.log 2024-03-23T14:58:49.686 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34865.log 2024-03-23T14:58:49.686 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.34446.log: /var/log/ceph/ceph-client.admin.34628.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.34446.log.gz 2024-03-23T14:58:49.686 INFO:teuthology.orchestra.run.smithi164.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.34628.log.gz 2024-03-23T14:58:49.687 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35703.log 2024-03-23T14:58:49.687 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37282.log 2024-03-23T14:58:49.687 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.34865.log: /var/log/ceph/ceph-client.admin.35703.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.34865.log.gz 2024-03-23T14:58:49.687 INFO:teuthology.orchestra.run.smithi164.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.35703.log.gz 2024-03-23T14:58:49.687 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37641.log 2024-03-23T14:58:49.687 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.37029.log 2024-03-23T14:58:49.687 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.37282.log: /var/log/ceph/ceph-client.admin.37641.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.37641.log.gz 89.6% -- replaced with /var/log/ceph/ceph-client.admin.37282.log.gz 2024-03-23T14:58:49.687 INFO:teuthology.orchestra.run.smithi164.stderr: 2024-03-23T14:58:49.688 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.34996.log 2024-03-23T14:58:49.688 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37438.log 2024-03-23T14:58:49.688 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.0.37029.log: /var/log/ceph/ceph-client.0.34996.log: 89.5% -- replaced with /var/log/ceph/ceph-client.0.34996.log.gz 2024-03-23T14:58:49.688 INFO:teuthology.orchestra.run.smithi164.stderr: 91.9% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-03-23T14:58:49.688 INFO:teuthology.orchestra.run.smithi164.stderr: 93.2% -- replaced with /var/log/ceph/ceph-client.0.37029.log.gz 2024-03-23T14:58:49.688 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.22601.log 2024-03-23T14:58:49.688 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.37438.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36542.log 2024-03-23T14:58:49.688 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph.tmp-client.admin.22601.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.37438.log.gz 2024-03-23T14:58:49.688 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34576.log 2024-03-23T14:58:49.689 INFO:teuthology.orchestra.run.smithi164.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.22601.log.gz 2024-03-23T14:58:49.689 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35756.log 2024-03-23T14:58:49.689 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.36542.log: /var/log/ceph/ceph-client.admin.34576.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33952.log 2024-03-23T14:58:49.689 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.35756.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.36542.log.gz 2024-03-23T14:58:49.689 INFO:teuthology.orchestra.run.smithi164.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.34576.log.gz 2024-03-23T14:58:49.689 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 88.7% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.35756.log.gz 2024-03-23T14:58:49.689 INFO:teuthology.orchestra.run.smithi164.stderr: /var/log/ceph/ceph-client.admin.36262.log 2024-03-23T14:58:49.689 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34029.log 2024-03-23T14:58:49.690 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.33952.log: /var/log/ceph/ceph-client.admin.36262.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37550.log 2024-03-23T14:58:49.691 INFO:teuthology.orchestra.run.smithi173.stderr: 92.2% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-03-23T14:58:49.691 INFO:teuthology.orchestra.run.smithi173.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-03-23T14:58:49.692 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.34029.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.33952.log.gz 2024-03-23T14:58:49.693 INFO:teuthology.orchestra.run.smithi164.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.36262.log.gz 2024-03-23T14:58:49.693 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34498.log 2024-03-23T14:58:49.693 INFO:teuthology.orchestra.run.smithi164.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.34029.log.gz 2024-03-23T14:58:49.693 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35098.log 2024-03-23T14:58:49.693 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.37550.log: /var/log/ceph/ceph-client.admin.34498.log: 87.4%/var/log/ceph/ceph-client.admin.35098.log: -- replaced with /var/log/ceph/ceph-client.admin.37550.log.gz 2024-03-23T14:58:49.693 INFO:teuthology.orchestra.run.smithi164.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.34498.log.gz 2024-03-23T14:58:49.693 INFO:teuthology.orchestra.run.smithi164.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.35098.log.gz 2024-03-23T14:58:49.693 INFO:teuthology.orchestra.run.smithi164.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-03-23T14:58:49.705 INFO:teuthology.orchestra.run.smithi164.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-03-23T14:58:49.715 INFO:teuthology.orchestra.run.smithi173.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-03-23T14:58:49.719 INFO:teuthology.orchestra.run.smithi164.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-03-23T14:58:49.719 INFO:teuthology.orchestra.run.smithi173.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-03-23T14:58:49.725 INFO:teuthology.orchestra.run.smithi173.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-03-23T14:58:49.727 INFO:teuthology.orchestra.run.smithi173.stderr: 2024-03-23T14:58:49.727 INFO:teuthology.orchestra.run.smithi173.stderr:real 0m0.161s 2024-03-23T14:58:49.727 INFO:teuthology.orchestra.run.smithi173.stderr:user 0m0.619s 2024-03-23T14:58:49.727 INFO:teuthology.orchestra.run.smithi173.stderr:sys 0m0.057s 2024-03-23T14:58:49.733 INFO:teuthology.orchestra.run.smithi164.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-03-23T14:58:49.772 INFO:teuthology.orchestra.run.smithi164.stderr: 91.3% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-03-23T14:58:49.773 INFO:teuthology.orchestra.run.smithi164.stderr: 2024-03-23T14:58:49.773 INFO:teuthology.orchestra.run.smithi164.stderr:real 0m0.230s 2024-03-23T14:58:49.773 INFO:teuthology.orchestra.run.smithi164.stderr:user 0m0.867s 2024-03-23T14:58:49.773 INFO:teuthology.orchestra.run.smithi164.stderr:sys 0m0.095s 2024-03-23T14:58:49.774 INFO:tasks.ceph:Archiving logs... 2024-03-23T14:58:49.774 DEBUG:teuthology.misc:Transferring archived files from smithi164:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-03-23_09:56:53-fs-wip-batrick-testing-20240323.003144-squid-distro-default-smithi/7618407/remote/smithi164/log 2024-03-23T14:58:49.775 DEBUG:teuthology.orchestra.run.smithi164:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-23T14:58:49.776 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 has been restored 2024-03-23T14:58:50.041 DEBUG:teuthology.misc:Transferring archived files from smithi173:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-03-23_09:56:53-fs-wip-batrick-testing-20240323.003144-squid-distro-default-smithi/7618407/remote/smithi173/log 2024-03-23T14:58:50.042 DEBUG:teuthology.orchestra.run.smithi173:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-23T14:58:50.133 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-03-23T14:58:50.154 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-03-23T14:58:50.154 DEBUG:teuthology.orchestra.run.smithi164:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-23T14:58:50.157 DEBUG:teuthology.orchestra.run.smithi173:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-23T14:58:50.218 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-03-23T14:58:50.227 INFO:teuthology.task.clock:Checking final clock skew... 2024-03-23T14:58:50.228 DEBUG:teuthology.orchestra.run.smithi164:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-23T14:58:50.230 DEBUG:teuthology.orchestra.run.smithi173:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-23T14:58:50.241 INFO:teuthology.orchestra.run.smithi164.stdout: remote refid st t when poll reach delay offset jitter 2024-03-23T14:58:50.241 INFO:teuthology.orchestra.run.smithi164.stdout:============================================================================== 2024-03-23T14:58:50.241 INFO:teuthology.orchestra.run.smithi164.stdout:+hv01.front.sepi 44.4.53.2 2 u 60 64 37 0.056 -1.642 0.923 2024-03-23T14:58:50.241 INFO:teuthology.orchestra.run.smithi164.stdout:+hv02.front.sepi 63.231.80.2 3 u 63 64 37 0.075 +0.809 0.934 2024-03-23T14:58:50.241 INFO:teuthology.orchestra.run.smithi164.stdout:*hv03.front.sepi 74.6.168.72 3 u 64 64 37 0.084 +0.271 1.045 2024-03-23T14:58:50.242 INFO:teuthology.orchestra.run.smithi164.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-23T14:58:50.244 INFO:teuthology.orchestra.run.smithi173.stdout: remote refid st t when poll reach delay offset jitter 2024-03-23T14:58:50.244 INFO:teuthology.orchestra.run.smithi173.stdout:============================================================================== 2024-03-23T14:58:50.244 INFO:teuthology.orchestra.run.smithi173.stdout:*hv01.front.sepi 44.4.53.2 2 u 3 64 77 0.111 -0.180 1.539 2024-03-23T14:58:50.244 INFO:teuthology.orchestra.run.smithi173.stdout:+hv02.front.sepi 63.231.80.2 3 u 62 64 37 0.065 +2.393 1.940 2024-03-23T14:58:50.244 INFO:teuthology.orchestra.run.smithi173.stdout:+hv03.front.sepi 74.6.168.72 3 u 62 64 37 0.089 +2.451 1.969 2024-03-23T14:58:50.244 INFO:teuthology.orchestra.run.smithi173.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-23T14:58:50.245 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-03-23T14:58:50.255 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-03-23T14:58:50.255 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-03-23T14:58:50.265 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-03-23T14:58:50.274 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-03-23T14:58:50.283 INFO:teuthology.task.internal:Duration was 760.025948 seconds 2024-03-23T14:58:50.284 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-03-23T14:58:50.293 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-03-23T14:58:50.294 DEBUG:teuthology.orchestra.run.smithi164:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-23T14:58:50.296 DEBUG:teuthology.orchestra.run.smithi173:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-23T14:58:50.330 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-03-23T14:58:50.331 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi164.front.sepia.ceph.com 2024-03-23T14:58:50.331 DEBUG:teuthology.orchestra.run.smithi164:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-23T14:58:50.375 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi173.front.sepia.ceph.com 2024-03-23T14:58:50.376 DEBUG:teuthology.orchestra.run.smithi173:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-23T14:58:50.385 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-03-23T14:58:50.385 DEBUG:teuthology.orchestra.run.smithi164:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-23T14:58:50.420 DEBUG:teuthology.orchestra.run.smithi173:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-23T14:58:50.449 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-03-23T14:58:50.449 DEBUG:teuthology.orchestra.run.smithi164:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-23T14:58:50.488 DEBUG:teuthology.orchestra.run.smithi173:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-23T14:58:50.546 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-03-23T14:58:50.556 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-03-23T14:58:50.557 DEBUG:teuthology.orchestra.run.smithi164:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-23T14:58:50.568 DEBUG:teuthology.orchestra.run.smithi173:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-23T14:58:50.577 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-03-23T14:58:50.588 DEBUG:teuthology.orchestra.run.smithi164:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-23T14:58:50.613 DEBUG:teuthology.orchestra.run.smithi173:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-23T14:58:50.620 INFO:teuthology.orchestra.run.smithi164.stdout:kernel.core_pattern = core 2024-03-23T14:58:50.628 INFO:teuthology.orchestra.run.smithi173.stdout:kernel.core_pattern = core 2024-03-23T14:58:50.661 DEBUG:teuthology.orchestra.run.smithi164:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-23T14:58:50.678 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-23T14:58:50.679 DEBUG:teuthology.orchestra.run.smithi173:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-23T14:58:50.705 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-23T14:58:50.705 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-03-23T14:58:50.716 INFO:teuthology.task.internal:Transferring archived files... 2024-03-23T14:58:50.716 DEBUG:teuthology.misc:Transferring archived files from smithi164:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-03-23_09:56:53-fs-wip-batrick-testing-20240323.003144-squid-distro-default-smithi/7618407/remote/smithi164 2024-03-23T14:58:50.717 DEBUG:teuthology.orchestra.run.smithi164:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-23T14:58:50.743 DEBUG:teuthology.misc:Transferring archived files from smithi173:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-03-23_09:56:53-fs-wip-batrick-testing-20240323.003144-squid-distro-default-smithi/7618407/remote/smithi173 2024-03-23T14:58:50.744 DEBUG:teuthology.orchestra.run.smithi173:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-23T14:58:50.764 INFO:teuthology.task.internal:Removing archive directory... 2024-03-23T14:58:50.764 DEBUG:teuthology.orchestra.run.smithi164:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-23T14:58:50.785 DEBUG:teuthology.orchestra.run.smithi173:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-23T14:58:50.806 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-03-23T14:58:50.817 INFO:teuthology.task.internal:Not uploading archives. 2024-03-23T14:58:50.817 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-03-23T14:58:50.826 INFO:teuthology.task.internal:Tidying up after the test... 2024-03-23T14:58:50.826 DEBUG:teuthology.orchestra.run.smithi164:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-23T14:58:50.832 DEBUG:teuthology.orchestra.run.smithi173:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-23T14:58:50.836 INFO:teuthology.orchestra.run.smithi164.stdout: 658466 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 23 14:58 /home/ubuntu/cephtest 2024-03-23T14:58:50.854 INFO:teuthology.orchestra.run.smithi173.stdout: 658638 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 23 14:58 /home/ubuntu/cephtest 2024-03-23T14:58:50.856 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-03-23T14:58:50.867 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-03-23T14:58:50.888 INFO:teuthology.nuke:Checking targets against current locks 2024-03-23T14:58:50.912 DEBUG:teuthology.nuke:shortname: smithi164 2024-03-23T14:58:50.912 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-23T14:58:50.940 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi164.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-03-23_09:56:53-fs-wip-batrick-testing-20240323.003144-squid-distro-default-smithi/7618407', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-23 14:37:20.370993', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBG8pFKkZbSowF13bo8/baC7ODVq9vuxuXd51QOBTdiI3/0A2eM6xZaLeP6WOiDQ2yG3mqamUVAmozkvqaQGo+hw='} 2024-03-23T14:58:50.943 DEBUG:teuthology.nuke:shortname: smithi173 2024-03-23T14:58:50.944 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-23T14:58:50.967 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi173.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-03-23_09:56:53-fs-wip-batrick-testing-20240323.003144-squid-distro-default-smithi/7618407', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-23 14:37:20.369037', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDYERFStNwl9aEPr/ypxY64Yl00iN9HbehUIiBGulOVP8YqeAQM6Ms8/NjAJgIZLbg0ZS+tGZV+5TDftGxwMUgA='} 2024-03-23T14:58:50.996 INFO:teuthology.orchestra.console.smithi164:Power off 2024-03-23T14:58:50.997 DEBUG:teuthology.orchestra.console.smithi164:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-23T14:58:51.019 DEBUG:teuthology.orchestra.console.smithi164:power off output: Chassis Power Control: Down/Off 2024-03-23T14:58:51.020 DEBUG:teuthology.orchestra.console.smithi164:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-23T14:58:51.039 INFO:teuthology.orchestra.console.smithi173:Power off 2024-03-23T14:58:51.040 DEBUG:teuthology.orchestra.console.smithi173:pexpect command: ipmitool -H smithi173.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-23T14:58:51.050 DEBUG:teuthology.orchestra.console.smithi164:check power output: Chassis Power is on 2024-03-23T14:58:51.064 DEBUG:teuthology.orchestra.console.smithi173:power off output: Chassis Power Control: Down/Off 2024-03-23T14:58:51.064 DEBUG:teuthology.orchestra.console.smithi173:pexpect command: ipmitool -H smithi173.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-23T14:58:51.089 DEBUG:teuthology.orchestra.console.smithi173:check power output: Chassis Power is on 2024-03-23T14:58:55.051 DEBUG:teuthology.orchestra.console.smithi164:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-23T14:58:55.090 DEBUG:teuthology.orchestra.console.smithi173:pexpect command: ipmitool -H smithi173.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-23T14:58:55.162 DEBUG:teuthology.orchestra.console.smithi164:check power output: Chassis Power is on 2024-03-23T14:58:55.201 DEBUG:teuthology.orchestra.console.smithi173:check power output: Chassis Power is on 2024-03-23T14:58:59.166 DEBUG:teuthology.orchestra.console.smithi164:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-23T14:58:59.201 DEBUG:teuthology.orchestra.console.smithi173:pexpect command: ipmitool -H smithi173.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-23T14:58:59.277 DEBUG:teuthology.orchestra.console.smithi164:check power output: Chassis Power is off 2024-03-23T14:58:59.311 DEBUG:teuthology.orchestra.console.smithi173:check power output: Chassis Power is on 2024-03-23T14:58:59.378 INFO:teuthology.orchestra.console.smithi164:Power off completed 2024-03-23T14:59:03.313 DEBUG:teuthology.orchestra.console.smithi173:pexpect command: ipmitool -H smithi173.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-23T14:59:03.425 DEBUG:teuthology.orchestra.console.smithi173:check power output: Chassis Power is off 2024-03-23T14:59:03.525 INFO:teuthology.orchestra.console.smithi173:Power off completed 2024-03-23T14:59:03.630 INFO:teuthology.run:Summary data: description: fs/shell/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1-mds-1-client-coloc conf/{client mds mgr mon osd} distro/ubuntu_latest mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/cephfs-shell} duration: 760.0259482860565 failure_reason: 'Test failure: test_cd_with_args (tasks.cephfs.test_cephfs_shell.TestCD)' flavor: default owner: scheduled_pdonnell@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=a5867e8f3df54845912472e1ea6e5857 status: fail success: false 2024-03-23T14:59:03.631 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-23T14:59:03.712 INFO:teuthology.run:FAIL